SPRU - Science Policy Research Unit

The Metrics Tide is Rising – Results of HEFCE Review

metric tideChaired by Professor James Wilsdon, the Independent Review of the Role of Metrics in Research Assessment and Management for HEFCE, has published its findings in a report, The Metric Tide

Based on 15 months of evidence-gathering, consultation and comprehensive data analysis, the final report concludes that metrics cannot replace the peer review process in the next REF and that it is not currently feasible to assess research outputs or impacts in the REF using quantitative indicators alone.

The report carefully considers the evidence provided in response to a call that uncovered considerable scepticism among researchers, universities, representative bodies and learned societies about the broader use of metrics in research assessment and management. Concerns include the ‘gaming’ of particular indicators, uneven coverage across individual disciplines, and effects on equality and diversity across the research system.

SPRU’s Professor Ben Martin, Professor Paul Nightingale and Dr Ismael Rafols submitted a collaborative response to the metrics review call for evidence. The response argued for a cautionary approach to the use of metrics in research assessment, which built on Professor Ben Martin’s seminal work on assessments of research performance, as well as Ismael Rafols’s work on novel and more multidimensional science metrics.

James Wilsdon said “SPRU was of course among the organisations to submit evidence, and I'd like to acknowledge a particular debt of gratitude to Ismael Rafols, Andy Stirling, Ben Martin, Michael Hopkins and Paul Nightingale, all of whom provided hugely valuable advice and input at various points. I hope the end result does justice to these contributions, and builds on a fine and longstanding tradition of SPRU and Sussex contributions to these debates”.

‘The Metric Tide’, takes a closer look at the potential uses and limitations of research metrics and indicators, exploring the use of metrics within institutions and across disciplines. Within the report are 20 specific recommendations for further work and action by stakeholders across the UK research system.

The Independent Review of the Role of Metrics in Research Assessment and Management was set up in April 2014 at the request of the Rt Hon David Willetts, then the UK minister of universities and science. 

To continue the debate, the metrics review team has created a Twitter handle, @ResMetrics, short for responsible metrics.

Full details of the review can be found on the HEFCE website: www.hefce.ac.uk/rsrch/metrics 

See also

The Times Higher - Metrics: how to handle them responsibly

Nature (editorial) - A numbers game

Nature - We need a measured approach to metrics

The Guardian Science Blog - The metric tide: an agenda for responsible indicators in research

Research Professional - Tensions bubble below metrics consensus

Research Professional - Tides wait for no man

Research Professional - Research assessment needs a local dimension (Ismael Rafols)