Can altmetrics be used as a basis for science policy?

altmetrics15
The 2015 Altmetrics Workshop
Amsterdam, 9 October 2015

Paul Groth
Peter Van Den Besselaar

Abstract

There has been increasing debate about the role of metrics within science policy [4,5], including whether altmetrics are useful at all. This debate has led to the reaffirmation of peer review as the primary mechanism by which to formulate science policy, and e.g. organize evaluation processes, and selection processes for grants and positions, in an evidenced based fashion as expressed by the recent Metric Tide report. This discounts some of the issues that manifest in the sole use of peer review.

Many of the critique on use of metrics is in our view based on the nature of the indicators, which are strongly data driven (e.g., Web of Science). The question we pose is whether altmetrics can provide a means to in fact provide answers to science policy questions? To do so, we believe that we need to recontextualize our view of altmetrics to a use case driven and theory grounded approach. Currently, most work in altmetrics has been about validating different indicators by comparing to well known existing measures. For example, whether mentions in tweets correlate to citation measures.

We propose the following seven-fold approach to develop a new “altmetric”. This approach tries to shift the role of altmetrics from one in which the data is central (i.e. one has a particular metric, what can it be used for?) to one in which the policy question is central. It makes the assumption that the data needed can be obtained. While we agree that data integration and cleaning remain proble, our assertion is that it is there and that we as a community need to look beyond what metrics are easy to obtain.

Furthermore, we conjecture that this approach will tackle some of the issues that critics of the use of metrics for science policy and research management have. In particular, by focusing on mechanisms one can ground measures in explanations of the particular social system in question. This can help provide the kind of context that is sometimes lacked when using metrics.

In conclusion, we hope to have a discussion at the workshop about this view of how to incorporate altmetrics into supporting science policy and research evaluation and management.

Supplementary materials

PDF