Open access for scholarly publications, part 5 – A national forum on research evaluation

The National Library of Sweden has held the appropriation directive from the Swedish Government to produce recommendations for the change to an open access scholarly publication system. These recommendations are presented by Henrik Schmidt, librarian here at KIB, in a series of blog posts.

The government has in propositions and commissions emphasized the importance of transitioning to an open access scholarly publication system. This is in order to promote both high research quality and continued social development. As part of the transition the current way to evaluate research, as well as the merit- and allocation systems, has to be reviewed and revised. The Strut report (only in Swedish) argues that today’s system stymies the transition to an open publishing system. 

The evaluation of research results – that is to identify, understand and reward good research – is a highly complex activity. It cannot be reduced to simple indicators. Research evaluation was discussed by the delegates at the annual meeting of the American Society of Cell Biology in December 2012. This was the beginning of the DORA declaration which is now widely known within scientific circles. DORA formulates a variety of recommendations describing what one should (and should not) take into account when research results are evaluated.

The DORA recommendations are intended for various actors, including research funding agencies, research institutes and universities, scientific publishers, individual researchers, and others. In addition to specific advice to different target groups, an overall recommendation is given to all actors. It discourages using journal indicators to evaluate individual research articles, individual researchers’ contribution to the field, or as the basis for employment, promotion or grant allocation. Among those who have signed the DORA declaration are Swedish universities and Swedish research funding agencies (Forte and Formas), along with a large group of researchers (among them about 40 from Karolinska Institutet).

One reason behind the so called Plan S (formulated by a number of large European research funding agencies) deals with the research evaluation: “We also understand that researchers may be driven to do so by a misdirected reward system which puts emphasis on the wrong indicators (e.g. journal impact factor). We therefore commit to fundamentally revise the incentive and reward system of science, using the San Francisco Declaration on Research Assessment (DORA) as a starting point.”

Nonetheless, few would reject the value of bibliometric analyses. They have become an integrated (and enlightening) part of the follow up on research results and the description of how, by whom and about what research is conducted. At the same time, bibliometry is often used uncritically, and without insight into its limitations. This is what a number of initiated researchers concluded in a Nature article from 2015. The article’s main task was presenting the so called Leiden manifesto, a list consisting of ten points summarising and demonstrating how quantitative research evaluation should be used and how one can avoid pitfalls. Read more.

In order to steer us toward an open access scholarly publication system, we need a coherent approach to research evaluation where open access should be included as one aspect. The approach should be established in continuous dialogue between different stakeholders, such as the boards of educational institutions, with support from its libraries, the funding agencies, and the research community, as represented by academic associations, and researchers in different stages of their careers. This in addition to other relevant authorities who have a role in the scientific system.

The fifth recommendation in the report presented by the National Library of Sweden (KB), in the spring of 2019, is to establish of a national forum for research evaluation.

 

Recommendation 5 reads as follows: That the Department of Education establishes a national forum for responsible research evaluation, where open access to scientific publications and open science are included as parameters.

Find out more about the description of and argument in favour of this recommendation in the report: Meriterings- och medelstilldelningssystemen i relation till incitament för öppen tillgång (only in Swedish).

All blog posts in this series

Henrik Schmidt (on leave)

Librarian engaged in research support in various forms. Teaches doctoral students and researchers in areas related to literature search, publishing strategies and publication analysis.