Open Peer Review

This blog has been written by Dr. ir. Sabine Vreeburg

 

Introduction

What is open peer review? What are the advantages and disadvantages and how can you put it into practice? Read on to find out more about open peer review, an emerging Open Science practice that is gaining momentum.  

Wageningen University & Research shares this blog to add to the upcoming Peer Review Week (19-23 September 2022). A week in which the scientific community celebrates the essential role that peer review plays in maintaining research quality.  

What is open peer review? 

Open peer review is an umbrella term for various alternative review methods that seek to make the traditional single or double-blind peer review process more transparent and accountable. Ross-Hellauer (2017)1 concluded that a peer review process is open when it includes one or more of the following elements: 

• Open identities: Authors and reviewers know each other’s identity. 
• Open reports: Review reports are published with the article, instead of keeping them confidential. 
• Open participation: Instead of, or in addition to invited reviewers, the wider community can add to the review. 
• Open interaction: Author(s) and reviewers, and/or reviewers amongst each other, are allowed and encouraged to interact and discuss their views. 

• Open pre-review manuscripts: Manuscripts are published before formal peer review, e.g. via pre-print servers like arXiv or BioRxiv.  
• Open final-version commenting (also known as “post-publication peer review”): When review or commenting on the “version of record” of publication is enabled. A good example is the post publication peer review platform Pubpeer.  
• Open platforms (“decoupled review” or “journal independent review”): Manuscripts are reviewed by reviewers not associated with the publisher. An examples of decoupled or journal independent review is Peer Community in. 

Growth of open peer review 

The number of open peer review reports has seen a strong growth in recent years. According to Wolfram et al. (2020)2 600 journals offered open peer review in 2019. Waltman and Van Eck (2022) calculated that the number of open peer reviews in Crossref increased from less than 10,000 in 2018 to over 60,000 in 2020 and 2021. As many publishers don’t register their peer review reports for a DOI in Crossref, these numbers only give an indication of the total number of open peer reviews.  

Reasons to get involved with open peer review 

Open peer review has advantages, both for you and for the community. Source: FOSTER open science project 

For your research: 

• Opening your research to wider review from an early stage can improve the quality of your methodology and subsequently the validity and credibility of your findings. Research suggests that the quality of open reviews tend to be at least as good as ‘traditional’ reviews3.  
• If you write an open peer review, it will be a new citable research output. Mentions and citations can reward your effort. 

• Some publishers and platforms give authors the opportunity to discuss and also to refute some of the reviewer’s comments and make both comments and response to these comments openly available. Others allow discussions between authors and reviewers behind doors, but publish the reviews after the completion of the review process.  

For your skills: 

• Seeing how others present their reviews can often help you to write more diplomatic and constructive reviews yourself. 

For the community: 

• Results are made available more quickly and can therefore be built upon more efficiently. 

• By engaging in open peer review, you are setting an example for others to follow – particularly your students. 
• You help to enrich science, as it gives readers the benefit of additional expert opinions.  

Are there concerns about open peer review?  

Open peer review is by no means a silver bullet. A major concern is that revealing reviewer identity could lead to negative career consequences for the writer of a critical review, especially for junior researchers. Also, open peer review does not eradicate all bias by itself. However, open peer review does make such occurrences more open and can in turn lead to reputation damage for a reviewer who misbehaves. 

There are also concerns about the effect that open peer review could have on work pressure. Publishing review commentaries could encourage reviewers to spend more time on their reviews to perfect the spelling and writing. This does not improve the science, yet could make the review more stressful and demanding. However, when the review is published your efforts can be rewarded by mentions and citations.   

How to put open peer review into practice ?

As an author: 

• You can publish a preprint and ask the community to review it or submit your manuscript to a specific platform. More information about preprints can be found in the guide ‘A practical guide to preprints: accelerating scholarly communication’. Please note that most, but not all, journals allow submission of preprints – please check the policy of the publisher before publishing your preprint.  

• You can submit your manuscript to a journal that supports open peer review methods. In the table below you will find a non-exhaustive list: 

---

	Journals & Publishers that support open peer review	Preprint Servers & Publishing Platforms
Life Sciences	– eLIFE  – Bio Med Central  – BMJ   – GIGA science    – Copernicus Publications	– BioRxiv   – ASAPbio  – F1000Research
Social Sciences	– SAGE open  – Wiley	– SocArXiv  – F1000Research
Arts & Humanities	– SAGE open  – Wiley  – digitalculturebooks	– F1000Research

---

 

As a reviewer: 

• You can, when deciding to agree or disagree to reviewing a paper, take into account whether the peer review process is open.  
• You can always sign your review, regardless of the journal’s policy.  
• When you have reviewed a manuscript, you can add it to your Publons profile (note: Publons is moving to Web of Science as of August 2022. See this blog for more information about Publons). On your Publons profile you can decide to hide or show the content of the review after the manuscript has been published).  

• When asked to review an article that is available as a preprint, you can publish your review alongside the preprint as encouraged by the ASAPbio “Publish your review” initiative.  
• You can join the public reviewing process by commenting on preprint articles that are published on preprint servers.  

Experiences with open peer review 

Hilje Doekes (WUR, Laboratory of Genetics) and her colleagues recently published a research article in the journal eLife. In this journal only manuscripts that are first published as a preprint can be submitted and the reviews and author responses are published online. Also, there is open interaction between the reviewers. They discuss their reviews with each other and the editor in a consultation session, which results in one combined review in the decision letter. ELife does not automatically publish the names of the reviewers, although reviewers can opt to publish their names. In Hilje’s case, the identity of one reviewer was known. Finally, in theory comments can be made on the final version on the eLife website, which had not been done when writing this blog. In all, the peer review process ticked five of the seven open peer review elements. 

Hilje says: “Although I do not think that I acted differently than I would have done if there was a traditional peer review process, our experience with the open peer review process of eLife was very positive. The review we received was thorough and constructive. I most liked the fact that the reviewers and editor had discussed their reviews before sending them to us, such that no overlapping or contradicting comments were made.” 

From a publisher’s perspective Prof. Jim Reekers, Professor Emeritus at the University of Amsterdam shares his experiences as an Editor-in-Chief at BMC, an Open Access publisher in his blog. He writes: “Open peer review has never been an issue in recruiting reviewers. (…) Authors do not seem to have any problem with open peer review, as the numbers of submissions is constantly increasing. We have noticed that the reviews are of a high level, to the point, helpful and supportive. The anonymous judge is transformed into a friendly and helpful mentor.” 

More information 

• Course: “Reviewing a Scientific Manuscript”, coordinated by the Wageningen Graduate School 
• A free introductory course to open peer review can be found here
• A template to help you review in a consistent and structured way provided by Publons can be found here  

References

1. Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6, 588. https://doi.org/10.12688/f1000research.11369.2 
2. Wolfram, D., Wang, P., Hembree, A. et al. (2010). Open peer review: promoting transparency in Open Science. Scientometrics, 125, 1033-1051. https://doi.org/10.1007/s11192-020-03488-4
3. Van Rooyen, S., Delamothe, T., & Evans, S.J. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ, 341,  c5729. https://doi.org/10.1136/bmj.c5729