Can peer-review be saved by private businesses? by Calum Kennedy

The peer-review process is meant to be a guardian of scientific integrity. That is not to say that it maintains the integrity of individual scientists, but that it attempts to ensure that results of these scientists’ intellectual endeavours are presented and discussed in a way that conforms to the ideals of the scientific community as a whole. Ideally, this means that the papers published in journals contribute positively towards scientific debates rather than being mistakes, distractions, or unhelpful repetition.

However, the state of the peer-review process in the publishing of scientific papers is by no means perfect and there are frequent worries over its ability to protect from bias, its ability to guard against fraud, and its contribution to delays in publication. Although the current information age has brought with it the ability for things like online pre-printing, where researchers can share an editable, non-peer reviewed version of their work, the high regard for peer-review makes it unlikely that such open publication of scientific papers will supersede standard journals. This leaves peer-reviewed journals at the centre of the science economy.

The persistent use of quantitative metrics in the assessment of the impact of published scientific work in tandem with capitalist nature of modern science funding has fed the ‘publish or perish’ mind-set of both scientists and journals. As grants upon which researchers are dependant are partially decided by the metric analysis of their previous work, scientists are encouraged to publish as much as possible, even to the extent of dividing papers into piecemeal parts to be published separately over time. This succeeds in both increasing their perceived productivity and reduces the likelihood of investing resources into a narrow line of research only to be scooped.

The problem that this has created is that the change in publishing standards has not been matched by the change in peer-review. The increased volume of publishing combined with the incentive of metrics for potential reviewers to spend more time producing their own work than reviewing others has placed the majority of the strain of reviewing manuscripts on altruistic volunteers. Additionally, the attention of journals on their own impact factors (again produced via metrics) means that editors have to sift through the increasing number of publications and select papers based on perceived importance, novelty, and the ability to attract readers alongside the actual quality of the scientific work.

This increasingly stringent selection process among the more established journals leaves a large number of researchers struggling to publish. This opportunity has been identified by so called ‘predatory journals’. These journals masquerade as open access peer reviewed publications that promise to get manuscripts peer reviewed and published for a fee (just like any other journal). However, often these predatory journals simply publish anything that is submitted to them, falsely claiming that it has been peer-reviewed. Researchers from less prestigious institutions, or who are less likely to increase the impact factor of journals are more often rejected by established journals and are thus more likely to turn to other means of publication.

Even if their manuscripts are accepted, the peer review process takes too long. With everything going smoothly it can still take more than 180 days from submission to publication. This can be too long for many researchers who either fear being scooped, or can’t afford to wait half a year to get their first paper published. It can also be too long from the point of view of the wider community as well. In medical sciences for example, the research and peer-review of findings that could help people would better serve the public if they were published now, rather than in 180 days. Medical journals have been using fast track peer review for quite some time and can allow for time sensitive papers to go from submission to online publication in four weeks (six weeks to print).

As with most things, there have been attempts to commoditise such expedited peer review. This has mostly been a tactic used by predatory journals to attract more submissions from unsuspecting researchers, but it has also come under consideration by more prestigious journals. One of the most recent such incidents is that of the Nature Publishing Group’s journal ‘Scientific Reports’ who had an editor resign over the announcement in March 2012 that the journal would be offering an option to pay for expedited peer review by outsourcing it to a third party. The editor, Mark Maslin, objected to the idea of a two tiered system and his objections were followed by a letter from members of the editorial board of Scientific Reports which threatened the resignation of the 150 signatories unless the situation was resolved or justified to their satisfaction. At the end of April NPG announced the cancellation of the fast-track-trial.

 

While I can see the concerns of the editors, that a two-tiered system may encourage further discrimination of researchers based on their ability to attract the funding required to pay for the option, that the         use of a for-profit third party organisation casts doubt on the adherence to the ideals of peer-review, and that the act of paying some reviewers and not others would disrupt the entire process. The proposed model for this fast-track process does pose some interesting questions.

A submitted manuscript could be accompanied by a $750 payment and be forwarded to Rubriq, part of the Research Square Company based in North Carolina. Rubriq guarantees the author a review within 3 weeks by using their staff of 100 expert employees to develop a network of reviewers around the world who the manuscript would be sent to along with a payment of $100 per review. These reviews would be completed by a standardised scorecard by two reviewers per manuscript. These reviewers would thus not necessarily have any particular connection to the journal that the manuscript was initially sent to and would not necessarily be known by the journal, editor, or author.

The question of paying reviewers for their services at all is an interesting place to start. As has already been mentioned, the impact of metrics on the standards and practices of researchers with regards to manuscript submission has placed allot of strain on the traditional peer-review process. It seems then that there may be benefits in rewarding or incentivising more experts to participate to a higher degree in the process. Care must be taken however as the current status-quo is maintained by the idea that the careers of referees are connected to the reputation of the journal as it is a journal in which they themselves publish.

One the other hand, another impact of such a paid system as Rubriq, is that it removes the peer review process from the journals entirely (apart from the final decision that the editor takes upon receiving the reviews from the company). This entirely overrides any impact that the payment of reviewers would have on the status-quo regarding the interaction of researchers, reviewers and journals, as under this paradigm the reviewer does not even need to know the journal to which the manuscript is being submitted. Further to this however is the impact this could have on the standardisation of the peer-review process. There are many different variations of the peer-review process depending on the journal in question. This means that different researchers are getting different responses to their submitted manuscripts. It also makes it difficult to implement any proposed changes to the system as that would require the collaboration of different, often competing, journals.

The current state of peer-review does not prevent bias, fraud, or conflicts of interest from impacting either the individual, or science as an ideal. While fraud is notoriously hard to detect if it is done well, and purposeful fraud is rare and so does not warrant overblown safeguards, both bias and conflicts can likely be alleviated through the assignment of the peer review process to independent organisations like Rubriq. Conflicts, of effort, conscience, and interest, from the reviewer can all be affected by the restructuring of peer review under a paid, double blind, collective, and professional service. The free time dependency that implies conflict of effort can be alleviated through the streamlined and less individually intensive process that such a system offers, rather than the current system which puts too much strain on altruistic individuals while others concentrate on improving their metrics. Both the internal and external bias that constitutes conflicts of conscience and interest respectively can be alleviated due to such a systems higher capability to track reviews and identify misconduct.

 

It seems then that the best outcome would be for all journals to outsource the peer-review process to a company like Rubriq. However, In order to get to such a situation there would likely be a need to transition through a two tired process where researchers were charged extra for the option to expedite. The question that needs to be asked if this is the case is whether we are interested in protecting the individual scientist, where the best possible outcome is for their interaction with peer review to be fair and allow them to grow as a researcher, or are we interested in protecting science in general, where the best possible outcome would be to encourage research of a higher calibre to proceed faster whilst maintaining quality? Editors, like those from ‘scientific reports’ have to decide then whether they think that the ends justify the means, whether they think that the current system can protect the individual from predatory journals and from the threat of bias, or whether a two tiered system would irreparably damage the ideals of the scientific community.

Whilst it does seem like the implementation of paid expedited peer review on a large scale would be detrimental to the fairness that is already under threat from institutional bias and economically entrenched disadvantages between scientists in different countries, it is also possible that fast-tracking well-funded science may be good for science in general. Labs that have enough money to pay for expedited peer review are likely involved in high quality, front line, research and would thus have a better chance of impacting the state of science. So the question remains, do we want peer review to protect the individual, or to protect science?

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s