This week we attended a fantastic event run by City University on the debates around peer review, is peer review broken? Can we fix it? Chaired by Connie St Louis, director for City University Science Journalism MA and award winning journalist, the panel consisted of an epic line up – Tom Reller, VP & Head of Global Relations for Elsevier; Richard Van Noorden, Senior Reporter at Nature; Tiago Villanueva, Editorial Registrar at the BMJ; Maria Kowalczuk, Deputy Biology Editor at BioMed Central; Peter Ayton, Associate Dean of Research at City University and Nikolaus Kriegeskorte, Principal Investigator at the Medical Research Council, Cognition and Brain Science Unit.
The debate kicked off with each panellist talking about the problems that peer review faces in its current state. Niko talked about the wide range of problems that do exist that are not necessarily going to have a simple solution, one issue being fraud. By expanding the evaluation pre publication process, this is not going to solve this problem and there was an overall agreement across the panel that fraud is not going to be able to be picked up by peer review – that is not its purpose. The assessment of science is a difficult problem and one which has been around in the form of peer review since well before the internet. As academics and publishers it is important to remember that the younger generation coming in now will be able to imagine a different method of assessing science and evaluating the current peer review process; this is an important development for the continued success of peer review to go forward. The real question is; how will academics and publishers want to organise the collective process in which they evaluate their science in the future? This is the challenge and it will take a strong collaborative approach from all parties in order to move this forward.
It seems that there is a constant conflict between academics and publishers around peer review – Niko went on to say that the publishing industry has done science a great service, but they have also made an awful lot of money out of that. By reinventing peer review, most of the new ideas amount to scientists using the internet to self organise papers and this will in turn cut publishers out as administrators of peer review. This is a huge conflict and one that needs to be addressed in any reassessment of peer review moving forwards.
Tom gave his perspective from Elsevier’s viewpoint, speaking about how publishers will continue to thrive and do well as long as they continue to administer the peer review systems that scientists want. There is a huge amount of innovation in how peer review is being done differently but that doesn’t mean that they have to start completely from scratch. He described an excellent analogy for peer review when he said that if you think of peer review as a rubber band and you keep stretching it, the more you stretch it, the easier it will be for bad things to filter through – we often hear about the bad things that have come from peer review, the fraud and increasing retractions for example, but it’s important to remember there are a lot of positive things as well.
Tiago explained the process of peer review from the very start, when a batch of papers arrives on his desk. From there, he sends it out to 6 peer reviewers which he finds on a database, 90% of the time Tiago doesn’t know who these people are and often gets poor reviews back. This means that often they end up using reviewers who give good reviews but does this then create a bias? With traditional peer review, the author and the reviewers don’t know each others identities, but with open peer review they do, which can create conflict on both sides.
The increasing retraction of papers was another element of the debate that came up, maybe because of more fraud, more competition and a higher pressure to get published. The general consensus from the panel was that this is a good thing; it means the papers that are being published are of a higher quality. Peer review cannot be expected to detect fraudulent papers, 2-4 people reviewing that paper cannot be tasked with the decision on whether that data is right or wrong, the scientific community is based on trust and that is how the peer review system has always operated. Tom at Elsevier confirmed that trust in the industry is absolutely critical, telling someone you don’t trust what they submit leads to a need for policing which then stretches the rubber band even further making things problematic. The internet is also hugely powerful, science is only just starting to see the power that the internet has and how it can help them in allowing for lots of information to be changed. We need this to allow post publication review and discussion but we also need filtration and for people to get the chance to improve a manuscript, both before and after publication. Niko favours open peer review, he said reviews should be published, reviews should be signed and it should be possible for everyone to contribute. There will always be the problem where people can’t see where a paper is incorrect (because it hasn’t been subject to pre-publication) but academics should come out with papers publicly, stand by their judgements and then organise a collective, cognitive process.
So, is peer review broken? Absolutely not, the notion of it being broken means it wouldn’t work or serve it’s purpose – it may be chipped and stretched and in need of some tlc but the tools are here. We have the power of the internet, the incoming younger generation mixed with the generation who have been using the current process for their whole lifetimes can come together to create an innovative, collaborative process for the assessment of scientific research.
Here at Inspired Selection, we are passionate about the publishing industry; we talk about publishing, read about publishing and attend all major publishing events like the one you’ve just read about. We would love to meet you at events so do feel free to come up and introduce yourselves! This year, we are sponsoring London Book Fair and invite you to take a look at our People Development seminar stream that is running throughout the 3 days. If you’re interested in opportunities within publishing do keep in touch and register for our Vacancy Update Service as well as keeping up to date with us on Twitter
Inspired Selection 