|
WHAT'S THIS?
|
PRESS RELEASES << Peer Review Survey 2009: Preliminary Findings
|
Peer Review Survey 2009: Preliminary Findings
Published 8 September 2009
Should peer review detect fraud and misconduct? What does it do for
science and what does the scientific community want it to do? Will it
illuminate good ideas or shut them down? Should reviewers remain anonymous?
On 8th September 2009 the preliminary findings of one of
the largest ever international surveys of authors and reviewers, the Peer
Review Survey 2009 1, were released. The findings were presented in the
session "Science Fact or Science Fiction: Should Peer Review Stop
Plagiarism, Bias or Fraud?" at the British Science Festival, where Tracey
Brown of Science About Science, David Adam of The Guardian and Peter
Hayward of Lancet Infectious Diseases debated the challenges of
publishing research.
Peer review is fundamental to integration of new research findings. It
allows other researchers to analyse findings and society at large to weigh
up research claims. It results in 1.3 million3 learned articles published
every year, and it is growing rapidly with the expansion of the global
research community. With that growth come new concerns - about getting the
next generation of researchers to review in sufficient numbers, about
maintaining the system's integrity and whether it can be truly globalised;
and also new ideas - about alternative quality measures, technologies to
prevent plagiarism, rewarding reviewers and training them.
Sense About Science has promoted understanding of peer review to help
people to work out whether research claims have been independently
scrutinised. But with all the proposed changes and expansion in research
publication, what do researchers think about peer review and its future? To
find out, Sense About Science developed the Peer Review Survey 2009, in
consultation with editors and publishers and administered with a grant from
Elsevier; the survey included some questions from the Peer Review Survey
20074 for comparison, and new questions about future improvements, public
awareness and pressures on the system.
Tracey Brown, Managing Director: "The 2007 survey had raised some of the
issues. We sought to broaden that, particularly to find out whether the
demand for all this free, independent scrutiny from the research community
is sustainable, and what the future of quality control is likely to be. It's
a matter of public as well as scientific interest."
Preliminary findings include:
Playing an active role in the community is top of reasons to review:
90% say they review because they believe they are playing an active role in
the community; only 16% say that increasing their chances of having future
papers accepted is a reason to review.
Researchers want to improve, not replace peer review:
- 84% believe that without peer review there would be no control in
scientific communication, but only a third (32%) think it is the best that
can be achieved; 20% of researchers believe that peer review is
unsustainable because of too few willing reviewers.
- 91% say that their last paper was improved through peer review; the
discussion was the biggest area of improvement.
- 73% of reviewers (a sub-group) say that technological advances have
made it easier to do a thorough job than 5 years ago. Whilst 86% enjoy
reviewing, 56% say there is a lack of guidance on how to review; 68% think
formal training would help. On average, reviewers turn down two papers a
year.
- Just 15% of respondents felt that 'formal' peer review could be
replaced by usage statistics.
- 61% of reviewers have rejected an invitation to review an article in
the last year, citing lack of expertise as the main reason - this suggests
that journals could better identify suitable reviewers.
High expectations:
- 79% or more of researchers think that peer review should identify the best
papers, determine their originality and importance, improve those papers
and, though lower scoring, also determine whether research is plagiarised or
fraudulent.
- While 43% of respondents thought peer review was too slow, 65% of authors
(a further sub-group) reported that they had received a decision on their
most recent paper within 3 months.
Reviewers want anonymity:
58% would be less likely to review if their signed report was published. 76%
favour the double blind system where just the editor knows who the reviewers
are.
Understanding of peer review:
Researchers agree that peer review is well understood by the scientific
community but just 30% believe the public understands the term.
Papers aren't recognising previous work:
81% think peer review should ensure previous research is acknowledged; 54%
think it currently does. This reflects current concerns in the research
community5.
Detecting plagiarism and fraud might be a noble aim but is not
practical:
A majority think peer review should detect plagiarism (81%) or fraud (79%)
but fewer (38% and 33%) think it is capable of this.
Reviewers divided over incentives:
Just over half of reviewers think receiving a payment in kind (e.g.
subscription) would make them more likely to review; 41% wanted payment for
reviewing, but this drops to just 2.5% if the author had to cover the cost.
Acknowledgement in the journal is the most popular option.
Comment from Editors:
Robert Campbell, Wiley-Blackwell & Chairman, Publishing Research
Consortium: "Most researchers give up time to review papers for no charge.
The whole scholarly communication system is dependent on this. Why do they
do it? This study helps us to answer the question. And I take heart in the
finding that 85% of the around 4000 respondents quite simply do it because
they enjoy being able to improve papers. "
Sir Brian Heap, Vice Chairman, European Academies Science Advisory
Council: "These latest insights from researchers show why the peer review
process continues to be so important - and enjoyable!"
Adrian Mulligan, Associate Director of Research and Academic Relations,
Elsevier: "I'm impressed by the vast amount of responses to the study - it
tells us just how engaged the scientific community is with peer review. Not
only do they greatly value it, but they have a strong desire to enhance it.
Elsevier in partnership with editors, plays an active role in developing
peer review, and we are moving forward with initiatives such as Crosscheck,
a pan-publisher plagiarism detection tool. We are keen to look at other ways
to respond to the views raised in this survey."
Irene Hames, Managing Editor The Plant Journal, Author 'Peer Review and
Manuscript Management in Scientific Journals: guidelines for good practice':
"It's very important to know what authors and reviewers actually think about
the current status of peer review. Too many commentators make broad
generalizations that are not evidence based. So I welcome this large-scale
survey from Sense About Science. Once again, the importance with which peer
review is viewed comes through, with the great majority of researchers
believing that without peer review there would be no control in scientific
communication. That is not to say there aren't problems - there clearly
are, and improvements and innovative solutions are needed. Crucial in this
is the need to professionalize this area of activity, which too often is put
in the hands of people who may have great academic reputations and research
expertise, but no experience of running a peer-review system."
Notes
The Peer Review Survey was an electronic survey conducted between 28th July 2009 and 11th August 2009; 40,000 researchers were randomly selected from the ISI author database, which contains published researchers from over 10,000 journals. Altogether 4,037 researchers completed our survey. The error margin was ± 1.5% at 95% confidence levels; reviewers answered a subset of questions aimed specifically at reviewers (3,597 - a subset of the base) the error margin for this group was ± 1.6% at 95% confidence levels.
The full findings and report are due to be published in November 2009.
Bjork et al (2008) 'Global annual volume of peer reviewed scholarly articles and the share available via different Open Access options' Proceedings ELPUB2008 Conference on Electronic Publishing - Toronto, Canada - June 2008
Publishing Research Consortium (2007) 'Peer Review in Scholarly Journals: perspective of the scholarly community. An international study'
Chalmers & Glasziou (2009) 'Avoidable waste in the production and reporting of research evidence' The Lancet; 374: 86-89.
|
Feedback: Do you have any comments about this article? Contact the Publisher, Peter Llewellyn.
|
|
|
