Peer reviewing / peer reviews

0:00 2:59

This is the process by which papers submitted to journals are sent to others, “peers”, to be reviewed, i.e. for problems with the paper to be picked up and also for the reviewer to recommend to the editor whether the paper seems of sufficient quality for the journal to accept it (at least in principle).

The process sometimes results in immediate rejection but if not, it is then often in two waves: a first in which the peer reviewers make a lot of criticisms/suggestions, and a second after the authors resubmit having made changes to the paper and accept and/or reject the criticisms/suggestions. The editor will generally send the paper back to the peer-reviewers for their view on whether or not they consider that any problems have been fixed though sometimes the editor may feel that decision is clear enough and not bother the reviewers again. Sometimes the process goes through one or ocasionally even more iterations of submission and review.

When it works well peer-reviewing can be a genuine contribution to keeping what gets published in the “peer-reviewed literature/journals” accurate and of good quality. Too often it’s a pretty erratic and unreliable system, arguably made worse by the pressures on everyone in the research world to publish more and more.

One huge issue is that peer review work is almost never paid which can be a disincentive to reviewers to give papers really careful thought and appraisal. This is part of the whole madness of academic publishing which makes it a perfect cash cow for the publishing companies (a small number of which have a near monopoly on the “respectable” journals). Authors/researchers aren’t paid for their work, they are generally expected to hand over copyright to the publishers; the peer reviewers aren’t paid and I think editors rarely are. Then the academic world (including of course the authors, researchers and reviewers) pay for the journals (or these days, for access to individual papers). “Open access” publishing (q.v.) was supposed to improve this situation but in my blunt view it has hardly improved it at all and publishers are charging more and more extortionate fees for papers to be “open access”.

Details #

Peer review was traditionally anonymous, either singly anonymous, i.e. the reviewers were not identified but the authors of the paper were, or doubly when the authors and editors were expected to adjust things in the paper, like removing the author details but also other things in the text that might have leaked the authors’ identities.

Increasingly there has been a shift toward open peer review where identities are not concealed and reviews are published with the paper. This is no panacea for the many problems of the peer-review system but it is probably a small improvement on the traditional methods.

Try also #

Open access publishing
Open peer review

Chapters #

Not specifically addressed anywhere in the book but chapter 10 is a plea for a different, less adversarial and more honest, more constructive mode for criticism and argument about change data more generally.