There’s an interesting question about peer-review of journal articles that I’ve never seen addressed: How long does it take to review a paper? I don’t mean the three weeks or so between getting a request and submitting a review, but rather the time spent actually reviewing. In other words, how many hours does a reviewer spend reviewing the paper and not doing anything else? For myself I would guess that this is 2-3 hours for a typical paper, which includes reading and re-reading as well as writing the actual assessment. (Some papers have been considerably more time-consuming than this; few have been less.)
The question was triggered by a comment from Andrew Gelman (who writes a very good statistics blog), who states that his reviews take 15 minutes each. (!) My reviews are certainly longer than most people’s, but I find it inconceivable that I could do anything meaningful in 15 minutes. The handful of people I’ve talked to report a time of a few hours per paper. I would think that journals might have a good estimate of this, from surveys of reviewers perhaps, since it would be useful for them to know how much labor they are asking their reviewers to provide (for free). However: I asked a journal editor friend of mine, who replied that they haven’t collected any data on reviewer’s time-per-review! (I don’t know about other journals.)
Journals complain (e.g. here) that it’s hard to find willing reviewers. Reviewers complain that they get too many requests to review papers. Authors complain that reviewers are slow and capricious. Everyone complains that peer review is “broken.” Perhaps a better tabulation of what time and effort peer review requires would help address all this!
If you, dear reader, would like to comment on:
(i) how long it takes you to review a paper
(ii) how many requests to review manuscripts you get
(iii) what fraction of requests to review you accept
(iv) what field you’re in
that would be great! I’ll see if any conclusions emerge…Also, of course, it would be great to know if someone else has already done this, with the outcome available to read.
(Today’s illustration: from a card the kids and I made a few days ago.)