Reviewing anger

In my book I write briefly (p.xiv) about the depressing nature of academic journal publication in organization studies, and have done so at more length in a post on this blog. In that post, written in 2014, I bemoaned the tendency for the anonymous reviewers of journal submissions to write reams of comments making demands of authors so excessive that whole books could not satisfy them.

Recently, though, I have noticed a different trend. I now find that papers I submit to journals come back with very brief comments. Occasionally, they are brief but positive. More often they are brief and not just negative but hostile, dismissive and, often absurd. For example, I recently had a damning review, endorsed by the editor, whose main complaint was that I had completely ignored a key piece of work in the field when in fact it was not just referenced but extensively quoted. Again in contrast to my experience a few years ago reviews are accompanied by an editorial letter that is curt and content-free apart, of course, from the message that the paper has been rejected. I should say that I am talking here about what are regarded within the UK and Europe (the US ranking of such things being different) “top journals” as defined by the ubiquitous Chartered Association of Business Schools’ (CABS) listing.

Anyone reading this who works in the field may be thinking that I am giving vent to sour grapes at having my papers rejected by journals. And of course I am. But there is more to it than that. I’ve been doing this job for a long time and so like anyone else I am used to papers being rejected. With the top journals’ overall rejection rates being in the order of 90% that is pretty much inevitable. So that is not the issue. What I see shifting is the manner of rejection.

That reviews are getting briefer is perhaps not surprising. I know, from the other side of the fence, how many requests to review I receive, and this activity is time-consuming to do thoroughly. Perhaps spending that time used to be seen as part of the communal reciprocity of being an academic, but with the time pressures now much-increased I can see why brevity might be becoming the norm.

What is more surprising, though, is how angry these reviews are. In recent weeks I’ve had reviews of papers which positively bark about, for example, there being ‘no contribution’. Several times there’s also been a sense of outrage at the impertinence of the paper under review for entering territory apparently regarded as ‘owned’ by the reviewer. It’s as if the reviewers feel insulted – and so insult the author.

Now, again, an obvious reaction would be to say that perhaps the problem is that my papers make no contribution and/or are amateurish forays into others’ fields. No wonder, then, that reviewers are angry. The problem with that diagnosis, though, is that the very same journals send me articles to review on the very same topics, on the basis that I have expertise in them. So whilst it is plausible that my own work has flaws that I am blind to – and I will happily own to that – it seems implausible that my critical faculties are so blinded that I can both be an expert reviewer of others’ work whilst myself producing work of such dire quality that angry dismissal is not only justified but positively demanded. And, if that is so, why do those same journals constantly send me papers to review?

So now let’s suppose another objection to what I am saying. Perhaps it is that I am sent papers to review on the basis of my expertise because in the past I did good work but I’ve now gone downhill and the papers I submit now are as risible as the referees say. But that can’t be true, either, because more often than not those same papers have gone on to be accepted by a different journal, with a different editor and different reviewers. Again, such discrepancies aren’t new – there’s always been a zone of judgment and ambiguity about what is a good paper – but they seem to me to be getting far more extreme, with a paper being as likely to be dismissed as worthless as lauded as excellent.

Based on conversations with other people I am pretty sure that the experiences I am describing are not unique to me. I am not entirely sure what the explanation is but I have a couple of ideas. One is that journal editors are not paying very much attention to their work and (despite the inevitable claim of having read the paper “very carefully”) are not exercising much editorial judgment; and bear in mind that ‘editor’ means, typically, one of a massive array of associate or assistant editors of varying ability and diligence. There are also, probably, cases of personal animosity: although submissions are anonymously reviewed it is often easy for reviewers to guess author identity (and vice versa) and, of course, handling editors know author identity anyway.

The other, more important, thing is that I think that reviewers have got angrier because as authors they have been on the receiving end of angry reviews. Which is chicken and which is egg is impossible to say, but there is surely a psychological logic in the idea that if your paper is trashed on spurious grounds you will look to trash those you review. That’s really the obverse of the point I made earlier about the incongruity of me being regarded as an expert reviewer and yet an incompetent author. It arises from that fact that all of us in the field are simultaneously authors, reviewers and, for that matter, editors.

All of this is annoying – infuriating, in fact – but it is worse than that. For me, the stakes are not very high. Deservingly or not I am reasonably well-established, securely employed, have no post I can be promoted to on research grounds, am not that far off retirement, and one way or another more than meet institutional systems of research evaluation. So a paper getting rejected in a spurious or unpleasant way is upsetting but doesn’t make any real practical difference to me. But many people, including many that I write with, are not in this happy position. For them, a publication in a “top journal” can make or break their careers, or can make huge differences to promotion chances or to salary levels.

Despite the huge power they wield there is no accountability whatsoever of journal editors and anonymous reviewers, and no possibility whatsoever of challenging their judgments. I believe that some authors do seek to challenge decisions, but I have always thought that this is a hiding to nothing and have never done so. Apart from being undignified, it’s not clear what it could achieve: if, for example, reviewers have trashed a paper and the editor has rejected it then unless an appeal yielded a whole new set of reviewers, which is highly unlikely, shifting an editor to allow a revise and resubmit is probably not going to lead to a better outcome. Pragmatically, it’s better to send the paper to another journal and hope for better luck with the editor and reviewers.

Be that as it may, it does not negate the fact that sloppy editing and angry reviewing is damaging to academic careers and – which may matter more to those who are not academics – to what gets published and therefore read. It’s a hidden scandal, and though a minor one in the general scheme of things no less important to its victims.