On The Conceptual Penis

Another day, another hoax paper published! For those who don't know, these hoax papers pop up every now and again, wherein academics deliberately write nonsense, submit it to an ostensibly serious journal, and find that lo and behold they can get it published, despite the checks that are meant to ensure only rigorous scholarship makes it through to the published literature. This is a worry, because in theory being part of the scholarly literature is meant to be a guarantee of quality and a sign that the scholarship can be relied upon.

It's always pretty hard to know what to conclude from these hoax papers -- I think it is over-determined that one should not, ever, take the fact that an individual paper has passed peer review to mean its conclusions are secure. Where one wants to be guided by academic work, one should form one's opinion on a matter based on a review of the literature, not just one or two papers therein. Too many journals reviewing too many papers by too many authors in too many fields -- all run by fallible humans. Even with the best of intentions (which is far from guaranteed) there is no way the peer review process could filter out all bad work, and by dint of the sheer size of the enterprise there is going to be plenty of rubbish out there even if a low proportion of the bad papers make it through. What's more, the replication crisis shows us that things can actually go pretty systematically wrong, and there's some theoretical reason to be a bit pessimistic about the quality of the average paper. So, one read of hoax publications is that they just dramatically illustrate this point. Whatever guarantee of quality the scholarly literature and its processes of review and double checking provide, it does not do all that much at the level of individual papers -- look to the collective beliefs or consensus arising out of such literatures, if there is reliable results to be found anywhere it will be therein.

The authors of this particular hoax, however, want a different conclusion to be drawn. They think their paper shows that a ``problem lies within the very concept of any journal being a “rigorous academic journal in gender studies.”'' Their reasoning is that their article sailed through peer review because it ``portrayed a moralizing attitude that comported with the editors’ moral convictions'',  namely ``an overriding almost-religious belief that maleness is the root of all evil''. Basically the paper kind of strings sentences together which loosely suggest that maleness is tied to a certain kind of abstract idea of the penis and this abstract idea of the penis is responsible for a great many social ills, inclusive of climate change. It really is largely nonsense, but an overall effect is conveyed and which is indeed suggestive of the almost religious belief they note. They think that it was because this effect was conveyed that they were not subject to serious peer review.

Predictably enough (pious left wing academical that I am!) I am not convinced by their account of what happened here. I'll set aside the fact that some of what the paper suggests would probably be considered morally offensive by the sort of people they have in mind -- it's reading tea-leaves to try and work out what the paper is really suggestive of, since it is, by design, genuinely nonsensical.  Rather, my objections to their account are as follows. First, boringly, I just don't think journal hoaxes provide the kind of evidence that could support their conclusion. A paper published in a journal is just poor evidence for conclusions concerning the practices of an entire field. Second, this paper was actually first rejected by NORMA: International Journal for Masculinity Studies! Why should the one accept count for more evidence of typical field practices than the one reject? Especially when you consider that, third, a relevant difference between the journal that accepted (Cogent Social Sciences) and the one that rejected was that the journal that accepted is a pay-to-publish journal. To my mind, we already have some reason to be highly suspicious of pay-to-publish journals. If one searches around one will find plenty of controversies about these journals, just going to link to a story about my favourite such controversy here. So, given where my priors are at, once I found out this detail it immediately suggested an explanation of what went wrong here! Personally, I am sufficiently suspicious of this business model that I think we as an academic community should institute some communal norms against publishing in pay-to-publish journals, and demand that any article that is published in such a journal is suitably marked as such and refuse to credit work so marked -- in fact, given the wider social role of academic work, I wouldn't even be opposed to legislation banning the business model.

Now, the authors do spend some time discussing the third of these worries. They acknowledge that indeed pay-to-publish journals might be a source of the problem. But they respond that, (A), NORMA's editor actually recommended transferring their paper to CSS in such a way that helped it get through the review process there, suggesting that even serious academic journals are in cahoots with these mobsters. (B), CSS did seem to actually implement peer review, (C), the journal is published by the apparently respectable Taylor and Francis group. These considerations move them to think that the problem is not mainly a general one with pay-to-publish journals, but rather the field. Again, I am not convinced. Regarding (B) -- editors have a lot of power in who they pick as reviewers, so they can just pick hacks to ensure things sail through, and in any case we don't know how the reviewers were themselves incentivised. Regarding (C), let's just say I am not too impressed with academic publishing groups more generally.

Regarding (A)... ok here I agree with the authors of the hoax paper. In so far as journals in gender studies (and for all I know this happens in other fields too) are collaborating with the pay-to-publish journals in this way, they are undermining their own field's scholarly standards. I didn't know this sort of collaboration existed, so for me this is the main thing I have learned from the incident, and I shall be looking out for this in future; maybe it occurs in philosophy too? The overall lesson I draw from this latest hoax is that as a community we should stop collaborating with these sharks, their invocation of the profit motive undermines the scholarly values we strive to maintain and represent, and it is only by our continued participation in the system that they are able to so destroy us.