Back to Basics

Recently I read this post -- "Behavioural Science Needs to Return to the Basics" -- and I didn't like it. In this blog post I am going to complain about it. To be clear my complaints are not going to dispute that there are lots of tiresome prigs in academia, that given the general demeanour of academics tiresome moralists tend to be of the liberal left variety, and even less that in work (both ideological and not) there is a lot of slapdash reasoning. All of those claims are certainly true, and (aha) the lived experience of any academic will confirm them adequately enough. (If you don't believe me then see here.) These are pretty central claims to the linked piece, so that's a substantial amount of agreement!

But alas in academia we don't just care that you say true things, we care that you say true things for the right reasons. The old joke that philosophers are the kind of people who say "I know it works in practice but does it work in theory?" only misses the mark in limiting the scope of nerd pedantry to philosophers. And my problems with this piece are very largely with the premises they deploy and the inferences they make from those premises. Rather than list my every complaint I shall simply reason through an example.

Take this paragraph:

A striking example is the American Psychological Association’s (APA) statement shortly after the death of George Floyd, which provides a textbook illustration of the availability bias, the tendency to overvalue evidence that easily comes to mind. The APA, the largest psychological organization in the world, asserted after Floyd’s death that “The deaths of innocent black people targeted specifically because of their race—often by police officers—are both deeply shocking and shockingly routine.”45 How “shockingly routine” are they? According to the Washington Post database of police killings, in 2020 there were 248 Black people killed by police. By comparison, over 6,500 Black people were killed in traffic fatalities that year—a 26-fold difference. Also, some portion of those 248 victims were not innocent—given that 216 were armed, some killings would probably have been an appropriate use of force by the police to defend themselves or others. Some portion was also not killed specifically because of their race. So why would the APA describe a relatively rare event as “shockingly routine”? This statement came in the aftermath of the widely publicized police killings of Floyd and those of Ahmaud Arbery and Breonna Taylor. In other words, these rare events were seen as common likely because widespread media coverage made them readily available in our minds.

It occurs in a section wherein they are trying to show that "[t]he empirical bases for some DEI concepts contradict social scientific principles. Additionally, certain DEI ideas run counter to important findings about human nature that scientists have established by following the required scientific principles." In this instance they are concerned especially to illustrate that "in the discussion of DEI topics, psychologists often fall victim to" cognitive biases. 

Now I will grumble a bit that falling prey to a cognitive bias is not per se to contradict any principle or run counter to some previous finding. The point can be made obvious by noting that a conclusion which contradicts that which is known is thereby false, whereas a biased reasoner can reach a true conclusion. If I always say any mathematical sum is 4 then my reasoning suffers from a rather strong and bizarre bias, but that does not entail that when I add 3 to 1 it does not equal 4. Whereas if we have established that Mount Everest is the tallest mountain then if I say that Mount Kilimanjaro is distinct from Everest yet the tallest mountain, even if I had very good grounds for thinking so, I am of necessity incorrect. They are thus, even if only subtly and implicitly, overstating their case. So, there, consider that grumble to have been engrumbled. But I will say no more on the matter since I think it not too important in the grand scheme of things; I think probably this just means they are saying both that the conclusion is false and also that it was only reached for bad reasons.

My issue is then that they simply do not establish their conclusion at all. They object to the assertion that "The deaths of innocent black people targeted specifically because of their race—often by police officers—are both deeply shocking and shockingly routine." In line with what I just concluded, they both dispute it on factual grounds and also give a bias-based error-theory for why the assertion was made. We examine these in turn.

Presumably to dispute the idea that something is "shockingly routine" you have to show at least one of: it is not shocking or it is not routine. There is some difficulty here, since both of these involve hazy appeals to an unstated baseline expectation - routine as in more than one might expect if it were just chance? Routine as in highly regular? Shocking as in commands much more attention than every day events? Shocking as in a particularly large violation of normative expectations? Unclear. But in any case you'd have to do something of the sort.  How do the authors go about this? 

Well they begin by noting that more black people are killed by traffic accidents. Off to a bad start! For, does that mean that that gun deaths are not shocking or not routine? I suppose if you thought gun deaths were shocking because they were the largest cause of death, or maybe if you thought that the rate of traffic incidents established the baseline for how much you should expect gun deaths to occur if routine, then this would be pertinent. But nobody thinks either of those things and there is certainly no reason to attribute them or anything like them to the APA! It's just irrelevant, the rhetorical equivalent to exclaiming "What in the world is that behind you!" Pure distraction from the issue at hand.

They then make two more claims which are, at least, pertinent to the issue at hand. They point out that "some portion of those 248 victims were not innocent" and that [s]ome portion was also not killed specifically because of their race". Both presumably true and if true in a large enough proportion of cases then might go some way to establishing the events are not routine (I am not sure how far they would go to showing that it is not shocking - you might think that the armed agents of the state only have to shoot dead a very few people before one reasonably becomes shocked, especially if one suspects they will face no reprisal for doing so.) So now we need to know whether it is a large enough proportion of cases. They say that of the 248 there were 216 that were armed - making it about once a fortnight on average that an unarmed black person was shot dead by police. I mean I personally find it reasonable to describe that as shockingly routine, I don't know what to say? But in any case I certainly don't know it's not, and once one takes into account that one could not so neatly line up justifiability with armed/not-armed (and doubts one may have about the source of those numbers, e.g.) and all the complexities involved in causal attribution w/r/t race then it's just apparent that at the least really not much has been said here.

So with regard to the factual claim I do not think we have really been given reason to suppose it false at all. We were instead offered one irrelevant distraction, and some claims that might be relevant were gestured too but nothing was actually done with them that might persuade. How about the error theory attributing the APA's claim to a specific cognitive bias? I quote again the entirety of the reasoning there:

So why would the APA describe a relatively rare event as “shockingly routine”? This statement came in the aftermath of the widely publicized police killings of Floyd and those of Ahmaud Arbery and Breonna Taylor. In other words, these rare events were seen as common likely because widespread media coverage made them readily available in our minds.

This in a piece which began by reminding people not to confuse correlation with causation, I hasten to add. I think it's reasonable (and certainly I trade on this good will!) that people do not hold blog posts aimed at a wide audience to full academic standards. So I shall simply say that I hope that in their published work the authors are more careful when attributing likely causality to do more than note temporal sequence in a single case. I leave it at that.

In my opinion this bit of reasoning was typical. It is not that I think the APA's statement on this was well reasoned or important - I am rather cynical about such organisations issuing such statements in cases just like this in a prior paper. I stand by my cynicism. But the reasoning given for why it was false and resultant from bias fails to stand up to even the remotest scrutiny, it's barely there at all. So I extend my cynicism about the APA to this piece from Sceptic dot com.

Obviously I have not established a general conclusion here! In particular, even if I am right about all the above, maybe their reasoning on other questions is better. Genuinely I could just be getting annoyed at something, fixating on it, and then seeing everything else through a bitter light. That happens, and to establish it's not what occurred here I would need to go through everything in a similar level of detail. (And in fact I think on some meta-level that has certainly occurred -- because I agree with the original authors, after all, that this sort of reasoning more usually happens with a leftward slant in the academy, yet here I am focussing on an example with a more centre-right slant. Isolated demand for rigour thy name is Kofi!) Since I am not willing to do that so I think it's reasonable enough if someone just disagrees with me; I can't reasonably ask for more! I can only invite readers to go through the piece with an eye to these inferential issues -- in fact, to do this always in all works about important topics, do not actually make isolated demands for rigour! -- in mind. Ask yourself not just whether something feels relevant ("Hey yeah, good point, why aren't people talking about traffic deaths!?") but, strictly speaking, how it actually pertains to whatever thesis the authors are trying to establish. If I am right then you will find these sort of laxities recur, while if I am wrong you will at least have avoided my error. 

But if I am right then frankly I think the authors fall victim to the exact kind of issue they themselves diagnose towards the end of the piece. When it comes to culture war issues (maybe especially around race? Or maybe I just pay attention to those more, I am not sure) many people simply do not reason seriously. They are correct that this happens plenty with a leftward slant in the academy! But they do the exact same thing. Whereas if you think it matters how exactly society is arranged and what may be done about that, especially on those issues most morally urgent and the subject of most heated debate, you simply must demand a higher calibre of reasoning from the intelligentsia. We owe the world better.