Du Bois on Da Vinci

A quick write up on a charming essay by the young Du Bois (from his time as a graduate student at Harvard), which I only found out about through the fascinating historical work of Trevor Pearce. The essay is entitled Leonardo Da Vinci As A Scientist and is available online here.

Leonardo Da Vinci -- ``I was even a pioneer in
side-eye and general shade throwing.''

Du Bois is concerned to argue that Da Vinci deserves credit as the founder of modern experimental science. The argument strategy is twofold. First, to show that Da Vinci has sufficient (and sufficiently impressive) scientific achievements to merit attention as an early scientist at all. This Du Bois achieves by just reviewing historians (apparently then - 1889 - relatively recent) reappraisal of Da Vinci's empirical work and work inventing scientific machinery and to show that it was indeed impressive. This in itself was interesting; so for instance I learned here that Da Vinci was already floating the idea that the sublunary realm and the broader cosmos should be understood as operating on the same principles, that Da Vinci has a
claim to being an early inventor of the telescope and also being the first to notice a parallel between how the camera obscura works and the operations of the human eye, and that on the basis of observational study of plants Da Vinci was developing ideas about plant respiration which now seem to have been on the right track. Cool!

The second step in the argument, however, is the more philosophically and conceptually interesting. Here Du Bois' task is to argue that Da Vinci deserves credit not just as a link in a great chain of scientific workers, but rather some sort of special credit as a founder figure in one sense or another. Here the point is largely drawn out by comparison with three other figures: Roger Bacon Gilbert of Colchester and Francis Bacon. While Du Bois is impressed with each of these figures, he thinks they were each lacking in a certain way. Roger Bacon was not enough of an empiricist: to be credited as a founder of modern science, Du Bois feels, empiricism must be one's epistemological foundation, where for R.Bacon ``empiricism was but a branch of the tree of which philosophy was the trunk''. Glibert of Colchester has, so to speak, the opposite problem -- he's all empiricism with no metatheory. While he's impressive in his collection of observational and experimental results, he's ``a mere experimenter, with little breadth of conception, or broad generalising powers''. F. Bacon, finally, came after Da Vinci, and is substantially the same in his metatheory (so Du Bois thinks! Please don't hurt me, Renaissance scholars), but just didn't achieve as much scientifically as Da Vinci. F. Bacon comes across, basically, as an especially talented expositor of Da Vincian method, but not himself worthy of the claim to priority on scientific method.

The philosophy of science young Du Bois is working with is interesting, and worth making more explicit than Du Bois himself does in the essay. In Da Vinci, Nature had found itself a man who could do both: patient skillful observational work, aided by machines of his own device, that uncovers particular facts of great interest and also general principles, and also explicit epistemological theorising of a sort which acknowledged and explained the importance of founding one's claims in such observations. Science, then, is the epistemologically self-conscious skillful application of empiricist method. R. Bacon was a skillful natural philosopher and epistemologically self-conscious, but not an empiricist. Gilbert of Colchester was a skillful empiricist, but did not evince the requisite degree epistemological self-consciousness. F. Bacon was an epistemically self-conscious empiricist, but just not quite good enough at the actual application. Da Vinci was the first person in whom all these qualities meet to a sufficient degree, or so Du Bois claims. (This essay also features a trait which is characteristic of all Du Bois' latter work on social matters -- explicit reticence and diffidence, with frequent reminders that one ought be cautious about one's conclusions given the difficulties of gathering evidence and being sure it is complete or representative.)

W.E.B. Du Bois -- ``The idea that the person
in this picture could ever be as enthusiastic
about anything as the person who wrote that
essay on Da Vinci is genuinely surprising.''

I've worked on Du Bois' philosophy of science before, but I have never in my published work explicitly remarked on the undercurrent of empiricism. None the less, it is there; most especially it can be seen in his lifelong habit of issuing scathing condemnations of a priori approaches to history and sociology, where he thinks that prejudice unchecked by experience has been the source of much racist balderdash concerning African (and African-descended) folk. It is remarkable to think, then, how closely Du Bois' scientific and social mission accords with the early philosophy of science he developed here. For, The Philadelphia Negro or Black Reconstruction can plausibly be described as epistemologically self-conscious skillful applications of empiricist method; in both these works (and many of his less famous essays besides) he mixes explicit methodological remarks exhorting a more carefully and rigorously observationally grounded approach to the study of black life in America, with the actual collection of novel results about social, political, or economic conditions, and in both the highlighted cases they have (nowadays) come to be seen as classics of their respective fields. His work is thus epistemologically self-conscious in its empiricism, involves the actual application of observational method as well as its exhortation, and skillful performance thereof. The philosophy of science underlying this essay by the young Du Bois seems to have set a pattern that he attempted to live up to for the rest of his scientific career.

Da Vinci, of course, is not just a great scientist and engineer, but also a great artist. Du Bois was evidently aware of this, and this fact about him is mentioned at various points in the essay. Da Vinci is indeed paradigmatic of the Renaissance Man, the individual who strives to hone diverse skills to a high degree and exhibit a broad culture. In this respect too Du Bois seems to have followed Da Vinci, being more acclaimed for his literary style and humanistic moral and political vision than his scientific career. Being attracted to the broad humanism of the Renaissance, and having great respect for Du Bois' work, seeing this essay where Du Bois develops his ideas about philosophy of science as part of an ode to Da Vinci and the Renaissance scientific humanism that Da Vinci pioneered, was in its own way quite affecting for me. Even if I cannot match these figures in their skill, I hope to at least preserve and advance the spirit of humanistic inquiry that they each embodied.

Significant Moral Hazard

What follows is a guest post by my comrade Dan Malinsky. After the recent publication of the paper `Redefine statistical significance' Malinsky and I attended a talk by one of the paper's authors. I found Malinsky's comments after the talk interesting and thought-provoking that I asked him to write up a post so I could share it with all yinz. Enjoy!

--------------------------------------------------------------------------------------------------------------------------

Benjamin et al. present an interesting and thought-provoking set of claims. There are, of course, many complexities to the P-value debate but I’ll just focus on one issue here.

Benjamin et al. propose to move the conventional statistical significance threshold in null hypothesis significance testing (NHST) from P < 0.05 to P < 0.005. Their primary motivation for making this recommendation is to reduce the rate of false positives in published research. I want to draw attention to the possibility that moving threshold to P < 0.005 may not have it’s intended effect: despite the fact that “all else being equal” such a policy should theoretically reduce false positive rates, in practice this move may leave the false positive rates unchanged, or even make them worse. In particular, the “all else being equal” clause will fail to hold, because the policy may incentivize researchers to make more errors of model specification, which will contribute to a high false positive rate. It is at least an open question which causal factors will dominate, and what the resultant false positive rate will really look like.

An important contribution to the high false positive rates in some areas of empirical research is model misspecification, broadly-understood. By model-misspecification I mean anything which might make the likelihood wrong: confounding, misspecification of the relevant parametric distributions, incorrect functional forms, sampling bias of various sorts, sometimes non-i.i.d.ness, etc. In fact, these factors are more important contributions to the false positive rate than the choice of P-value convention or decision threshold, in the sense that any plausible decision rule no matter how stringent (whether it is based on P-values, Bayes factors, or posterior probabilities) will lead to unacceptably high false positive rates if model misspecification is widespread in the field.

Note that the authors Benjamin et al. agree on the first claim. Benjamin et al. mention some of these problems, agree that they are problems, and frankly admit that their proposal does nothing to address these or many other statistical issues. Model misspecification, in their view, ought to be tackled separately and independently of the decision rule convention. The authors also admit that these and related issues are “arguably bigger problems” than the choice of P-value. I think these are bigger problems in the sense specified above: model misspecification will afflict any choice of decision rule. This is important because the proposed policy shift may actually lead to more model misspecification. So, the issues interact and it is not so straightforward to tackle them separately.

P < 0.005 requires larger sample sizes (as the authors discuss), which are expensive and difficult to come by in many fields. In an effort to recruit more study participants, researchers may end up with samples that exhibit more bias -- less representative of the target population, not identically distributed, not homogenous in the right ways, etc. Researchers may also be incentivized, given finite time and resources, to perform less model-checking and diagnostics to make sure the likelihood is empirically adequate. Furthermore, the P-value critically depends on the tails of the relevant probability distribution. (That’s because the P-value is calculated based on the “extreme values” of the distribution of the test statistic under the null model.) The tails of the distribution are rarely exactly right at finite sample sizes, but they need to be “right enough.” With a low P-value threshold like 0.005, getting the tails of the distribution “right enough” to achieve the advertised false positive rate becomes more unlikely because with 0.005 one considers outcomes further out into the tails. Finally, other problems which inflate false positive rates like p-hacking, failure to correct for multiple testing, and so on may be exacerbated by the lower threshold. The mechanisms are not all obvious -- perhaps, for example, making it more difficult to publish “positive” findings will incentivize researchers to probe a wider space of (mostly false) hypotheses in search of a “significant” one, thereby worsening the p-hacking problem -- but it is at least worth taking seriously that these factors may offset the envisaged benefits of P < 0.005. (I think there are some interesting things which may be said about why these considerations are less worrisome in particle physics, where the famous 5-sigma criterion plays a role in announcements. I’ll leave that aside for now.)

I’m not disputing any mathematical claim made by the authors. Indeed, for two decision rules like P < 0.05 and P < 0.005 applied to the same hypotheses, likelihood, and data, the more stringent rule will lead to fewer expected false positives. My point is just that implementing the new policy will change the likelihoods and data under consideration, since researchers will face the same pressure to publish significant results but publishing will be made more difficult in a kind of crude way.

This worry will be relevant for any decision threshold convention, and so it speaks against any strict uniform standard. However, Benjamin et al. raise the important point that “it is helpful for consumers of research to have a consistent benchmark.” My friend and colleague Liam Kofi Bright reinforces this point in his blog post: there are all sorts of communal benefits to having some mechanism which distinguishes “significant” results from “insignificant.” I’d like to propose a different kind of mechanism.

Sometimes statisticians casually entertain the idea of requiring “staff statistician reviewers” to review (the data analysis portions of) empirical articles submitted for publication. I think we can plausibly institutionalize a version of this practice, and it can function as a benchmarking procedure. Every journal will pay some number of professional statisticians (who should be otherwise employed at universities, research centers, etc.) to act as statistical reviewers, and specifically to interrogate issues of model specification, sample selection, decision procedures, robustness, and so on. Only when a paper receives a stamp of approval from two or more statistical reviewers should it count as having “passed the benchmark.” The institutionalization of this proposal would have some corollary benefits: there are a lot of statistician professionals who are employed with “soft money,” i.e., they have to raise parts of their salaries by applying for grants. This mechanism could partially replace that grant-cycle: journals would apply regularly every few years for funding from the NIH, NSF, and other funding agencies to compensate statistical reviewers (an amount dependent on the journal’s submission volume); the statisticians get to supplement their incomes with this funding rather than spend time applying for grants; and the public gets some comfort in knowing that the latest published results are not fraught with data analysis problems. I can image a host of other benefits too: e.g., statisticians will be inspired and motivated to direct their own research towards addressing live concerns shared by practicing empirical scientists, and the empirical scientists will be alerted to more sophisticated or state-of-the-art analytic methods. Statistician’s review may also reduce the prevalence of NHST, in favor of some of the alternative analytical tools mentioned in Benjamin et al. The details of this proposed institutional practice need to be elaborated, but I conjecture it would be more effective at reducing false positives (and perhaps cheaper) than imposing P < 0.005 and requiring larger sample sizes across the board.

[I should acknowledge that, depending how my career goes, I could be the kind of person who is employed in this capacity. So: conflict of interest alert! Acknowledgements to Liam Kofi Bright, Jacqueline Mauro, Maria Cuellar, and Luis Pericchi.]

Supporting the Redefinition of Statistical Significance

Recently an article entitled `Redefining Statistical Significance' (RSS) has been made available. In this piece a diverse bunch of authors (including four philosophers of science - represent) put forward an argument with the thesis: ``[f]or fields where the threshold for defining statistical significance for new discoveries is P<0.05, we propose a change to P<0.005.'' In this very brief note I just want to state my support for the broad principle behind this proposal and make explicit an aspect of their reasoning that is hinted at in RSS but which I think is especially worth holding clear in our minds.

RSS argues that, basically, rejecting the null at P<0.05 represents (by Bayesian standards) very weak evidence against the null and in favour of the hypothesis under test, and further than its communal acceptance as the standard significance level for discovery predictably and actually leads to unacceptably many false-positive discoveries. P<0.005 taken as the norm would go some way towards solving both these problems, and the authors emphasise most especially that it would bring false positive levels down to within what they deem to be more acceptable levels. RSS doesn't claim originality for these points, and is a short and very readable paper; I recommend checking it out.

The authors then have a section replying to objections. They note that they do not think that changing the significance level communally required for discovery claims is a cure-all, and deploy a number of brief but very interesting arguments against the counter-claim that the losses in terms of false-negatives would outweigh the gains in avoiding false positives. This is all interesting stuff, but the point at which I wish to state my broad agreement comes when they consider the objection that ``The appropriate threshold for statistical significance should be different for different research communities.'' Here their response is to say that they agree in principle that different communities facing different sorts of puzzles ought use different norms for discovery claims, but note that many communities have settled on the idea that given the sort of claims they are considering and tests they can do  P<0.05 is an appropriate standard for discovery claims. They are addressing those communities in particular with their proposal, so are addressing communities which have already come to agree that they should share a standard for discovery claims.

My one small contribution here, then, is in following up on this point. They briefly note in their reply to this objection that -- `it is helpful for consumers of research to have a consistent benchmark.' I think this point deserves elaboration and emphasis, and it is why I feel that, although I do not feel sufficiently expert to comment on the specific proposal they made, the broad contours of their argument are right. Why, after all, do we actually have to agree on a communal standard for what counts as an appropriate significance level for `claims of discovery of new effects' at all? Couldn't we leave that to the discretion of individual researchers? Or maybe foster for some time a diversity of standards across journals and let a kind of Millian intellectual marketplace do its work? To put it philosophically, why have something rather than nothing here?

I take it that a lot of what the communal standard is doing is providing a bench mark for those not able to make expert or highly-informed personal assessment of the claims and evidence to know that the hypothesis in question is confirmed to the standards of those who are able to make expert or highly informed assessments. These consumers of the research are those for whom the consistent benchmark helps. Especially for the kind of social scientific fields which have in fact adopted this benchmark, a pressing methodological consideration has to be that non-scientists or folk not able to assess statistical claims, and more pointedly people with policy or culturally influential positions, will consume the research, and take actions based on what they believe to be reliable, or at least take action on the grounds of what convinces them. The trade off between Type 1 and Type 2 errors, then, must be made with it in mind that there is an audience of non-experts to the claims made in this field, and an audience who will shape actions and lives and self-perceptions (in part) upon the results these fields put out. As a scientific community we must therefore decide what we think of our own work can be vouchsafed to these observers, or validated to the standard this cultural responsibility entails.

In theory, of course, we could still leave this up to individuals or allow for a diversity of standards among journals. But I think awareness of the scientific community's public role tends to speak against that. Such diversity, I'd wager, would either result in a cacophonic public discourse on science in which the media and commentators constantly reported results, then their failure to replicate, and then their replication once more (as well as contrary results, their failure to replicate...). This because the diversity of standards led to non-experts picking who to believe randomly among folk with different standards, or according to who they judged to have the flashiest smile, or whichever university PR department reached out to them last, or factionally choosing their favourite sources. Or, it would result in silence, as gradually scientific results come to be seen as too unreliable, too divided among themselves, to be worth paying much attention to at all. If you think that scientifically acquired information can make a positive difference to public discourse, either of these seem like bad outcomes. (The somewhat self-promoting Du Bois scholar nerd in me can't resist pointing out that Du Bois brought similar considerations to bear in responding to widespread failures of social scientific research in his day.) In fact, I think this epistemic environment makes a conservative attitude sensible, and speak in favour of adopting a very low tolerance for false-positives. This because is much harder to correct misinformation once it is out there than it is to defer announcing until we are more confident, and the very act of correction may induce the same loss of trust worry mentioned before. This means that in addition to elaborating upon RSS' reply to an objection, and without feeling competent to quite judge whether P<0.005 in particular is the right standard, I also think the overall direction of change advocated by RSS is the right one, relative to where we are now.

Decolonise Philosophy!

The following thoughts, prompted by this article, will (I suspect) almost all be super obvious to anybody who has been thinking about decolonising philosophy for an extended period of time. But my audience is largely composed of people, methinks, who do not regularly think about such things.

Lots of people would agree with the slogan ``We ought decolonise philosophy!'' but, philosophy being what it is, the meaning of the slogan is highly contentious. I'll work with one account thereof, based on this and related papers by Kwasi Wiredu, but bear in mind that it's not the only account of what it would take to decolonise philosophy that is out there. I think this particular account makes my point very stark, but something essentially similar to what I say would go if I had worked with some other prominent accounts. Wiredu begins by saying that what it means to decolonise African philosophy would be  ``divesting African philosophical thinking of all undue influences emanating from our colonial past.'' This is then cashed out in terms of taking conscious control of the concepts deployed in philosophical reasoning, as well as the substantive positions covered and the questions asked, by means of subjecting them to critique via cross cultural comparisons. The idea, basically, is to try and ferret out aspects of philosophical thinking now going on in Africa which can't earn their keep on their own merits but rather persist simply because the colonialists imposed them during their occupations -- and to ferret them out by using the fact that indigenous languages, conceptual schemes, and thought traditions have resources that can make incongruences stark by means of comparison, undermine false claims to necessity by evincing in practice alternate ways of going on, or may occupy regions of logical or conceptual space that the colonists never bothered to explore.

So, for instance, Wiredu argues that there are certain puzzles about existence or the nature of capital-b-Being which simply cannot arise if you are to formulate your thoughts in certain West African languages. In the essay linked, for instance, he argues that the notion of creation-ex-nihilo which has caused so much debate in philosophy of religion is nigh-on-incomprehensible if one tries to discuss it in his native Akan language. It is not that he thinks this therefore proves that those questions of Being are pseudo-puzzles, or that creation-ex-nihilo is impossible, but rather simply that it would be a colonial attitude to simply assume that this difference must be due to an expressive fault with the West African languages rather than a tendency to produce misleading linguistic confusions in the European traditions which concentrate on those puzzles and on the basis of nothing more than this assumption work to import the European concept. If it is a genuine improvement on the indigenous conceptual scheme that must be argued for. Further, having realised the incongruity, and not uncritically accepting the Western mode as just obviously superior, one can see whether and how thinking with the concept derived from one's own linguistic tradition would ramify through philosophical issues -- and in this paper he concludes, for instance, that attempts to harmonise or synthesise religions indigenous to Ghana and Christianity are probably not as coherent as some claim, but are relying on equivocation at key moments. In his own work he has applied this method to a number of other problems to interesting effect -- to give some of the provocative examples, he concludes that Descartes' cogito would fairly immediately have been seen to be an invalid argument had Descrates attempted to formulate it in a West African language, or on other occasions that the correspondence theory of truth is a tautology in an Akan language.

The point, then, is not simply to reject everything associated with the colonialists. (As he says, the emphasis in his initial definition of decolonising philosophy should be on the word `undue' before `influences'.) Rather, the point is to ensure that the tools we think with are up to task, and to use the availability of alternative tools as a means of facilitating test and comparison. So whether or not you one ultimately ends up accepting the problems-in-Western-languages as genuine or pseudo-problems, the decolonised philosophy is that which has used the conceptual resources and intellectual traditions of the former colonised nation to put itself in a position to consciously decide whether or not its inherited problems are worth pursuing in light of consideration of a fuller range of facts, rather than uncritically (or without due consideration of the facts adduced by considering the thought of the colonised) accepting the concepts, problems, and solution space given to it by Western tradition.

Before drawing out my intended moral, some comments on Wiredu's account as an account of decolonising philosophy. I think it does a pretty good job of rationalising a lot of what people tend to actually do under the aegis of decolonising the field (I usually see people try and change: (i) what is taught, and (ii) who does the teaching), since it is basically an attempt to leverage cognitive diversity in a way that tends to align with the various reform efforts now going on. Wiredu would, I think, also be of the opinion that this is a contribution to the broader project of decolonisation -- since the historic task of former colonies at this moment is to deal with the legacy of colonialism by taking the reigns of history and no longer simply having Western modes of life and government imposed, but rather consciously weighing the colonists mode of life against the indigenous tradition and attempting to forge a synthesis that allows for the best of both as far as is possible. That is to say, Wiredu's account of what African nations should be up to during periods of post-colonial modernisation looks a lot like his account of what African philosophy should be up to. He might therefore think that each can reinforce the other. If you do not agree with Wiredu on what broader cultural and political decolonisation means, I think one could reasonably fault this as failing to properly contribute to the broader decolonial project. I am not sure what I think the broader decolonial project will or should amount to, so I am agnostic on this point.

Ok here's the thing that Wiredu's account makes especially for me: I have never seen an account of decolonising philosophy that does not make it seem like it is just a generally desirable thing to do. I can understand why it is of especially pressing importance in departments in former colonies. But the thing Wiredu described just sounds like a corollary of enlightenment, assuming you don't a priori limit the capacity for interesting thought or concept creation to Westerners. (This would have to be a corollary of some version of the enlightenment that did not share the patronising assumptions of many actors within the actual historical enlightenment! Enlightenment itself is, of course, famously a concept subject to much critique.) Everyone, citizen or descendent of former (or presently) colonised nation or not, should want to decolonise philosophy in Wiredu's sense. Could we still claim our mantle as true philosophers if we, as a matter of policy, uncritically made use of our inherited concepts? Can we really vow to just set aside pertinent information about the limits or oversights of our own conceptual scheme, or are we so sure that there is no pertinent information to be drawn from the kind of cross-cultural comparisons which examination of various world traditions makes possible? In addition to Wiredu's aforementioned work, I am reading David Wong's Natural Moralities at the moment, which through its comparative approach between Confucian and western liberal ethics seems to be another proof in practice of the possibility of drawing pertinent information for philosophical puzzles from this -- and, by Wiredu's lights, is an instance of decolonising philosophy.

The only people I see talk about decolonising philosophy tend to be people from former colonies or right-on-lefties in the West. But when I read accounts of what decolonising philosophy would amount to it seems like anybody committed to the enlightenment ideals held by most philosophers should likewise find themselves engaged in full sympathy with this activity.

Remonstration

A recent conversation with some friends has me thinking about roles we can fruitfully play as philosophers of science. I just thought I'd write up on a blog post my thoughts on something that came out of that, which is a role we sometimes play that I feel is not often enough highlighted.

In philosophy we learn about tools and methods of critical thinking and argument construction and evaluation. For instance, a standard part of philosophical training is going through some basic logic. You should learn therein what it takes for an argument to be valid, and, going in the other direction, how one can demonstrate the invalidity of an argument by constructing counter-models. (If this doesn't mean anything to you, I will be going through an example later in this post!) That is just part of basic philosopher training. If you go into philosophy of science you will further specialise, perhaps learning about experimental technique, statistical methods, or theories of confirmation along the way. All of these can put somebody in a decent enough position to evaluate the cogency of arguments that scientists put forward, providing one familiarises oneself with the particular theoretical background the scientists one is evaluating are working within.

And it matters that scientists are making cogent arguments! Science has a lot of social cachet; with some well noted exceptions, folk trust scientists and will tend to believe claims that scientists put forward about the world. What scientists conclude is therefore deeply significant to our worldview and senses of self. Further, in many spheres of life we base policies on recommendations from scientific experts. Just the enterprise of science itself involves moving huge amounts of people and resources around, and the opportunity cost of having all these smart folk spend their time in this way rather than on other socially valuable tasks is itself huge. We want scientists to be basing their claims, recommendations, and activities, on sound argumentation and good reasoning, so as to ensure that this cachet and those resources are used as best we can.

So then putting these two together we get a natural thought about how philosophers of science should use our skills. We should monitor the arguments scientists make, and where we find that their methods or modes of argument are not capable of supporting the conclusions or recommendations they are making in light of those arguments, we should bring to bear our expertise in the evaluation of inferences or arguments (broadly construed) on calling this out and suggesting better practice for the future. (I recall reading, but do not recall where I was reading, E.O. Wilson once write that this is exactly what he thought of as the point of philosophers of science, people looking over his shoulder saying `Oh no I don't think this is good enough, what about such and such counter argument, eh?' He noted that while this could be pretty irritating in the moment, on reflection thought it valuable to him.) I call this kind of thing `remonstration', it's a kind of `speak truth to power!' norm, and I think we should see it as a valuable part of our mission as philosophers of science.

I am going to go through an example from my own work in a bit of detail below, but for some more illustrious examples one might want to check out: Clark Glymour's critique of the statistical reasoning that underlay the famous Bell Curve book and much of the rest of social psychology at the time, Nancy Cartwright's long running project critically evaluating the limitations of randomised control trials for medical or social research, or Roman Frigg's work (discussed, say, at the end of this excellent episode of the generally excellent Sci Phi podcast) on over-confident and over-specific claims made on the basis of models of climate change. 

But for an example of remonstration I am most familiar with (and also to allow me to explain and slightly reframe this previously published work of mine) I'd like to go through my paper On Fraud. One of the motivations for that paper was thinking about claims currently being made about how we should deal with the replication crisis in social psychology. Broadly, lots of claims in social psychology that were thought to have been securely established are being found not to stand up to sustained scrutiny when people attempt to replicate the initial experiments which led to their acceptance, or redo the statistical analyses with bigger/better data sets. In thinking about why this is occurring, a number of scientists have come to conclude that one (but not the only) source of the problem is -- scientists are not just seeking the truth for its own sake, but instead being encouraged to pursue credit (esteem, reward, glory, social recognition by their peers in the scientific community) by various features of the incentive structure of science. This pursuit of credit itself incentivises bad research practices, ranging from the careless to the outright fraudulent. If only we could remove these rival incentives which are causing the misconduct, and instead encourage pure pursuit of the truth, we'd have removed the incentive to involve oneself in such research misconduct. Since I had seen some very similar arguments come up before in my more historical scholarship on W.E.B. Du Bois, my interest was very much piqued and I got to thinking about whether this argument should be accepted as a sound basis of science policy.

I came to conclude that the psychologists and sociologists of science making these arguments were making a subtle mistake in how they reasoned about policy in light of scientific evidence. They were doing good empirical work tracing out the causes of much of the research malpractice we witness in science. But on the basis of this they were concluding that if we removed the actual causes of fraud we'd see less fraud. That is to say, they were establishing premises about the causes of fraud in the actual world, and concluding that a policy which intervened on (in fact removed or greatly lessened) these causes would mean that there would be less fraud after our intervention. After all, it's a natural thought; if X was what was causing the fraud and now there's no more (or much less) X, well you've removed the cause and so you should remove the effect, right?  Not so. Such arguments are not valid -- their premises can all be true, while their conclusion is false. So I constructed a counter-model, which is to say a model which shows that all of their premises can be true while their conclusion is false.

Without going into too much detail, I produced a model of people gathering evidence and deciding whether not to honestly reveal what evidence they received when they go to publish. Fraud is an extreme form of malpractice, of course, but it would do no harm to my arguments to interpret the agents as deciding whether or not to engage in milder forms of data fudging or other research malpractice. We can model the agents as pure credit seekers, they just want to gain the glory of being seen to make a discovery. Or we can model them as pure truth seekers, they just want the community to believe the truth about nature. (We can also consider mixed agents in the model, but set that aside.) In the model credit seeking can indeed incentivise fraud, and for the sakes of the counter-model we may grant that in the actual world all fraud is incentivised in this way. But what I show is that in this model, even if suppose that there were some policy that could successfully turn all scientists into pure truth seekers, it does not guarantee that there is less fraud -- in fact truth seeking can, in some especially worrying circumstances, actually lead to more fraud!

There is a general lesson here, in fact, that I wish I had done more to bring out in the paper. The point is: if you are basing policy on empirical research, it is tempting to think that what you need to know is whether the policy would be effective in the actual world. That, after all, is where you will be implementing the policy! But that's the wrong causal system for evaluating the effects of your proposed policy. What you need to know is whether the policy would be effective in the world (or causal system) that will exist after the policy is implemented. In the actual world -- sure, credit seeking is causing malpractice. But the fact that you remove that incentive to commit fraud does not by itself mean you've removed the incentive to commit fraud. It may be that in the world that exists after this intervention there are new temptations to commit fraud. Truth seeking itself may be one of them. Policy relevant causal information must include counter-factual information, information about the world that will exist after a not-yet-implemented policy has been carried out.

If you want the real details of my argument -- read the paper! But what I want to note here is how this is me trying to be the change I want to see in philosophy of science. I found some scientists making policy recommendations in virtue of their empirical research (in this case it was policy affecting science itself). I thought about the structure of their arguments, and realised they were making implicit assumptions about counter-factual reasoning. A general philosophy education gives you tools for reasoning about counter-factuals, so I could bring that to bear. What is more, general critical thinking (or logic) training that is part of being a philosopher points the way to counter-model construction as a means of critiquing arguments. Finally, disciplinary specific training in the philosophy of the social sciences gave me training in tools for building models of social groups, which was what was of particular relevance here. I was therefore able to remonstrate, to bring to bear my training in calling attention to an error in scientists' reasoning, and what's more an error that (since it was supposed to be the basis of policy) has the potential to be of some social and opportunity cost. I don't claim, of course, that this is the best example of remonstration in the literature (c.f. my illustrious colleagues above!) -- but I hope going through an example I am intimately familiar with in depth gives people a better example of how philosophy of science as remonstration is a good use of our disciplinary tools and expertises.

Now, it is certainly not my claim that only philosophers of science engage in this kind of remonstration. Statisticians very often engage in a very similar activity -- Andrew Gelman's blog alone is full of it. There is also a fine tradition of scientific whistleblowers who call foul when misconduct is afoot. Remonstrating with scientists whose reasoning has, for one reason or another, gone astray, ought not be, and fortunately is not in fact, left to philosophers alone. And, in case it needs to be said, nor is this (or nor ought this be) all of what philosophers of science get up to. Most of my own work, for instance, is not remonstration.

But when I see accounts of the tasks of philosophy of science they typically fall into one of three categories. Concept construction or clarification, where the goal is something like producing or improving a tool that it might help scientists do their job better. Scientific interpretation, where the goal is to do something like provide an understanding of scientific work that would make sense of the results of scientific activity, and tell us what the world would be like if our best evidenced theories were to be true. And meta-science, where the goal is to do something like provide an explanatory theory which tells us why it is that scientists reason (or ought to reason) in some ways rather than others. All of these can be valuable and I hope philosophers of science keep doing them. And I can even understand why people aren't keen to advertise the disciplinary mission of remonstration: it makes us into the stern humourless prigs of science, somewhat akin to Roosevelt's critic on the sideline hating on the folk actually getting stuff done. But, since I think it can be good and necessary, I hope that, even if it doesn't win us friends, and along with our comrades elsewhere in the academy and with our eye on the social good, we hold true to the mission of remonstrating against scientific overreach, malpractice, or just plain old error, where-ever we should see these arise.

The Diversity of Formal Philosophy

I've just come back from the Formal Epistemology Workshop! It was a lovely conference, and I highly recommend it to up and coming formal epistemology folk who want to get a sense of what's going on across the field. I was struck by the diversity of projects, and also by the interesting fact that multiple people said something like ``I feel like I am the least-formal formal-epistemologist here.'' So! I invented a taxonomy of projects in Formal Philosophy, which I'll present here with examples then comment on below.

About -- some formal system that touches upon matters of prior philosophical interest is either itself the object of study, or some feature of it or result therein is, or it is useful for stating/reformulating a prior philosophical problem. One does not reason within the system, but rather one either reflects on it, or some aspect of it, or draws out morals from it and reflects upon how they bear upon another problem.

Examples of work of this sort: Bertrand Russell's On Denoting, Plato's Meno, Audrey Yap's Idealization, Epistemic Logic, and Epistemology, Jason Stanley's Know How, David Lewis' On The Plurality of Worlds, Kenny Easwaran's Why Physics Uses Second Derivatives, Beall and Restall's Logical Pluralism, Danielle Wenner's The Social Value of Knowledge and the Responsiveness Requirement for Biomedical Research, Michael Weisberg's Who Is A Modeller?

Within -- the author(s) themselves use an established formal framework to prove results which are of philosophical interest. Perhaps they are taken to be interesting because of what they tell us about the formal system which is itself taken to be philosophically interesting, or how various such systems can be related, or perhaps because the result is itself intrinsically interesting or part of a family of results which collectively are taken to be interesting. The point is that Within projects gain whatever philosophical interest they have because of the relationship between a result the author has proven and something which is taken to be of philosophical interest.

Examples of work of this sort: Ruth C. Barcan's The Identity of Individuals in a Strict Functional Calculus of Second Order, Christian List and Philip Pettit's Aggregating Sets of Judgments: An Impossibility Result, Robert Stalnaker's On Logics of Knowledge and Belief, Catrin Campell-Moore's How To Express Self-Referential Probability, Cailin O'Connor's The Evolution of Guilt, Bertrand Russell's eponymous Paradox, the Marquis de Condorcet's Jury Theorem (and related results), Amartya Sen's The Impossibility of a Paretian Liberal, David Lewis' Probability of Conditionals and Conditional Probabilities, I. J. Good's On The Principle of Total Evidence, Harsanyi's Utilitarian Theorem (and related results).

Without -- the author invents or constructs a novel formal system that allows us to generate results or extract information about a new area of discourse not previously amenable to formal analysis, or which if there was a previous formal theory it took a markedly different form.

Examples of work of this sort: Aristotle's syllogistic, Russell and Whitehead's Principia Mathematica, Turing and Post on computability, Ruth Barcan's A Functional Calculus of First Order Based on Strict Implication, Kripke's A Completeness Theorem in Modal Logic, Carlos E. Alchourrón, Peter Gärdenfors and David Makinson's On The Logic of Theory Change, David Lewis' Convention, Frank P. Ramsey's Truth and Probability. Peter Spirtes, Clark Glymour, and Richard Scheines' Causation, Prediction, and Search.

Some comments on this, starting with remarks about the examples.

1 ) While I didn't put too much thought into constructing the example lists (which probably resulted in a demographic skew, alas, in what I highlighted -- on this see point (7) below) I did want to highlight a couple of points. Formal philosophy as a whole interacts with very diverse areas of philosophy and very diverse sets of formal tools. As has recently been discussed, logic gets the bulk of pedagogical attention in philosophy graduate programmes. But at a glance the above list contains work in ethics, social and political philosophy, metaphysics, philosophy of language, philosophy of physical, social, and biological sciences and mathematics, and epistemology. (I don't know of any formal aesthetics, but I would have liked to have been able to include that.) And the formal theories touched upon or deployed do indeed include logic, but also include probability theory, game and decision theory, statistical reasoning, calculus, social choice theory, and geometry.

2) I also wanted to use the examples to highlight that each of the sections contains work that would presumably be thought of as classical or canonical, as well as recent work by younger scholars.... This latter was a bit harder for the third section, since (for, I guess, reason discussed in (4) below) one hears about such work less. I decided that in the grand scheme of intellectual history though late 20th century is extremely recent philosophy, so it suffices to make my point. Which is that each of these modes of formal philosophy has shown itself both capable of making classic contributions and generating novel work. This is not a hierarchy of value, and none of these streams are yet dry. (When I reflected on my own work, I think I have some papers in the About category, and some papers in the Within section.) On the flip side, each of these is part of the grand tradition of formal philosophy, and there's no reason to think that some is more properly formal philosophy than the rest.

3) These are of course fuzzy categories. Gödel's incompleteness theorems are to some extent Within, but (if I understand the history correctly) the technique of Gödel-numbering was developed during these proofs and that was probably a significantly novel enough contribution to be its own Without work. For more recent work, I wanted to include more ethics, but couldn't decide whether this was About or Within. Is this Within or Without? Nothing, I think, really turns on subtleties here, I just wanted to acknowledge that there are plenty of edge cases. However, just to give people something to disagree with me about: I hereby claim that while this is fuzzy in the sense that some work can plausibly be in multiple categories, anything that could be called Formal Philosophy will recognisably fit into at least one of the categories.

4) While I don't think this is a hierarchy of value, my sense is that in terms of credit or repute the Without category is the high-risk high-reward category. It's the kind of work that is most likely to fail, but most likely to secure one's lasting glory if one can pull it off.

5) Work in the About category is probably the easiest sell to philosophers who don't work in formal philosophy. When formal philosophers are designing introductory lectures, outreach-y summer programmes, presentations for conferences in which there will be mixed company, or just in general interacting with a field that can be territorial and sceptical about things which fall outside the recognised boundaries, I think there is some reason to be cognisant of the distinction between About and Within work, and opt for About work. Nobody is going to argue that Plato's Meno isn't real philosophy.

6) I am less confident here, but I know there are metaphilosophical debates about what counts as experimental philosophy. I feel like a similar taxonomy would work there, with experimental philosophy being work that either is centrally based upon reflections on empirical work, is founded upon novel discoveries made by the authors, or comes up with a new way of testing things or generating results.

7) I didn't max out on demographic diversity in constructing the example lists, since it wasn't really to my point here. But I did find when making the lists that white blokes came to mind much more easily in all of the categories, and I guess especially dear to my heart given previous work -- I could scarcely think of any black folk! On reflection I can think think of more I didn't include -- for instance Kwasi Wiredu's Logic and Ontology (I cannae find a link!) could have gone in the About section, and I just met Lisa Cassell at the Formal Epistemology Workshop that sparked this very post. Still, even as I try hard not many brothers and sisters come to mind. This does not tell one much about the actual demographics of the field -- maybe I am just bad at remembering people, and I am myself very much trained in a certain tradition. But it is what it is.

On The Conceptual Penis

Another day, another hoax paper published! For those who don't know, these hoax papers pop up every now and again, wherein academics deliberately write nonsense, submit it to an ostensibly serious journal, and find that lo and behold they can get it published, despite the checks that are meant to ensure only rigorous scholarship makes it through to the published literature. This is a worry, because in theory being part of the scholarly literature is meant to be a guarantee of quality and a sign that the scholarship can be relied upon.

It's always pretty hard to know what to conclude from these hoax papers -- I think it is over-determined that one should not, ever, take the fact that an individual paper has passed peer review to mean its conclusions are secure. Where one wants to be guided by academic work, one should form one's opinion on a matter based on a review of the literature, not just one or two papers therein. Too many journals reviewing too many papers by too many authors in too many fields -- all run by fallible humans. Even with the best of intentions (which is far from guaranteed) there is no way the peer review process could filter out all bad work, and by dint of the sheer size of the enterprise there is going to be plenty of rubbish out there even if a low proportion of the bad papers make it through. What's more, the replication crisis shows us that things can actually go pretty systematically wrong, and there's some theoretical reason to be a bit pessimistic about the quality of the average paper. So, one read of hoax publications is that they just dramatically illustrate this point. Whatever guarantee of quality the scholarly literature and its processes of review and double checking provide, it does not do all that much at the level of individual papers -- look to the collective beliefs or consensus arising out of such literatures, if there is reliable results to be found anywhere it will be therein.

The authors of this particular hoax, however, want a different conclusion to be drawn. They think their paper shows that a ``problem lies within the very concept of any journal being a “rigorous academic journal in gender studies.”'' Their reasoning is that their article sailed through peer review because it ``portrayed a moralizing attitude that comported with the editors’ moral convictions'',  namely ``an overriding almost-religious belief that maleness is the root of all evil''. Basically the paper kind of strings sentences together which loosely suggest that maleness is tied to a certain kind of abstract idea of the penis and this abstract idea of the penis is responsible for a great many social ills, inclusive of climate change. It really is largely nonsense, but an overall effect is conveyed and which is indeed suggestive of the almost religious belief they note. They think that it was because this effect was conveyed that they were not subject to serious peer review.

Predictably enough (pious left wing academical that I am!) I am not convinced by their account of what happened here. I'll set aside the fact that some of what the paper suggests would probably be considered morally offensive by the sort of people they have in mind -- it's reading tea-leaves to try and work out what the paper is really suggestive of, since it is, by design, genuinely nonsensical.  Rather, my objections to their account are as follows. First, boringly, I just don't think journal hoaxes provide the kind of evidence that could support their conclusion. A paper published in a journal is just poor evidence for conclusions concerning the practices of an entire field. Second, this paper was actually first rejected by NORMA: International Journal for Masculinity Studies! Why should the one accept count for more evidence of typical field practices than the one reject? Especially when you consider that, third, a relevant difference between the journal that accepted (Cogent Social Sciences) and the one that rejected was that the journal that accepted is a pay-to-publish journal. To my mind, we already have some reason to be highly suspicious of pay-to-publish journals. If one searches around one will find plenty of controversies about these journals, just going to link to a story about my favourite such controversy here. So, given where my priors are at, once I found out this detail it immediately suggested an explanation of what went wrong here! Personally, I am sufficiently suspicious of this business model that I think we as an academic community should institute some communal norms against publishing in pay-to-publish journals, and demand that any article that is published in such a journal is suitably marked as such and refuse to credit work so marked -- in fact, given the wider social role of academic work, I wouldn't even be opposed to legislation banning the business model.

Now, the authors do spend some time discussing the third of these worries. They acknowledge that indeed pay-to-publish journals might be a source of the problem. But they respond that, (A), NORMA's editor actually recommended transferring their paper to CSS in such a way that helped it get through the review process there, suggesting that even serious academic journals are in cahoots with these mobsters. (B), CSS did seem to actually implement peer review, (C), the journal is published by the apparently respectable Taylor and Francis group. These considerations move them to think that the problem is not mainly a general one with pay-to-publish journals, but rather the field. Again, I am not convinced. Regarding (B) -- editors have a lot of power in who they pick as reviewers, so they can just pick hacks to ensure things sail through, and in any case we don't know how the reviewers were themselves incentivised. Regarding (C), let's just say I am not too impressed with academic publishing groups more generally.

Regarding (A)... ok here I agree with the authors of the hoax paper. In so far as journals in gender studies (and for all I know this happens in other fields too) are collaborating with the pay-to-publish journals in this way, they are undermining their own field's scholarly standards. I didn't know this sort of collaboration existed, so for me this is the main thing I have learned from the incident, and I shall be looking out for this in future; maybe it occurs in philosophy too? The overall lesson I draw from this latest hoax is that as a community we should stop collaborating with these sharks, their invocation of the profit motive undermines the scholarly values we strive to maintain and represent, and it is only by our continued participation in the system that they are able to so destroy us.