#NoHeroes

I am opposed to meritocracy, and the institutionalised celebration of genius. We should not allocate rewards and positions of institutional power simply on the basis of who has the most (domain appropriate) talent, or merit, or skill, at whatever task the institution at hand concerns itself with. I am opposed to these things quite generally, but being a social epistemologist by trade I focus especially on the harms and burdensome opportunity costs these things impose on communities of inquirers. What follows are some papers or essays which I think support me in this attitude as regards to social epistemology, roughly arranged by theme and with a brief description for each. Note that in all cases this is my personal take on what the papers teach us: the authors cannot be held responsible for my spin on their work!

Theme 1: we are not good at detecting and rewarding merit

Academic Superstars: Competent or Lucky? - Remco Heesen: argues that even if we make assumptions quite favourable to the defender of meritocracy in science, we are still not in a position to say whether or not our evidence of past academic performance tracks luck or genuine competence, and if it is the former then social stratification according to past performance may be epistemically harmful to science.

Bias in Peer Review - Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang, and Blaise Cronin: good run down of the historical and sociological evidence concerning the various ways in which scientific peer review processes systematically fail to pick out the best work.

Decision Theoretic Model of the Productivity Gap - Liam Kofi Bright: argues that even if we assume equal talent and time to allocate to work, background social inequalities informing expectations of uptake and scrutiny of one's work can still lead to a gap in the production of papers by men and women, and thus differential access to reward and esteem which tracks gender rather than talent. (Pairs well with a number of papers: this by Carole J. Lee which looks in more detail at how said background inequalities can inform expectations in the academy, and this by Erin Hengel which uses a more empirical approach to come to a similar conclusion. Also worth noting that the framework the Decision Theoretic Model paper is developed in is drawn from the `analytic egalitarianism' model of Sandra J. Peart and David M. Levy, which is at least morally similar to the whole #NoHeroes thing and is discussed (among other places) in their book The ``Vanity Of The Philosopher''.)

Dynamics and Diversity in Epistemic CommunitiesCailin O'Connor and Justin Bruner: argues that even if we assume no difference in skill level or merit of any sort, background social inequalities can still lead to members of minority groups being under-rewarded for their participation in scientific research. (Pairs well with Discrimination and Collaboration in Science by Cailin O'Conor and Hannah Rubin which argues that where epistemic communities learn to discriminate one may well see demographic segregation which tracks research area, and that such segregation in turn impedes the spread of new and interesting ideas among scientists.)

Epistemic Diversity and Editorial Decisions - Remco Heesen and Jan-Willem Romeijn: argues that even if journal editing practices were an impartial attempt to pick out the best research, they would still tend to be unfairly biased against less prestigious or less well established research groups, denying them access to the means of prestige and recognition and thereby harming scientists' attempts to secure a beneficial epistemic diversity.

Expectations of Brilliance Underlie Gender Distributions Across Academic Disciplines - Sarah-Jane Leslie, Andrei Cimpian, Meredith Meyer, and Edward Freeland: I went back and forth as to whether and how to classify this here, but broadly speaking their results suggest that the more people think that a certain kind of native brilliance underlies success in a field, the lower the proportion of women there is among those who work in that field. This is a result which has generated a lot of discussion which is still on going, but it is at least suggestive of the position that something about the way in which people currently understand merit as working in a number of fields is presently having a merit-neutral effect in putting people off participation. Whether it is more than suggestive in this regard, or has a deeper relationship to meritocracy, awaits further investigation.

Huygens's 1688 Report to the Directors of the Dutch East India Company on the Measurement of Longitude at Sea and the Evidence it Offered Against Universal Gravity - Eric Schliesser and George E. Smith: charming case study illustrative of a general point (drawn out by Schliesser here) that little known and oft-forgotten figures can play a decisively important role in driving scientific success forwards.

Why was M. S. Tswett’s chromatographic adsorption analysis rejected? - Jonathan Livengood: another nice case study of a minor figure making a very important contribution to the progress of science, in this case with the added bonus (for my purposes!) of meddling figures, high up the prestige hierarchy established by past achievement, somewhat delaying acceptance of the innovation. (Though Livengood argues that the role of this has been overhyped in some of the standard histories.) The more such case studies the better!

The Birth of a New American Meritocracy - Matthew Stewart: really good write up about the ways in which the purportedly meritocratic system sustaining America's labour aristocracy (within which are the professoriate, culture and tech industry workers, and scientists) is really anything but.

The Matthew Effect in Science - Robert K. Merton: argues that the scientific reward system has historically shown a tendency to over-reward those who were initially more famous or prestigious in cases of co- or simultaneous- discovery. This leads to a cumulative advantage effect, where the rich get richer, and generates inequalities in prestige or status quite not reflecting the degree of difference in talent or merit. (Pairs well with this paper, by Andrew TomkinsMin Zhang, and William D. Heavlin, which has a neat experimental design to illustrate the Matthew effect at work in the selection of manuscripts for a prestigious computer science conference.)

Power, Bargaining, and Collaboration -  Justin Bruner and Cailin O'Conor: the kind of bargaining situations that occur in science may well lead to worse results in terms of esteem or credit for work received for less prestigious or socially powerful agents, independently of their actual contribution to the work. Our esteem generating mechanisms can thus predictably fail to track actual role performance.

White‐washing the Canon: ‘Minor’ Figures and the History of Philosophy - Beverley Southgate: uses the decline in status of Thomas White (1593-1676) to discuss the general issue of historical remembrance. Considered highly eminent in his day and now barely remembered, the tale of White's fall from historical grace illustrates the fairly arbitrary factors (e.g. political and religious fashions turning against him, the desire by teachers to have clear exemplars speaking against his more nuanced positions, a writing style that never received as many apologists as Kant or Hegel)  that prevented his inclusion in the canon. Ends with a plea to drop the whole notion of `great' figures as the proper focus of intellectual history.

Productivity, Prominence, and the Effects of Academic EnvironmentSamuel F. Way, Allison C. Morgan, Daniel B. Larremore, and Aaron Clauset: argues, on the basis of novel experimental work, that it is the work environment in fancy universities, rather than something inherent selected for by the job market, that explains why academics at fancy universities are more productive. (Pairs well with this paper by Clauset, Larremore, and Samuel Arbesman which had previously just noticed the fact that prestige hierarchies track faculty productivity.)

status quoi? - Simine Vazire: a series of short interviews inspired by Vazire's observation that eminence is an unreliable marker of wisdom. She specifically sought out junior or early career researchers who she thought would offer interesting or novel insights on the problems with their fields' prestige hierarchies. Interesting stuff!

Theme 2: even if we could detect and reward merit better institutions would not focus on this.

A Role for Judgement Aggregation in Coauthoring Scientific Publications - Liam Kofi Bright, Haixin Dang, and Remco Heesen: while it is not the central focus of this paper, part of what is argued therein is that, in order to be consistent with widely agreed upon and established norms of scientific behaviour, decisions about what to publish in a scientific paper should be decided in a manner which gives all participants, regardless of rank, prestige, or prior accomplishment, an equal say or equal opportunity to veto.

Does Science Need Mavericks? - Adrian Currie: this essay considers whether or not science needs a class of people whose special task it is, in virtue of their clear sighted original thinking and willingness to buck the trends and fashions of their era, to disrupt the theoretical status quo and try out bold new ideas. Argues that while performing such tasks is important, the notion of their being special `maverick' individuals tasked with performing them fails to pay attention to preferable alternative means of designing social and institutional mechanisms that can ensure this work is done in a more diffused and evenly distributed manner.

``Excellence R Us'': university research and the feitishisation of excellence - Samuel Moore,
 Cameron Neylon, Martin Paul Eve, Daniel Paul O’Donnell, and Damian Pattinson: run down of various pieces of evidence suggesting that the internationally prevalent focus on achieving `excellence' at research universities has largely been detrimental to the actual business of running a university, and what is more often promotes the precise opposite of that which it claims to - encouraging fraud and all manner of epistemic vice.

The Independence Thesis - Conor Mayo-Wilson, Kevin J.S. Zollman, David Danks: argues that a community composed entirely of agents who all follow the best strategy for an individual learner to come to the truth can be outperformed at a truth seeking task by a community which includes agents who do not follow the individually optimal strategy. (See here for related formal results by the same authors, and the paper pairs well with Scientific Rationality and Human Reasoning by Miriam Solomon which comes to a similar conclusion via examination of a case study paired with results from social psychology. This seems as good a place as any to mention Paul Feyerabend's Against Method which surely deserves mention on this list as some kind of spiritual precursor)

Economists as Experts: Overconfidence in Theory and Practice - Erik Angner: argues that various facts of social psychology and institutional arrangement make it likely that if economists are asked to act as experts, transferring their prestige in the academic to actual political power beyond, they are liable to overconfident, with potentially devastating affects - backs this up with a case study of post Soviet nation building.

Experiment by Industrial Selection - Bennett Holman and Justin Bruner: not the main focus of the paper, but notably concludes that even a fairly administered genuinely meritocratic scientific system could fail to protect us from corporate interests biasing the scientific process towards certain results; more a note on what meritocracy or heroes in power can't do for us.

Groups of Diverse Problem Solvers Can Outperform Groups of High-Ability Problem Solvers - Lu Hong and Scott E. Page: very famous model arguing that groups which are diverse in terms of their approach to problem solving can outperform a group of high ability problem solvers... it's a very informative title? Not actually my favourite work in this genre, but if you are going to read in this area you should know this paper, and in any case whatever I think of this particular model Hong and Page are good economists who work on related topics and you should follow the links back to their papers and explore.

The Limited Effectiveness of Prestige as an Intervention on the Health of Medical Journal Publications - Carole J. Lee: pairs well with the Holman & Bruner, where the former paper used a model to argue that even if we assumed a functioning meritocracy we can't necessarily protect ourselves from the bad effects of industry, this paper uses evidence from the actual performance of medical journals to argue that the allure of attaching prestige to work of higher epistemic standards has not been all that successful in actually protecting the field against bad work.

On Fraud - Liam Kofi Bright: not the main focus of the paper, but along the way this paper argues that social stratification, which presumably any version of meritocracy or the institutional celebration of genius must involve, can directly incentivise fraud by the most influential among scientists, under the assumption that such scientists are concerned that their peers come to know the truth. (Pairs well with this paper by John Earman and Clark Glymour, which describes behaviour by Arthur Stanley Eddington leveraging his social position within the scientific community to garner support for a theory he was antecedently convinced of but which his evidence didn't quite confirm that is certainly consistent with the model of the influential scientist engaging in noble lies.)

Optimal Publication Strategy - Kevin J.S. Zollman: the results of this paper defy easy summary, but suffice it to say that it explores a variety of situations in which selecting the best papers to publish is not the epistemically optimal strategy for a journal editor.

Science Funding is a Gamble so Let’s Give Out Money by Lottery - Shahar Avin: exactly what it says on the tin. Shahar actually has a lot of work on this so I would recommend checking out his website and seeing what else you like there. Pairs well with this nice use of the economic theory of competitions to study funding mechanisms by Kevin Gross and Carl Bergstrom, which finds that "proposal competitions are inevitably and inescapably inefficient mechanisms for funding science when the number of awards is small. The contest model presented here suggests that a partially randomized scheme for allocating funds — that is, a lottery — can restore the efficiency lost as paylines fall, albeit at the expense of reducing the average scientific value of the projects that are funded."

Is Peer Review a Good Idea? - Remco Heesen and Liam Kofi Bright: runs through various arguments to the effect that pre-publication peer review isn't a good way of working out what science should be publically disseminated, and we'd do better to just allow everything to put up on archiv and let the community sort through what sort of work it prefers. Lovely accessible run through of the issues from Remco here, and at its heart this is a basically Condorcet-esque point so related to the strengths of the Condorcet Jury Theorem discussed in the book under review here.

Judgement aggregation in scientific collaborations: The case for waiving expertise - Alexandru Marcoci and James Nguyen: the paper argues that the practice of deferring to expertise during collaborations (which I take to be at least related to setting up meritocratic hierarchical structures) can actually make it impossible to successfully aggregate the judgements of the collaborating scientists. Instead, people should at least sometimes waive their claim to expert deference.

Better than Best: Epistemic Landscapes and Diversity of Practice in Science - Jingyi Wu contains a neat argument to the effect that a community of inquirers who always make use of the best methods available can be outperformed by agents who actually randomly adopt methods to address their problems. It seems to me that an emerging theme from all this is that randomisation has, at scale, significant advantages over the meritocratic model of inquiry.
 
(A lot of the previous works presuppose a degree of familiarity with the sociology and economics of science. I think they should all be such that you can follow them without that, but it feels incomplete to not include some references to background reading. So for thems that are interested, here are some book length background on the social reward system of science, how it creates and sustains hierarchies, and how these relate to the distribution of resources: How Professors Think, by Michèle LamontSocial Stratification in Science, by Jonathan Cole and Stephen ColeMaking Science, by Stephen Cole, Creativity in Science, by Dean Simonton, and Laboratory Life by Bruno Latour and Steve Woolgar. A book length reflection on some of the philosophic upshot of all this to be found in Science as a Process by David L. Hull.  For some essays on the general economic reward system of science I would see this paper by Partha Dasgupta and Paul A. Davidthis paper by Paula Stephan, and this paper by Kevin J.S. Zollman is then a good introduction to the epistemic significance of this economic tradition, and here are some links maintained by Shen-yi Liao on related matters concerning the distribution of funding for research projects. I would also recommend some familiarity with the Condorcet Jury Theorem, for which you may as well start with this paper by Christian List and Robert E. Goodin.)

Meritocracy was once a progressive force. It represented an important rebellion against the principle of heredity nobility. I recognise that even now it is not the worst way of organising ourselves, and does lead and has led to real improvements in our mode of life. But its historical moment has passed, and if we are to make further progress in and through the efficient arrangement of our social matters we must jettison this outmoded governing ideal. An important task for social epistemologists going forward is the articulation of a rival ideal which can allow us to enjoy meritocracy's benefits while avoiding these pitfalls so far as is possible.

Comments

  1. Thank you for compiling this very useful list, I was thinking a lot about meritocracy and even considering writing a book on it until I started thinking deeply about its underlying problems, conceptually and philosophically. I chanced upon your profile from the Daily Nous blog, which reported you have won a prize. Congratulations.

    ReplyDelete
    Replies
    1. Thank you for the kind words, though I suppose there is some irony in acknowledging a prize on this page!

      Delete

Post a Comment

Popular posts from this blog

How I Am A Marxist

On Not Believing In One's Work

Arguments in Philosophy