HOISTED FROM THE ARCHIVES: For a True Meritocracy
Hoisting this & separating it out so I can find it easily in the future: from 2021-12-28.. <https://braddelong.substack.com/p/against-meritocracy>.
If you have a test or another activity to select who is “the best”, you had better make damned sure that the test or activity accurately measures or reflects what you really want done. In the limit, you optimize for test-takers—and people who have devoted all of their time to become expert test-takers have spent none of their time learning to do anything else. They are thus likely to be far from “the best” at what will be their real job.
How can you tell if you are falling into this trap? Divide your population up into groups. See how well the best test-takers are spread out among those groups, If there is no reason to think that those who will actually be best at the real job are anything other than uniformly distributed among the groups you have chosen, and yet if it turns out that those who score highest on the test are strongly concentrated—then it is highly likely that you are missing huge numbers of potential candidates, and wasting a huge amount of talent.
Rajiv Sethi ran across a very interesting paper on Finland that is on point here:
Rajiv Sethi: Notes on a Remarkable Finding from Finland <https://rajivsethi.substack.com/p/notes-on-a-remarkable-finding-from>: ‘Ursina Schaede and Ville Mankki…. A change in the manner in which… teachers were selected…. A first stage based largely on scores on a high school matriculation exam, followed by a second stage…. For a number of years, acceptance into the second stage was based on a quota, ensuring that at least 40 percent… were male…. The first post-quota cohort… graduated in 1994…. Students differentially exposed to the quota-constrained cohorts of teachers ended up with better educational attainment and labor force participation…. “Male quota teachers contributing positive qualities to the school environment that are not sufficiently captured by the selection criterion in absence of the quota.”… Important characteristics (unmeasured by scores) were not identical across male and female applicant pools. The quota was picking up individuals with these characteristics by proxy…
