Mini Case Study: The Dictatorship of Papers
How The Prevailing Style of Management Has Distorted Academia
Ranking is a farce. Apparent performance is actually attributable mostly to the system the person works in, not to the individual himself…
Ranking creates competition between people, salesmen, teams, divisions. It demoralizes employees.
Ranking comes from a failure to understand variation from common causes…
The so-called merit system introduces conflict between people. Emphasis goes to achievement of rank, merit, not on the work. The merit system destroys cooperation.
— Deming, Dr. W.E. The New Economics. 3rd Ed (p. 19), 2nd Ed. (pp. 25-26)
SOMETHING DIFFERENT for you today: How can the prevailing style of management ruin academia? According to academic YouTuber to the stars, Dr. Andy Stapleton, it begins with a simple metric: the h-index. Originally intended as an impartial method for quantifying the quality of contributions scientists make to their field, it has unsurprisingly been seized upon by administrators as a tool for rating and ranking and exclusion, which Stapleton argues is ruining academia with a “dictatorship of papers”.
On viewing his video, I knew I had good material for a short Deming analysis case study on the h-index and the effect it was having in academia. As it turns out, Dr. Stapleton does much of the heavy lifting for us when it comes to the symptoms, but we will need a backgrounder on what the index is and how it works to contextualize it in our study.
Let’s get stuck-in, shall we?
tl;dr
Former academic researcher turned YouTuber, Dr. Andy Stapleton, explains in a recent video how a scheme to rate and rank scientists impacts based on a recently-invented metric, the h-Index, is destroying academia by dividing colleagues into camps in fights over access to funding, promotions, and fame. He calls this the “dictatorship of papers”, but through a Deming lens is another example of the tyranny of the prevailing style of management brought about by an assortment of faulty practices with predictable outcomes.
What is the h-Index?
Stapleton’s video begins with question, “What if I was to tell you there was one single number that would ruin and dictate people's academic careers [and] we’ve done it to ourselves?”. Well, as a student of Deming, I’d say: “Color me surprised”, but I digress.
That number is called the h-index, and it is the creation of one Jorge E. Hirsch, Prof. of Physics at the University of California, San Diego, who details it in his 2005 paper, An index to quantify an individual’s scientific research output. (Your Deming senses should already be tingling…)
An established academic in his field, Hirsch had observed a long-standing problem in the way an individual’s scientific contributions were measured when competing for grants, promotions, or recruitment opportunities: either by productivity (total number of papers published) or impact (total citations garnered). This tended to give an unbalanced assessment, favouring candidates who could either rest on their laurels as long as they got citations, or crank out low-impact papers in volume. He resolved this imbalance by combining the two KPIs (well, that’s what they are!) into one: the h-index.
The method he used is simple:
Arrange an author’s papers in descending order by total citations, numbering each starting at 1;
Working down the list, the h-index is the last publication number that is less than or equal to the count of citations.
So, for a hypothetical researcher who published ten papers, we might see the following by applying Hirsch’s method to their publications:
As shown, the researcher’s h-index would be 5 as it is the last publication number that is less than or equal to the count of citations. You can probably guess how this is going to go sideways…
Hirsch applied his method to physicists who won the Nobel Prize over a twenty year period, discovering that a significant majority (84%) had an h-index of at least 30, indicating they had published a significant body of broadly-cited papers over their career before winning. Incidentally, Google Scholar shows Hirsch’s h-index at 70 overall and 28 since 2019, so that Nobel should be just around the corner.
As of this writing, Stapleton’s h-index is a respectable 15. No Nobel predicted for him…
Keep reading with a 7-day free trial
Subscribe to The Digestible Deming to keep reading this post and get 7 days of free access to the full post archives.