Lessons of the Funnel

We Are Being Ruined by Best Efforts

We are going to conduct a simple experiment. We won’t actually do the experiment here, but I will describe it to you and you will understand the implications.

Imagine that we have a funnel, a stand to hold the funnel about a half meter above a table, and on the table we have a target.

Imagine that we drop a marble down the funnel. The marble will roll down the funnel in a random fashion, regardless of how we might make the drop. The marble then falls from the bottom of the funnel and we mark the spot on the table with a pencil. Simple.

We will follow certain rules for aiming the funnel over the target. These rules correspond to decision rules we make in running equipment, processes, and systems.

- Dr. W.E. Deming as quoted by Latzko, William J. and Saunders, David M. Four Days with Deming. (p. 150)

SO begins the lessons of Dr. Deming’s Funnel Experiment, a thought-exercise inspired by Lloyd S. Nelson of Nashua Corp. that he used in his seminars and books to explain what happens when we over-react to normal, routine fluctuations in our processes and attempt to control them with well-intended yet un-aided interventions or adjustments. Deming referred to this as tampering, the consequence of not knowing about variation nor its causes and subsequently misinterpreting statistical noise (common-cause) for signals to act (special-cause).

Four rules govern the experiment, each an analog to a reactive action a manager might take in response to common-cause variation in a process or system. Let’s look at each, along with some examples of situations they describe.

Rule 1: No Adjustment

We begin by establishing a baseline for our system by aiming the funnel directly over a target and dropping a marble through the apparatus 50 times, marking where the marble comes to rest each time. We might observe a pattern with marks clustered in a circular pattern as shown below.

Question to Aspiring Leaders and Managers: How can we do better and land more drops closer to the target?

Rule 2: Adjust Funnel to Last Position

A suggestion is made to compensate for each drift off target by adjusting the funnel in equal and opposite proportion to its last position. For example, if the marble rests 15cm north-west of target on drop one, the funnel would be moved 15cm south-east; on the second drop, if the marble stopped 22cm north-east, the funnel would be moved 22cm south-west of its last position. After 50 drops we see a larger cluster of marks emerge: We’ve managed to double the variation!

Question to Aspiring Leaders and Managers: Why did this rule fail to correct for the drift? Why are we continuing to over/under shoot the target but to a larger degree?

Examples:

• Punishing/rewarding individual performance;

• Management by surveys / governing by polls;

• Adjusting budgets according to amount spent in prior period;

• Adjusting estimates by adding/removing padding based on prior actuals:

Rule 3: Adjust Funnel to Original Target

An observation is made that perhaps the wide variance is a result of chasing the last marble drop with each adjustment when really we need to focus on distance from the original target. The method proposed is to re-aim the funnel in equal and opposite proportion to the distance the marble rests from the target. Thus, if on drop one the marble stops 15cm north-west of the target, the funnel is moved 15cm south-east of the target; if the next drop stops 22cm north-east of the target, the funnel is moved 22cm south-west. After another fifty drops, an even larger pattern of marks emerges in the shape of a bow-tie, with none landing anywhere near the original target. Our system is exploding!

Question to Aspiring Leaders and Managers: What happened here? Why did this rule expand the drift of each drop so significantly? How can we control this?

Examples of Rule 3:

• Escalating rules and countermeasures in response to changes in a system state;

• Trade barriers, tariffs, sanctions;

• Hiring/firing staff according to shortages/surpluses;

• Win/lose competition

• Management by Results, eg. compensating feedback loops, fixes that fail system archetype.

Rule 4: Adjust Funnel to Last Drop

It’s apparent that despite our best-efforts to improve our accuracy by compensating for each marble drop using Rule 2 and 3, we’re worse off than our baseline under Rule 1. We finally resolve to move the funnel to wherever the last dropped marble comes to rest, with the theory that perhaps we can improve our accuracy by going where the marbles tend to cluster - it might lead to staying closer to the target rather than swinging this way and that. After another 50 drops we are horrified to see the marks wander all over the place and eventually off the grid entirely.

Question to Aspiring Leaders and Managers: Why did this occur? What assumptions were made that contributed to this outcome?

Examples of Rule 4:

• Blind copying of examples without the aid of understanding supporting theory;

• Benchmarking;

• Worker training worker in succession without the aid of standards;

• Using past work product as a template for future work, in succession.

Example of an Antidote:

• Chili pickers for McIlhenny Co., makers of the world-famous Tabasco Red Sauce, use a small, red-painted dowel to gauge the ripeness of peppers called a “petit baton rouge” - a simple means of reducing variation in their product.

  

Summary

I think the lesson of the funnel experiment can be summarized by the inimitable Dr. Russell Ackoff who once said “the more efficient you are at doing the wrong thing, the wronger you become.” With every action we took to correct the “normal” behaviour of the experiment’s system, the worse off we became - no matter how clever we thought we were. This was due to misinterpreting symptoms as signals for action, which increased the variation and instability of the system, triggering cycles of destabilizing actions/reactions in a vicious cycle.

As we discussed in the post on variation, the path out is to use statistical analysis to separate true signals from random noise in the data of the system to restrict our interventions.

Reflection Questions

Consider the experiment in totality: What are the components of the system? What is the aim? What are the sources of common-cause variation in the system? What are the special-causes of variation? Knowing this, how could the system be improved to increase accuracy? Is it really possible to aim the funnel at the target? Why or why not?

What examples of Rules 2-4 can you think of from your past experience and observations? What were the outcomes? How were they corrected?