Theory and Prediction
Management In Any Form
Theory of management now exists. There is now a theory of management for improvement of quality, productivity, and competitive position…
Experience alone, without theory, teaches management nothing about what to do to improve quality and competitive position, nor how to do it. If experience alone would be a teacher, then one may well ask why are we in this predicament? Experience will answer a question, and a question comes from theory. The theory in hand need not be elaborate. It may be only a hunch, or a statement of principles. It may turn out to be a wrong hunch.
Deming, W. Edwards. Out of the Crisis (MIT Press) (p. 19). The MIT Press. Kindle Edition.
Theory of Knowledge:
Management is prediction. The theory of knowledge helps us to understand that management in any form is prediction. The simplest plan—how may I go home tonight—requires prediction that my automobile will start and run, or that the bus will come, or the train.
Knowledge is built on theory. The theory of knowledge teaches us that a statement, if it conveys knowledge, predicts future outcome, with risk of being wrong, and that it fits without failure observations of the past…
Theory is a window into the world. Theory leads to prediction. Without prediction, experience and examples teach nothing. To copy an example of success, without the aid of theory, may lead to disaster.
Any rational plan, however simple, is prediction concerning conditions, behaviour, performance of people, procedures, equipment, or materials…
A statement devoid of rational prediction does not convey knowledge.
No number of examples establishes a theory, yet a single unexplained failure of a theory requires modification or even abandonment of the theory.
Deming, W. Edwards. The New Economics, 3rd ed. (pp. 69-71).
The 14 points of Chapter 2 constitute a theory of management. Their application will transform Western style of management.
Deming, W. Edwards. Out of the Crisis (MIT Press) (p. 97). The MIT Press. Kindle Edition.
It is a hazard to copy. It is necessary to understand the theory of what one wishes to do or to make. Americans are great copiers (QC-Circles, Kanban or just in time, for example). The fact is that the Japanese learn the theory of what they wish to make, then improve on it.
Deming, W. Edwards. Out of the Crisis (MIT Press) (p. 129). The MIT Press. Kindle Edition.
TODAY’S POST is inspired by some recent conversations that have revealed to me that we require an operational definition for what is meant by “theory” in the context of transforming the management of organizations. It was put to me, not for the first time, that management customers aren’t interested in theory or anything theoretical, but concrete, actionable, directive guidance. Or, as it was less eloquently said to me by a senior manager a couple of years ago: “Look, I know you’re trying to teach me how to fish, but sometimes, I just want the goddamned fish.”
Indeed.
The impatience to learn and desire for quick results and instant pudding is as entrenched now as it was in Dr. Deming’s time. It’s no wonder that we see many naive management interventions and changes that improve little but disrupt many.
How Do We Define Theory?
In the context of the coaching and consulting work I do to re-orient managers with the real aim and purpose of their work, I have found a common misunderstanding that theory means dry, detached, academic expositions that have limited application in reality, as suggested by the old, hoary chestnut of “that may work in theory, but…”. I have found it useful to re-cast theory as much more active and pervasive, per Dr. Deming’s guidance in the above excerpts:
Theory is the prediction of future consequences of actions (or inactions) we or others enact which carries a risk of being wrong in the past or present. Thus, management in any form is prediction.
Experience Is Not Always The Best Teacher
So defined, we can push back against the well-worn and insufficient bromide that “experience is the best teacher”. As the good Dr. Deming well-explains, if we haven’t made a prediction prior to the experience we learn nothing, or deceive ourselves by retconning the results (“I knew that would happen…”). Ergo, poorly-defined and predicted interventions into organizational systems (ie. tampering) produces limited understanding from the resulting feedbacks.
Instead, we might choose to ask what did we expect to happen as a result of our actions. What basis do we have to make such a prediction? How will we know whether our actions contributed to the result and not another phenomena? Continuing down this path of inquiry will inevitably lead to preparing a test using a method such as a Shewhart/Deming PDSA cycle or Toyota Kata Target Condition.
How I Use Theory
It is not unusual in my line of work to be called in for help after a decision has been made to use this or that framework or methodology to promote faster delivery of software solutions. In such cases, I try to ask three questions to reveal management’s theory behind their decisions to me, and ultimately themselves:
Why Agile/Lean/What-Have-You? What is your motivation?
What’s preventing you from achieving it right now?
What are your expectations from me, in this context?
Each question is designed to probe into the manager’s understanding of their situation and the predictions that have led to my involvement. For example, answers to #1 will tell me whether they can cogently explain what the theory of agile or lean is and why they want to use it in their context to solve an organizational/system problem, or whether they are just looking to “copy” what others have done. Answers to #2 tell me their understanding of how organizational systems work and their role in improving them. #3 tells me whether and to what extent I can successfully help them, ie. am I here to fix their problems for them (instant pudding) or to help them help themselves over the long-term?
If the answer to #1 is just to speed up delivery, my theory or prediction of management is that this is a reaction to a symptom masking much deeper problems, which can be revealed by #2 (limited resources, no time, excessive WIP, no real engagement or participation by upper management), and ultimately #3 as immediate pain relief. This tells me a lot about where I need to focus my efforts!
More Theory Is Needed…
A recent article in the science journal Nature was brought to my attention the other day after I mentioned an observation Deming had made about organizations producing reams of data and figures but little knowledge from them, an apparently shared problem among biologists. This passage jumped right off the screen, emphasis mine:
More theory is needed… Seeking to be led by theory and knowledge will probably require shifts in research culture. Theorizing should be encouraged, and theories should be included in experimental papers to put data in context. Attempts to do this should not be dismissed by editorial and funding processes as idle speculation. As Darwin said, it allows ideas to be attacked and either dismissed or modified. A sort of ‘tyranny of the field’ sometimes inhibits the generation of explanations different from the current consensus, but this is a mistake. If the new ideas are not satisfactory, then they will soon be eliminated and progress will be made.
Nurse, Paul. “Biology must generate ideas as well as data.” Nature 597,305 (September 13, 2021). https://doi.org/10.1038/d41586-021-02480-z
Four striking observations from this excerpt that are well-aligned with Dr. Deming’s thinking on the importance of theory and how it leads to the generation of new knowledge:
First, the stated need for more theory in biology and the desire to be led by theory and knowledge;
Second, the realization that a culture-shift is needed to effect change, in other words a transformation;
Third, a recognition that change is inhibited by a ‘tyranny of the field’ that promotes consensus-thinking, in much the same way Deming saw the ‘tyranny of the prevailing style of management’ constraining and directing how organizations are led and managed.
Fourth, the need to revise or abandon failing theories in light of new evidence.
Reflection Questions
What is your theory of management? Take a moment to consider how you know what you think you know about the causes and effects of actions, inactions, and other phenomena that you rely upon to guide your daily decisions. What operational definitions could you compose to explain them to others? How well do they align with what Dr. Deming outlines in the above excerpts? How often have you revised or abandoned your theory in light of new information? Why?