The Ongoing State of Agile Transformations
Examining the Venerable Industry Survey from a Deming View
A core ritual in my erstwhile career as an “Agile Coach” (a fast-disappearing role in the Canadian IT landscape) was reviewing and analyzing the annual State of Agile industry survey to see how peers were doing in adopting this “new” way of delivering software, what impediments they were facing, and what trends were emerging. Over the weekend I learned the 17th edition had dropped, and as I skimmed through its over-produced pages I thought it would make good material for a quick Monday morning post to view it through a Deming lens.
You can download your own copy of the report here.
Meta-Analysis
Since its inception in 2006 the State of Agile industry survey has suffered from some critical shortcomings that directly affect the quality of results:
As shown in the PBC below, there has been significant variation in the counts of collected responses, owing to entirely voluntary participation, with almost as many responses gathered this year as in the first (788 vs. 722), with an apparent signal of special causes in 2011 when 6,042 surveys were returned;
No identified long-term participants who could provide a commensurate view of how their adoption has evolved over time, or for consistency of results across surveys;
Continually evolving questions, making any long-term analysis difficult if not pointless;
Any analysis in the report itself is limited to “two-datapoint comparison theatre”, with the present year’s result compared, up or down x%, to the last or some other arbitrary time frame. No effort is made to show how responses to questions have varied in-context, over time. A run-chart would be helpful, but not really useful given the the previous two points.
In sum, the survey is a reflection of the system that created it: a bit chaotic and somewhat unaware of itself. Keep this in mind as I step through the responses I did find interesting.
32% use OKRs (Objectives and Key Results) tied to Epics and NPS (Net Promoter Score)
This was an eye-opener. Respondents are reporting the use of OKRs to measure their delivery performance. It’s not clear how this is being done, ie. team, project, individual level and whether it has been tied to the annual performance appraisal.
OKRs, the red-headed stepchild of Management by Objective (MBO) that was popularized by John Doerr in his book Measure What Matters, have become the en vogue management practice among agile practitioners, with many blogs and books and courses dedicated to this next miracle of modern management.
It is forever an irony to me that a community of practice ostensibly based on a systems view for better ways of working falls prey to managing by objectives and by results, however, I have been assured that OKRs are a superior to MBOs because they add “key results” to the “objectives” (still no methods, though) and if they are being abused as a form of measurement, as seems to be the case here, well, that’s on management not the tool.
This will sound familiar to some readers who may recall my Sept 17/21 newsletter on Management by Objective and the story of how Dr. Deming alerted Dr. Drucker about how his invention (MBO) was being (ab)used in the wild. Quite contrary to Drucker’s intent (and I think similar to what’s happening to OKRs in some quarters), MBO had devolved from a strategic business tool to a series of tasks and measurements, predicated on the faulty presumption that a business objective could be atomized into discrete parts, distributed for execution, and later reconstituted for optimization of the whole.
OKR adherents keep telling me I’m getting this wrong, but I’m not so sure.
As Deming noted in The New Economics:
In M.B.O. as practiced, the company’s objective is parceled out to the various components or divisions. The usual assumption in practice is that if every component or division accomplishes its share, the whole company will accomplish its objective…
Unfortunately, efforts of the various components do not add up. There is interdependence. Thus, the purchasing people may accomplish a saving of 10 per cent over last year, and in so doing raise the costs of manufacture and impair quality. They may take advantage of a high volume discount and thus build up inventory, which will hamper flexibility and responsiveness to meet unforeseen changes in the business.
Peter Drucker was clear on this point with deep understanding. It is unfortunate that many people do not bother to read his warning (Management Tasks, Responsibilities, Practices, Harper & Row, 1973).
- Dr. W.E. Deming, The New Economics, 3rd ed. (p. 22)
The efforts of various divisions in a company are not additive. Their efforts are interdependent. One division, to achieve its goals, may, left to itself, kill off another division. Peter Drucker is clear on this point.
- Ibid. (p. 66)
Your mileage may vary…
41% said not enough leadership participation (in the transformation); 38% said inadequate management or leadership support
These findings resonate with my long-term experience working on “agile transformations” where leadership support has been variable and tenuous and participation almost non-existent. None totally understood what agile was nor how it would benefit quality and customers beyond promises of “Better!, Faster!, Cheaper!”, so the efforts always felt like dislocated local sub-optimizations.
Deming must have heard the phrase “you have our full support” from leadership many times over his career, causing him to retort:
Support of top management is not sufficient. It is not enough that top management commit themselves for life to quality and productivity. They must know what it is they are committed to — that is, what they must do. These obligations can not be delegated. Support is not enough: action is required…
A quality program for a community, launched by ceremonies with a speech by the governor, raising of flags, beating of drums, badges, all with heavy applause, is a delusion and a snare.
- Dr. W.E. Deming. Out of the Crisis. (p. 21)
Rx? Leadership needs to begin learning what they must do in order to improve quality, don’t just delegate it to a consultant or Centre of Excellence or Quality Lead. Lead by teaching and modelling new techniques and behaviours. Everyone needs to participate, no one is too important.
One bright spot is that 32% of the respondents indicated that top-management were leading and participating in their transformations…
44% of respondents crave a systems view of their operations
First, some context here: this is in specific reference to a particular philosophy of software development that’s come to be known as "DevOps” or the marrying of “front of house” (operations) with “back of house” (developers) into a cohesive team with a shared culture and practices (typically agile) that are focused on rapid and consistent delivery of value to customers. While you’d think this would be the norm in every organization, it sadly is not mostly because even in allegedly “agile” companies, the prevailing (non-systems) style of management still prevails where parts are managed over their interactions.
Unsurprisingly, those doing the work want greater visibility into their system from inception to customer and back again:
A step toward this would be to do as Deming advised and create a system map to better understand who fits where and what they need to accomplish this improvement.
The first step in any organization is to draw a flow diagram to show how each component depends on others. Then everyone may understand what his job is. If people do not see the process, they can not improve it. Anyone needs to see the process as a catwalk, a flow diagram— Paul Batalden, M.D., 13 November 1990.
The New Economics, 3rd. ed. (p. 22)
Of course, this requires a clear understanding of our aim and constancy of purpose. Which leads to my last observation:
28% attribute lack of clear priorities and direction from leadership as key challenge for agile adoption
The only surprise for me here is that the figure is so low. That there are challenges for adopting a new way of working without clear leadership from top-management is a foregone conclusion. Perhaps the other 72% are happy with the status quo? Maybe they are in organizations where fear keeps everyone from pointing the obvious out?
As we learned in Scholtes’ Sixth New Leadership Competency, organizational and individual purpose and direction are linked: If we lack a clearly communicated and understood constancy of purpose for the organization, we sow confusion throughout:
This is why Deming identified lack of constancy of purpose as the first of Seven Deadly Diseases of management that need to be “cured” before undertaking any effort to improve quality, such as an adoption of “agile” ways and means of delivering software to customers.
Reflection Questions
What do you think? Are you in a software-based service or product organization? What are your impressions of the survey? What do your Deming-senses tell you? Is agile adoption as a method for delivery well-aligned for success in your organization? Why or why not? What impediments have been identified? How engaged is leadership and top-management?
What advice would you give to the survey’s publishers to make it better for next year?
Related Newsletters
Management by Results (Jul. 15/21)
Management by Objective (Sept. 17/21)
Support of Top Management is Not Sufficient (Oct. 4/21)
The Deadly Diseases (Oct. 8/21)
Book Review: Deming's Journey to Profound Knowledge (Sept. 15/23)
New Competency #6: Giving Vision, Meaning, Direction, and Focus to the Organization (Jan 24/24)
Chris, you've written an important critique. Very thoughtful.