This Is One Big Reason Why So Much Business Thinking Is Crap
Sometime in the 1980s, Harvard Professor John Kotter became intensely interested in how change succeeds and fails within organizations. He examined roughly 100 firms, evaluated their performance, and then interviewed executives in an effort to understand what went right, what went wrong and how things could be done better.
That led to Kotter’s 8-step change process, which still forms the basis for most change management efforts today. Six years later he teamed up with Deloitte to interview 200 more executives and, incredibly, learned nothing new but found the identical 8-steps at work. With such massive corroboration, who could question the results?
This is generally how business ideas get established and it is a very shoddy way to go about things. Case study interviews of self-serving executives are prone to enormous amounts of bias. Unless controls are put in place and corroborating research from other fields is examined, the result is likely to be more superstition and lore than fact-based analysis.
The Path Of Least Resistance
Organizations—especially profit-seeking corporations—are notoriously difficult to research. With shareholders, donors and customers to please, not to mention colleagues and superiors to impress, executives need to be very careful about what they say. They have incentives to overemphasize their own role and downplay other factors. This adds to the human tendency that scientists call a fundamental attribution error.
So it shouldn’t be surprising that Kotter’s 8 steps tend to validate leaders’ agency, advising them to, “create a sense of urgency,” “communicate the vision” and “remove obstacles.” There’s nothing about how to identify key stakeholders, understand organizational networks or how to empower others to help scale change.
Another issue with case study research is that it is necessarily limited. When researchers did a case study on a company I used to run, to take just one example, they interviewed insiders (including me) and did their best to interpret what they heard and what they could glean from background information regarding the market.
While I don’t think anything was inaccurate, it wasn’t exactly the truth either. Only a handful of people were interviewed, almost all of them were concentrated in a single part of the business and none of them, besides me, were involved in making decisions. The issues presented in the case study simply weren’t the ones we were actually wrestling with.
Research based on case study interviews typically doesn’t offer extensive documentation. In more rigorous fields like, say, sociology or psychology, researchers are expected to share their data so that others can interpret it. Unfortunately, that’s often not true in business research.
Survivorship And Confirmation Bias Kick In
After the Bay of Pigs fiasco, President John F. Kennedy famously told reporters, “Victory has a thousand fathers, but defeat is an orphan.” This phenomenon often manifests itself as survivorship bias and it greatly affects what cases are studied as well as who researchers interview and draw their conclusions from.
It’s a simple truth that organizations are more forthcoming about their successes than their failures, so that affects what cases researchers get access to. Of course, if the failures are bad enough, enterprises fail and people get fired, making it even harder to obtain information and gain insights into what’s really going on and how things actually work.
Now consider sociologist Doug McAdam’s research on recruiting for Freedom Summer during the civil rights movement. He was able to analyze the applications of not only 720 volunteers, but 239 others that withdrew and 55 that were rejected. He conducted 80 in-depth personal interviews and, because the applications asked for social contacts, McAdam was able to document social ties. The source material was shared in an appendix, so that others could make their own conclusions.
Clearly, with so many so limits, you’d want to be cautious about sharing results, but that’s not how the marketplace of business ideas works. You get much more traction out of a pithy “8-steps” than by a nuanced analysis. Consultants get wind of a marketable product and the sales engine kicks into gear.
Once that the zeitgeist is properly primed, confirmation bias kicks in and people start seeing the 8 steps everywhere. At this point, contrary evidence can be safely disregarded as anomalies. The narrative is set, conformity takes over and few even think to question the validity of the conclusions or even ask where they came from.
The Semmelweis Effect
In 1847, a young doctor named Ignaz Semmelweis had a major breakthrough. Working in a maternity ward, he discovered that a regime of hand washing could dramatically lower the mortality rate in hospitals. Unfortunately, instead of being lauded for his accomplishment, he was castigated and considered a quack.
The prevailing theory at the time was the miasma theory, the idea that bad air was what made made you sick. There was no germ theory of disease at the time, so it made sense that bad smells, usually emanating from rotting organic material or excrement, were the source of infection. Once that principle was established, practice followed theory and the medical establishment identified with, and became protective of, these flawed ideas.
The phenomenon is now known as the Semmelweis effect, the tendency for professionals in a particular field to reject new knowledge that contradicts established beliefs. The Semmelweis effect is, essentially, confirmation bias on a massive scale. It is simply very hard for people to discard ideas that they feel have served them well.
Stack Ranking, six sigma, war for talent and profit from the core, just to name a few, were all management fads that were, at some point, seen as gospel in boardrooms. Widely lauded in the business press and at management conferences, they were eventually largely discredited, but not before an enormous amount of damage was done.
Don’t Believe Everything You Think
Management fads usually come from people who did well in school. Many of these are business school professors and consultants, who’ve never operated a business. They are often people who’ve never failed, been told that they’re smart all their lives and expect others to be impressed by their ideas, not to examine them thoroughly.
They tend to come up with their ideas by talking to other smart, successful people about their experiences. These ideas get picked up by more smart, successful people and are propagated further. The elite hivemind then puts these ideas into practice, rarely checking what evidence the ideas are based on. When the ideas fail, they are rarely questioned. Shortcomings are blamed on poor execution by less smart, successful people.
We need to own up to some basic truths. Case study research can be useful, but is enormously flawed and highly susceptible to bias. People recounting events usually tell self-serving accounts and researchers conducting interviews are often trying to confirm their own hunches. The interviews themselves are almost never subjected to any serious review.
We need to be more vigilant. We can do better. While it is true that researching organizations is notoriously difficult and that interviewing executives is often the best we can do, we can seek to corroborate findings from other fields, explore counternarratives and apply greater scrutiny. We can’t just go around believing everything we think.
As Richard Feynman put it “The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that.”
Greg Satell is Co-Founder of ChangeOS, a transformation & change advisory, an international keynote speaker, and bestselling author of Cascades: How to Create a Movement that Drives Transformational Change. His previous effort, Mapping Innovation, was selected as one of the best business books of 2017. You can learn more about Greg on his website, GregSatell.com, follow him on Twitter @DigitalTonto, his YouTube Channel and connect on LinkedIn.
Like this article? Sign up to receive weekly insights from Greg!
Image by Microsoft Designer
