Managing Complexity
Back in the 15th Century, Leonardo da Vinci, the great genius of the Middle Ages, said that “simplicity is the ultimate sophistication.” Most modern managers would agree. Every good operation works hard to streamline its processes down to the barest essentials.
However, the world is a complex place and it’s only getting more so, which is why many management thinkers have been urging businesses to embrace complexity, to become, in effect, system thinkers rather than reductionists.
As Richard Straub noted in a recent article in HBR, that effort has largely failed and we shouldn’t be surprised. Executives are paid to be accountable and are understandably reluctant to give themselves up to the complexity Gods. In truth, complexity is not something we need to embrace, merely something we need to accept and manage.
What Is Complexity?
Before we can manage complexity, we first need to understand it and much of the literature on the subject obscures more than it reveals. In actuality, the term is used to refer to three very different things and if we are to understand complexity, we need to account for all three:
The Complexity of an Entity: Technically known as Kolmogorov-Chaitin complexity, the complexity of an entity is defined through the amount of information it takes to describe it. So a huge entity like the number googol (i.e. 1100) can, in fact, be much simpler than a relatively small prime number (like 842,293)
Good managers have an intuitive feel for this type of complexity and work hard to reduce it. Operational excellence is strongly correlated to the ability to simplify complex entities.
Nonlinearity: Most of the things we deal with in operations are linear. If we sell 10 widgets a month, we expect to sell 120 in a year. However, many important aspects of business are non-linear and that’s hard for many managers to accept, especially with respect to accelerating returns.
Nonlinearity can be extremely dangerous for an incumbent company faced with an disruptive competitor. What looks relatively small and inconsequential can get big in a hurry and upend even the most dominant of firms. Innovation researcher Tim Kastelle recently wrote a great post about this phenomena in connection with electric cars.
Emergent Complexity: This is probably the most confusing form of complexity, because unlike nonlinearity, it’s not simple matter of accounting for rates of change, but of understanding how interactions between entities (even simple ones, such as transistors or neurons) can give rise to something completely unexpected (like intelligence).
I think it was this type of complexity that Mr. Straub was referring to in his article and he points to some good reasons why managers haven’t embraced it. Namely, that managers like to be in control, the tools we’ve had haven’t handled complexity well and that the new technologies of non-human decision making are unnerving.
And he’s right, managers have not embraced complexity and I strongly suspect many never will. So rather than expecting executives to embrace complexity, what we should be doing is coming up with strategies for managing it.
Managers vs. Executives
Most managers didn’t start out by managing. They worked their way up and got noticed because they were able to execute. This often entailed mastering complex information and working within a specific context of people, processes and competitive environment. Not surprisingly, they feel confident in their ability to bring order to chaos.
However, managing is different than executing. To manage effectively, you need to account for that which is beyond your understanding. You have to lead people who have expertise that you don’t, operate in market situations that you did not foresee and evaluate opportunities which are uncertain.
So the first step toward managing complexity is to think like a manager rather than an executive (unfortunately, many never actually make that leap). Simply getting a promotion does not make you a manager. Until you are ready to take responsibility for that which you cannot control, you are just someone with a title, not a leader.
Micromotives and Macrobehavior
A particularly unnerving aspect of emergent complexity is that interactions between factors under our control can often lead to phenomena that is difficult, if not impossible, to foresee.
Nobel prizewinning economist Thomas Schelling was probably the first to identify this problem with his segregation model, in which even those who prefer to live in mixed neighborhoods (but not to be outnumbered) can give rise to extreme segregation.
Another example is the Mandelbrot set, which uses a simple formula to create almost unimaginable complexity as does Stephen Wolfram’s Class IV automata. There is no evidence that working with simple, understandable elements gives us any more control, only perhaps the illusion of control.
So another thing that managers will have to accept is that their job is not about nodes, but about networks. Just because they are adept at managing entities doesn’t mean they understand the interactions between them.
Becoming Less Wrong Over Time
Managers are trained to deal in “hard facts,” to be data driven. They are supposed to research the market thoroughly, analyze with statistical rigor and make informed, rational decisions. Good managers know their numbers.
Unfortunately, our numbers are always wrong. They tend to be backward looking, based on small samples and are often gathered in a shoddy manner. In fact, a recent study in the journal Nature found that a majority of cancer research could not be replicated. If that’s true of matters of life and death, how accurate do you think marketing studies are?
I’ve suggested that we take a more Bayesian approach to strategy, where instead of assuming that we have the answers, we strive to become less wrong over time. Good strategy is never being, it is always becoming.
Seek Simplicity, But Distrust It
So while I agree with much in Mr. Straub’s article, I don’t believe managers will ever embrace complexity to a great extent. Complexity, is messy and uncomfortable. It’s fine and well to enjoy thinking about complexity as an intellectual exercise, but when you are accountable for results, it’s a nuisance.
However, while embracing complexity may be quixotic, ignoring it is not an option. We can factor down entities to make them more manageable, account for a wide variety of variables in our analysis and even deploy the latest big data technology to ensure the most robust examination of known facts possible, but we’ll still come up short.
So we should, as Alfred North Whitehead suggested, seek simplicity, but distrust it. We need to take a network view, manage what we don’t understand and become less wrong over time. Simplicity, after all, is not so simple.
In order to manage effectively in the modern world, we must not only hold ourselves responsible for those things that are within our control, but also account for that which is not.
– Greg
Focusing on things within your control does work magic….
According to Steve covey said “.. focusing on your circle of control, helps grow your circle of influence”. You focus on things you can control and the rest run on automatic(with little interference).
Yes. It does same time:-)
Complexity may be managed providing you are able to measure it. Since 2005 this is possible and numerous companies are now using the technology to shed excess complexity which is a formidable source of vulnerability. Some examples are here (note the Complexity Profiles):
http://www.ontonix.com/Complexity/Examples
Best regards
Dzieki Jacek. Bardzo cziekawe!
– Grzes
Greg, I would never have imagine that you speak Polish!
Serdeczne pozdrowienia!
Jacek
Another great piece.
Addendum: Amongst the significant issues which impacts those who are unfamiliar with either or both of systems theory and complexity are the interactions between complexity, fragility and risk: and how these play out in – respectively – closed and open systems.
For every extra added complexity we add to any system we also add fragility and risk and this increase occurs in a non-linear manner. When we move complexities around, when we shift existing stable patterns of systems complexity, we also change the risk profiles and potentially also greatly increase fragility: the impacts of changes in the topology of complexity are not directly predictable from a knowledge of the existing topology, especially in any open system.
To address this in the context of product innovation: when Steve Jobs decided to “reinvent” the MP3 player and the mobile phone one of the things he had realised was that Apple needed to move all of the accepted and “standard” User Interface complexity into the background: to simplify things for the users whilst retaining the management and control power for those who needed it. The challenge was to do so in a manner which [at worst] retained or [at best] improved upon the existing levels of fragility whilst attempting to reduce the risk factors. To make the products as simple foolproof and robust as possible.
The success of the iPOD and iPHONE and their descendents are a direct reflection of the vast effort which Apple put into addressing this hugely difficult issue. Had they screwed up any single part of that complexity management issue then the entire iPOD/iPHONE ecologies would have ended up being fragile and potentially also a high risk to consumers [and to Apple itself, who more or less bet the farm on their success].
That effort could not have been started – let alone completed – without an appreciation of complexity and systems theory informing almost every aspect of Apple’s work on these products.
When we look to more futuristic IT projects complexity and systems theory now interlocks directly into the way we will design and operate our global networks: the quantum entangled switches that will soon be deployed across the world to speed up and manage our Tier 1 fibre backbones use both complexity and systems theory not only to deliver far higher capacity but also to secure and protect traffic between endpoints: we will be randomly shifting the complexity on a quantum basis between switches and routers.
To return to my starting point: complexity, fragility and risk. Businesses often fail to appreciate the impacts of this interaction. Making any system or process more complex increases the risk attached to that process, and it *also* increases the scale and scope of the impacts of any system failures. This is the so-called incident pit – the route from a minor unplanned event to a complete disaster. Which is why it’s so essential for managers and others within any organisation to be able to step back and look not just at the nodes of their network but at the wider “systems” view of that network. The big picture. They need to learn how to look for and understand their risk factors and potential points of failure across their entire network – where the network is fragile – and the chains of failures which can develop in unquantified complex systems. Avalanches start with the tiniest things.
Mieszkalem 6 lat w Warszawa!
– Greg
Very good point Robert! I discussed some of these issues in an earlier post about complexity, where I recommended that managers factor down, minimize options and stay robust.
Interestingly, the same strategies are also effective in negotiations.
Nice to see you again!
– Greg
Ah yes: Mr Shannon. One of the world’s most underrated geniuses, and a seminal figure in today’s high-tech world.
Completely agree about simplification. To switch metaphorical tracks, one of the great automotive engineers of the last century – Colin Chapman of Lotus – had a wonderful mantra: “Add Lightness!”
He was obsessive about systems thinking; viewing the ‘whole’ vehicle and its interactions with the road surfaces as a single dynamic entity.
NB: that same paper by Shannon does a lot more than just detail complexity; it also defines the noise>wealth/wisdom chain and how top-down and bottom-up knowledge transfers function, and it defines the foundation of modern telecommunications. A real whizz-bang piece of elegantly simple mathematical scientific writing that still stands scrutiny today alongside anything by Alan Turing or others of that generation.
Very true. Although I wouldn’t want to denigrate Turing in any way, I do think that he has unjustifiably overshadowed Shannon.
– Greg
Great article. A couple of comments/thoughts. My sense is that non-linearities are a key driver of emergent complexity. For example, in many networks the number of linkages grows non-linearly with the number of nodes. Also many complex networks have hidden feedback loops that create non-linear self reinforcing behavior. I suspect that developing an awareness of and sensitivity to non-linear behavior is key to navigating many complex situations.
Second, I think that an adaptive approach is necessary to navigate emergent situations. We need an approach in which learning and action co-evolve (another way of saying the we need to be progressively less wrong). I think this involves an iterative cycle of experiential learning, design thinking and experimental action.
btw, I also think that Shannon (Kolmogorov-Chaitin) complexity has a place in thinking about emergent complexity. Emergent situations have a high level of Shannon complexity because they are irreversible, ie they are highly dependent on their context. Therefore, in order to describe (repeat) them you would need to describe the entire context.
Thanks for you thoughtful and stimulating article.
Great points! Thanks Markus.
– Greg