Aligning the Prefect World Paradigm with Normal Chaos

Ever since I first used the term "Normal Chaos" in my 2013 book "It Should Never Happen Again", I have been trying to refine what I mean by it. In my 2015 book "In Pursuit of Foresight" I expanded my thinking further, juxtaposing the idea of Normal Chaos with that of the Perfect World paradigm. This practice of setting one idea against an alternative is common within the academic world where their arguments are based on the fact that two competing ideas cannot both be correct. In my early research I explored the debate between the merits of High Reliability Organisations and, what is referred to as, Normal Accident theory. As a former practitioner, I could see merits and flaws in both constructs. However, within the academic literature each researcher seemed to believe that one was right and the other was wrong. I was surprised how bitter this discussion became! In the end I wrote to Charles Perrow asking him to explain his intent in writing about Normal Accidents. He replied that he was not theorising but warning that complex closely coupled systems will inevitably go wrong and, if the consequence of that failure was significant, we should think again before we commission such systems. My exchange with Prof Perrow made me start to think about complexity as a factor in organisational failures.

Despite this previous scholarship, when it came to my exploration of Normal Chaos and what I then saw as its antithesis, the Perfect World paradigm, I still approached my research on the flawed basis that if one was true then the other must be false. I have now changed my mind. I now see one (the Perfect World paradigm) as being a subset of the other. Let me explain.

The basic proposition of the Perfect World paradigm is that if we recruit the perfect people, produce perfect plans, train them perfectly, supply them with exactly the right resources (including perfect unambiguous information) and execute the resulting plan flawlessly (eliminating all slips and lapses) then the desired outcome will be delivered. Within this paradigm is the belief that individuals should be able to learn, retain and use the knowledge they require perfectly. All of this perfection is then supported by having perfect foresight leading to individuals being blamed and punished where they fail to achieve these standards. Embedded within this construct is the desire to remove uncertainty and to control the world around us. The label Perfect World paradigm is used to reflect the phrase often heard when discussing failure; that is “but in a perfect world …”. At this point we need to ask the question as to whether this perfect paradigm could ever hold true. My answer would be both "yes" and "no"! I see the dividing line between yes and no coming down to the granularity (in Normal Chaos terminology "scale") of the criteria used to judge perfection and the paradigm’s practical utility.

The basic premise behind the Perfect World paradigm is that you can produce closed systems and operate within them. That is a system that can be isolated from external disruption. This assumption can be seen in operation across many practices ranging from the micro to the macro.   Rule-based systems are an example of the paradigm working at the micro level. At the mezzo level we can see examples such as the functioning of a factory, and at the macro level, we might cite international just-in-time logistic processes. All these systems are basically linear in nature, their inputs are thought of as being predictable and these processes are set within clear boundaries.  Whether the answer to my question is yes or no will therefore depend on the scale of variance that can still be considered to be operating perfectly, against an absolute standard set for the system. In practice this would be judged on whether the system or processes broke down or not. In practical terms the idea of perfection is quite elastic!

To give the answer "yes", would be to accept the minor fluctuations, slips and errors that are inevitable within the system. Experienced managers will recognise how much of their working day is absorbed by deciding on what fluctuations within their system are acceptable or the action required to ensure the process is brought back to within acceptable boundaries. Within the terminology that I used, this would be seen as being a self-regulating system. The criteria for what is acceptable generally comes down to whether the product or service continues to be delivered to the client in a manner that is acceptable to all stakeholders that the key parties care about [who these stakeholders might be will be the subject of a future blog]. The utilisation of the Perfect World paradigm is therefore seen to be useful to practitioners when they feel that they can control all the key parameters within their system. The linear nature of this paradigm also makes it easier to apply. The test for such systems is that they are rule-based and, in general, the rules remain valid and can be applied.

Use of the Perfect World paradigm starts to fail when, in practice, managers find that they do not have as much control over their system as they thought or hoped they had. They see their rule-based systems (including its contingency options) are not able to cope with the variance encountered. This is the point my "yes" answer would turn to "no". In practice (and in order to ensure the continuing delivery of a product or service) the use of this paradigm fails when an input to the system causes a major problem that requires a reconfiguration of the existing system, or the necessity for the rules (as written) to be violated. Within the terminology that I used, this would be seen as being a self-organising system. In this context the Perfect World paradigm continues to have practical utility up to the point that self-organisation and rule violations become the daily norm. In practice however, as the Perfect World paradigm is the predominant analytical lens taught at business schools and other centres of learning, managers often have to continue to use this paradigm way past its usefulness as they have no other means of assessing their management problems.

In practice, the point that self-organisation and rule violation becomes the daily norm is the point at which operations are likely to be recognised as being complex, dynamic and often, high tempo. In this context things happen quickly, and the patterns that we base our decision-making process around are more difficult to discern. In these circumstances, inexperienced practitioners are likely to see chaos (disorder). The more experienced practitioner would however be able to see the more complex patterns at work, including seeing what is not present that should be. They see order in the disorder; now we are approaching the ideas of Complexity, and the related Chaos, Theories. (I will elaborate my use of these theories separately). These theories consider open systems that function in non-linear ways. The open nature of these systems means that they can be expected to be subjected to many uncontrolled factors in what might appear to be a random manner.

The non-linear nature of these systems is likely to mean that many of the cause-and-effect relationships that exist within the system are difficult to determine. In practical terms these characteristics will be apparent from the number of external influences that disrupt the working routine. As these become more frequent, the rules that govern such systems will become increasingly complex and are frequently unworkable without violation. [It is worth noting that the production of rules or guidance in these circumstances is a very interesting stream within the academic literature on safety. From this work there is clear evidence that the application of rule-based systems as envisaged by many public inquiries is not appropriate for complex, dynamic and high tempo situations.]

Going back to the original question as to whether the Perfect World paradigm could ever hold true, the answer (from the point of view of scientific utility), would also be “no”. There is an old Yiddish adage that says, “Man Plans, and God Laughs”. This is taken to mean that despite our efforts, our plans are often wrecked by the unpredictable nature of life. In the more colourful language of the military, “shit happens”!  The probability of internal missteps and unwanted external influences mean that, unless the scope is very restricted or the planner is incredibly lucky, it is unlikely that any complex plan will ever be successfully enacted without adjustment. The irony is that the imperfection of the Perfect World paradigm cannot just be ignored.  This means that, of the two ways of seeing the world, only Normal Chaos has scientific validity. However, as it does have practical utility, to reject the Perfect World from this debate would also, in my view, be wrong. 

So, what is Normal Chaos? It is the recognition that the world in which we operate is an open system where all our plans are liable to disruption by both internal and external factors. It recognises that, while we may try to isolate our systems and processes from external factors, they are always liable to the effect and disruption of such influences. The Nobel winning physicist Ilya Prigogine called these spaces "islands of order in a sea of disorder". Normal Chaos recognises that as our systems and processes become more complex, they are likely to produce unexpected results; they produce patterns of activity that the inexperienced are likely to see as being chaotic (disordered) while the more experienced practitioners are more likely to see the patterns in play. One of the features of such systems is that they produce a regular outcome multiple times through the mechanism of dynamic stability: these repeated results can be mistaken for the system being stable. This means that when they unexpectedly produce a different result, this takes the people affected by surprise: these are often seen as Black Swan events while in fact that are just emergent events to which people have been blinded by the induction fallacy. And so, we start a new way of thinking.

The implication for practitioners of accepting the Normal Chaos paradigm is that they are always aware that nothing in their process or system actually stays the same and therefore they recognise that it is always in a state of flux that can produce apparently anomalous results. They recognise that while such a system may appear to be stable in the short-term, it is a dynamic stability that can change at any moment and therefore, in the long-term, it is not. Here one of the key judgements that practitioners have to make is an assessment of how long the current apparent status quo is likely to last, for this will give their deliberation structure. They need to assess the forces acting on those structures and to be prepared to adapt at the appropriate time. In order to plan their adaption strategy they will need to be alert to the way the relationships (seen as patterns) between the structures may change and how the structures might evolve in the longer-term. They have recognised that even small fluctuations can have dramatic effects on the results their system produces. As a consequence of this, those who embrace the concept of Normal Chaos are aware that their system might falter at any moment and therefore focus more on effectiveness, robustness, resilience and agility rather than producing the most efficient system possible.