My adventure into "the complex"
My training and background (project planning) made me see the world as being linear. Here you are schooled to define the critical path and link all activity to that path. The planner task to to produce a perfect plan, ensure the workforce have all the skills needed and, providing they had the resources requirement and executed the plan are written without errors then the result required would be delivered.
I took this mind set into my work on risk management where it served me well in practice. In practice we expected small deviations from the plan and these were managed as they arose. I did not however notice the each deviance as a signal the my basic world view was flawed.
This world view also lasted throughout my doctoral studies. The way I posed my question (this was about identifying causes of failure so that they can be eliminated) was about perfecting the system. I was only during the writing of my first book ('It should never happen again') did the reason why these event keep happening dawned on me. The key reason why failure occurred was the complex nature of the system in play. The complex nature of these system meant that they could never be perfected and so failures would continue to plague our world.
At this point I was able to break free from what I now refer to as the "perfect world paradigm". I now see the world where chaos is normal: this paradigm I have labeled "Normal Chaos".
Here we need to be clear about what I refer to when I invoke the idea of chaos. The word chaos is often associated with disorder; this is not what I mean when I use this word. What I am referring to is:
"the state of a system in which its stable cycles and processes give way to complex, seemingly unordered behavior, governed by so-called strange or chaotic attractors. In this state, the system is responsive even to tiny, sometimes immeasurably small fluctuations.
In a condition of chaos, the system is ultra-sensitive, and prone to sudden change and transformation."
At this point my research was draw towards the work done on understanding complex systems. Much of this work is theoretical and uses quantitative methods to make sense of it. What I was interested in is how the ideas refined within this literature might be used to help the "average Joe" (me) make sense of the complex world in which we have to exist.
As will all academic fields there are many schools of thought when it comes to the study of complexity, I am sure they will all disagree with what I postulate here. I accept that I have grossly simplified some very nuanced arguments and I have conflated several more. However, my aim is to use (and sometime abuse) the core ideas to help me make sense of what I see and hear around me.
By setting out my thoughts, I hope it helps others as well.
My research into organisational failure highlighted the role of complexity in these failures. I see complexity having two main roles;
It makes the interactions of component parts (cause and effect relationships) more difficult to see important data, to appreciate its significance and then to act appropriately (See, Appreciate, Act)
Emergence, where the routine interaction of two apparently standard factors produce a significant variation in the output of the process due to some minor variation in one of these factors.
My research comes in two parts: my Normal Chaos Framework and cases studies.
Normal Chaos Framework
To help me make sense of the complex world I have produced what I have called my Normal Chaos Framework. I will, in due cause, describe the history behind the current framework. In it current state, it has taken 10 years to develop and this has involved several iterations.
The framework provides multiple view point from which to examine and issue or situation. This serves a twin purpose. the first is to provide a richer picture and therefore a warning to the analyst team might not have a full picture as it will be clear tho them when they have stopped going deeper. The second is that the multiple viewpoint with generate a wider understanding taking in a more diverse range of view. It is hope that this will have some effect in preventing any group-think developing.
The current framework is routed in a review of academic literature that has used one or more ideas taken for complexity theory and has used them in an intuitive (qualitative) manner. I collated these ideas and the framework emerged. It has been refined through use and so it not quite stable.
For a description of how the prefect world paradigm aligns with the idea of Normal Chaos see here.
A brief description of my Normal Chaos framework can be found here.
My work now involved exploring and refining each idea in an attempt to warrant (justify) each idea and to make them useful.
I have used this framework to review Part 1 of the Moore-Bick Report in the fire at Grenfell Tower. My analysis can be found here.
In this section I will also use the framework to examine past events mainly focused on D-Day at Normandy (6 Jun 44). While this might appear to be quite random, it was a conversation about D-Day that started me down this track. My father-in-law was a D-Day veteran; as a infantry soldier he land on Gold beach. He said that "it was chaos but it worked". My research started in order to make sense of his statement.
Examining Complex Systems
We live as part of a complex system. This means that we struggle to make sense of what is really happening around us. To cope, we greatly simplify what we see, we build false narratives that explain what is going on and we create in our own minds a world as we would like it to be rather than how it is. In many ways we are still quite primitive creatures driven by an instinct to survive. Having said that, we have tried to develop the way we think and to learn about the world in which we exist.
Ever since the enlightenment Western societies have been schooled to believe in logic and rationality. This schooling makes us value the reduction of the systems that affect us into cause-and-effect mechanisms as a way to understand the world around us. By doing this we believe that we can understand why things happen in the way that they do. By increasing our understanding, we hope we can see ways to control what happens to us. By having this control, we seek to make our world a safer place to live. However, there is one major flaw in this logic. While some people have a capacity to understand far more than others, no member of humanity has the mental capacity to understand the system as a whole. Everybody needs, to some extent, to simplify the whole to make sense of it.
To understand the world, we build mental models. These models are given many different names. They may be called theories (both academic and lay), hypotheses, propositions, frameworks or other system models. However, it has to be recognised that all models are full of assumptions and simplifications. George Box has therefore provided a valued heuristic when we consider the models we use. He said, "all models and wrong but some are useful." To me models are useful when they make us question what we see. To me they are useful when they make us probe the assumptions we are making, examine the ways we have simplified what we are looking at and where they make us conscious of making these choices.
The very nature of learning means that we are building a set of assumptions about how the world works. Some of these assumptions will be found to hold true every time. Some of these assumptions will be found only to hold true in certain circumstances and others will prove to be false. Life is a continual process of learning where we refine our assumptions and learn when they are likely to hold true and when they probably will not. It would be good if this process was a linear journey of accumulating understanding but, unfortunately, it is not. While we do learn, we also forget and so learning has to be seen as another complex (non-linear) process. We forget because we have a somewhat limited mental capacity to make sense of things.
This limited mental capacity also means that we instinctively simplify what we see in order to make sense of it. We simplify the world in many ways such as placing boundaries around problems, limiting what is being considered. We use heuristics as a shortcut to decision making and reduce non-linear systems to a linear process. As humans the approach we take to tackling complex problems is to take them one step at a time. This is referred to as 'muddling through'. The problem with this approach is that it often produces non-optimal outcomes compared to if the problem had been tackled holistically from the start. The problems produced by muddling through are compounded by what Kurt Gigerenzer calls one-reason decision-making: this is when we make decisions based on finding the first option that meets a certain criterion. In short, in both muddling through and one-reason decision-making, we trade a thorough understanding of an issue with knowing enough (sufficing) to take the next step.
The dilemma is acknowledged more often than you might think. Every time someone utters the phrase "but in a perfect world" they are acknowledging, at least sub-consciously, the dilemma. The phrase acknowledges that there is, in theory, a perfect outcome but it laments that such a perfect outcome may not be possible to achieve. During my earlier work I have defined what such a perfect world would look like: see here. The problem is however that the perfect world paradigm is a linear system while the real world is non-linear. I have accepted that in order to act we will often have to simplify the problem we face in line with the ETTO principle. However, from a risk governance perspective, every false assumption and simplification provides a gap between hopes and reality through which an unwanted event may emerge. I therefore see each thoroughness/ efficiency trade-off requiring a cost benefit analysis. I do however recognise that each trade-off will also be the subject of its own trade-off and so many of these gaps will be overlooked. In the end, the outcome I would hope for is that decision-makers become more aware of the risks they are taking due to unexamined false assumptions and simplifications where they had previously never been conscious of them.
The purpose of my research is to develop a framework that provokes debate that exposes hidden assumptions and simplifications amongst those tasked with managing a complex system. An additional benefit of such debate is that it should help to develop cross-understanding and therefore alignment of thinking within the team.
As I have said previously, while there are many researchers who look to manage complexity through quantitative methods, my approach is more qualitative and focused on sense-making. My personal preference when it comes to sense-making is to try to visualise the relationships between the parts that make up the system. As an example of how a complex system might be visualised, I offer below one produced by PA consulting. Their visualisation tries to depict the non-linear dynamics that affect the social stability of Afghanistan in 2009. It should be noted that, at that time, NATO had over 55,000 troops in country to support their political ends. Over time this number was reduced to 10,000 and yet the country's dynamic of macro stability was maintained. However, it only took the withdrawal of the 2,500 US troops in September 2021 for the country's political system to collapse. While the dynamics of this system collapse would no doubt be fascinating to study, that is not my purpose. I will however be referring back to this diagram to help readers visualise the points I will be making about  sense-making and  learning when it comes to complex systems.
In order to understand the true complexity of the system that this diagram depicts, readers will need to imagine that there would be a similar detailed diagram to describe each component (node) of this network of interactions. This would be similar to the cube I depict here. In this diagram each set of words should be considered to be a node.
Having said that I do not plan to examine the dynamic stability of Afghanistan, this illustration does provide a good starting point for the discussion of how we might make sense of the situation depicted using my normal chaos framework. My starting point would be the social collapse that took place in September 2021 following the withdrawal of the US component of the NATO force present at the time. Here I provide a very rudimentary example to set the scene for the work that follows.
The transition in societal dynamics that took place in September 2021 would make my starting point the lens labelled "Edge of Chaos". The immediate question this raises is one that would look at what in the dynamics changed to tip the society over the edge from what may be seen to have been normal chaos to one of abnormal chaos (crisis).
I would see that you could look to answer this question through at least two routes. You could either start locally by looking at the issue through the lens of attractors and energy flow or you could look more at the fundamentals of the system by starting by looking through the lens of structures and patterns that affect the dynamics. For the sake of this illustration, I will adopt the first approach.
If I start with the energy flow lens, I would ask what energy it took to maintain the society's dynamic stability (see 'Red Queen Effect'). I would also ask whether what had already taken place meant that it was already inevitable that the society sank into chaos (here we examine the subject through the energy gradient lens). In practical terms, this would mean that we would need to examine whether the societal collapse was a consequence of the US decision to withdraw their last troops or whether decisions made by the previous administration meant that the collapse was inevitable at some point in the future and the decision to finally withdraw only hastened the collapse.
The collapse also begs the question of whether it was only an illusion that the NATO presence (again a Red Queen issue) was holding the country in a pattern of dynamic stability. It also begs the question whether the structures supposedly put in place, such as a 300,000 strong army, were also an illusion. In terms of the other political structures put in place, we would need to understand how they operated. This would mean looking at the attractors in play (be they political, social, cultural, tribal or religious) that induce cliques to self-organise and then to work against the formal structure set up to govern the society.
The PA illustration depicts the system at a particular scale: in this case it seems to be set at the level of institutional interactions. Understanding this leads to an important understanding of the simplifications that are likely to have been made with the model. There is however a second important consideration exposed when the scale lens is used. Let us, for example, examine the role of the President; here we may have a completely different picture when we consider the President as an embodiment of the State, an embodiment of the Government or as an individual. It may be that the President features at different places in the model when considered at these different scales. The illustration also depicts the interdependencies between the various nodes; in this case we may be asking how the President interacts with others. The PA depiction is greatly simplified and suggests that there is a single relationship, mode and channel of communication. While it may take a full network analysis to understand the total relationship between any two nodes, the questioning of what these may be might be enough to point the discussion in the right direction. We may also have to consider how the President plans to achieve the goals that he has set for himself in the ever changing social and political environment in which he is forced to operate; this would be considered using the fitness landscape lens. Finally, we may ask whether the patterns of behaviours that we see around the President repeat at different scales and in different parts of the system (where they do, they can be seen as being fractals.) The assumption that such patterns do or do not appear is often the cause of an unwanted event emerging.
In summary, this is just a brief outline of the types of discussions that might be generated by applying the lens of the normal chaos framework to a complex problem. In reality many more aspects of the systems would need to be scrutinised and, as you will see as the framework is developed, many more lenses are available through which each factor can be examined. The intention is that these multiple lenses will provide a greater understanding ('greater sense-making') of any complex problem.