20 January 2016: This page has now been superseded by “Complexity – 101”…
Complexity Theory is the Study of Evolution.
Chaos & Complexity are related; both are forms of “Coarse Damping”. While chaos is a form of coarse damping in “time”, Complexity on the other hand is a form of coarse damping in “Structure”.
Complexity arises from the ubiquitous “Collaborative Interplay” of “Entropy & Symmetry-Breaking” in all naturally damped-driven systems – Complexity is a form of coarse damping to uniformity. A form of coarse symmetry. “Complexity is Coarse Entropy”.
Progressive Complexity arises from the ubiquitous “Competitive Interplay” of “Entropy & Coarse Entropy” in all naturally damped-driven systems – Evolution is a form of coarse damping to complexity. “Evolution is the Progressive Upheaval and Rejuvenation of Coarse Entropy”.
In this document we will address the 3 basic stages of the evolution of complexity
1. Locally: The self-reinforcing symmetry-breaking that brings about Emergence from Thermal Equilibrium!
2. Regionally: The collaborative interplay of symmetry-breaking-emergence and entropic-co-emergent-integration that brings about the Emergence of Complex Structures of Integrated Diversity!
3. Globally: The competitive interplay of complex-emergence and entropic-decay and that brings about the Progressive Evolution to Ever Greater Complexity!
Part 1 – Emergence & The Reverse Law
There are two fundamental forces at work in nature and all evolutionary systems; one is the entropic force of spontaneous decay and disorder (otherwise known as “The Second Law of Thermodynamics”); the other is a universal, and somewhat mysterious, capacity for self-organization and spontaneous emergence.
Emergence is the spontaneous collective self-organization that can occur in systems of freely interacting parts. Self-organization is a fascinating subject not least because it seems to be in direct defiance to the Second Law of Thermodynamics (SLOT).
The SLOT is the law of physics that deals with the spontaneous distribution of energy. In everyday terms the SLOT is simply the fact that cold milk and hot coffee, if left unstirred, will spontaneously mix themselves. The SLOT states that left undisturbed all systems gravitate towards “Thermal Equilibrium”.
Despite the fact that the SLOT describes what seems to be rather innocuous common everyday behavior, it nevertheless is considered to be one of the most important laws of physics; and the reason for this exalted status is that the SLOT is also the “Law of Maximum Entropy”.
A Quick History of Entropy
In 1850 the German Physicist Rudolf Clausius formulated the Laws of Thermodynamics and in doing so he equated the SLOT with the idea that all spontaneous change is a process of maximizing a thermodynamic quantity which he termed “entropy”, and this maximization of entropy is ultimately achieved at thermal equilibrium.
During the 1870s Austrian Physicist Ludwig Boltzmann associated the overall state of a system with the internal activity of its atoms. He put forward a theory that the spontaneous movement to maximum entropy and thermal equilibrium is in fact nothing more than a system – of freely interacting parts – spontaneously moving to its “most mixed together condition”; which mathematically is its “Most Probable Distribution” (MPD).
Boltzmann theory was suggestive of two things. Firstly that the SLOT is not actually a real universal law – in the sense that a real law is something that always happens – but is in fact a probabilistic law; in the sense that the movement towards the MPD happens with such an absurdly high degree of probability that it might as well be a real law. And secondly that maximum entropy – given that it is the most mixed together condition – equates with the state of “maximum disorder”…
Although Boltzmann’s probabilistic explanation of the SLOT initially faced hostility – not least because many prominent physicists of the time didn’t believe in atoms – it eventually became the accepted reasoning for why all systems are spontaneously pulled towards thermal equilibrium.
[Click here for a full history of the SLOT, and the concept of entropy]
Entropy + LLN
With the acceptance of Boltzmann’s theory came the general understanding that all spontaneous change ultimately leads to maximum entropy; maximum disorder and decay.
What seems to have been consistently overlooked however (or at least never, mentioned in textbooks) is the fact that the spontaneous movement to maximum entropy relies completely on the absurdly large numbers of particles, in a thermal system, to ensures that the system can fine-tune to equilibrium. In other words; maximum entropy relies heavily on the “Law of Large Numbers” (LLN).
Complex Adaptive System
Most people are familiar with the concept that if you toss a coin four times, you won’t necessarily get a 50/50 split of heads and tails: indeed, you could get 4 tails, suggesting (wrongly) that the coin will always land on tails. But if you toss a coin a million times, you will get something close to a 50/50 split between heads and tails. It is the LLN that ensures that the one million coins tosses will produce an average of 50% heads and 50% tails.
Thermal equilibrium is a consequence of this law of probability; and so anything that weakens the LLN will also weaken the system’s ability to fine-tune to equilibrium.
In a thermal system billions of tiny particles interact with each other through collisions, but other than that they behave completely independently of each other. Maximum entropy is achieved in a thermal system when the particles within the system have maximum independence.
Systems where the parts – be they particles, elements, components, entities, agents, organizations, etc – behave independently of each other are actually quite rare. Many systems are populated by adaptive agents, and such systems have a tendency to reinforce collective behavior. These systems are best described as a “Complex Adaptive Systems” (CAS). CAS’s and collective behavior have serious implications for the SLOT.
Random Emergence of Positive Feedback
The LLN ensures the spontaneous movement to thermal equilibrium in large thermal systems. However for very small regions within these large thermal systems, there are not enough particles to ensure a “local equilibrium”. At the very lowest level within these large systems, random fluctuations are occurring all the time which means that local imbalances are constantly and randomly flittering in and out of existence.
Occasionally these random temporary fluctuations can randomly be very persistent. In a thermal system this is naught but a mere statistical curiosity, but in a CAS it can easily happen that some parts within the system will begin to adapt to these persistent fluctuations; and often such adaptation only serves to amplify the imbalance even further, and in so doing, further extend the fluctuation’s duration.
Thus random local fluctuations can lead to the localized emergence of positive feedback and the associated collective correlated behavior.
Positive Feedback and the Build-up of Viscosity
CAS’s can be thought of as “Fluid-Like Systems” (with varying degrees of viscosity…). In a CAS, collective behavior increases the fluid-like viscosity within this fluid-like system. In a CAS, collective behavior reduces individual independence; and any reduction in independence weakens the LLN! This weakening of the LLN causes the system’s behavior to become more “coarse” and jumpy which, of course, weakens the system’s ability to fine-tune to a stable equilibrium.
Consequently excessive collective behavior among the parts within any fluid-like system, will act to hold (or pull) the system as a whole away from equilibrium; by effectively engineering a “Reverse of the Law of Large Numbers” (RLLN).
It is in the LLN and the RLLN that we see the two potential extremes of agent and system behavior. Just as a million coins tosses pretty much guarantees an average of 50% heads and 50% tails, and a single coin toss absolutely guarantees either 1 head or 1 tail (but not 1/2 a head + 1/2 a tail); so too the LLN pretty much guarantees thermodynamic equilibrium when the agents are totally independent of each other, and the RLLN absolutely guarantees far-from-equilibrium collective-behavior when the agents are behaving entirely as one.
We can think of this difference in system behavior as being like the difference between two voting systems. The LLN voting system produces a balanced proportional representation result; while the RLLN, on the other hand, produces a collective choice, “a winner takes all” result. Thus while the LLN finds the collective average, the RLLN on the other hand ultimately pulls the system as a whole collectively towards making a choice between all of the available options.
All fluid-like systems, will experience a constant tug-of-war between these two extremes. This constant push and pull can mean that as agent behavior moves from extreme individualism to extreme collectivism, system behavior will transition from self-stabilizing to self-reinforcing. And ultimately should the RLLN manage to secure the upper hand, it can mean that, an initial random emergence of the collective behavior could in due course actually reinforce a small local imbalance into a collective system-wide “symmetry-break” – and the reason that this is interesting is that a system symmetry-breaking is a system making a decision.
So effectively it is the “RLLN” that is the mysterious source of spontaneous emergence. “Emergence” is actually the mathematical result of the RLLN driving systems away from the bland uniformity of maximum entropy at thermal equilibrium…
“Emergence” occurs in fluid-like systems when “positive reinforcement and the RLLN drive the system to make Symmetry-Breaking Decisions”.
[NOTE: Realistically we could say that, even when an individual element stops flip-flopping between alternatives and instead makes a decision (whether they are individual decisions, or decisions influenced by others), that this action alone is an emergent symmetry break. So we could say that when any entity makes a decision away from the norm, it effectively becomes an emergent entity…]
Part 2 – The emergence of complexity
LLN and The Emergence of Diversity
At its extreme the RLLN can be an outright choice between alternatives, but Mother-Nature more usually restricts the RLLN to the emergence of local and regional imbalance – rather than a full system-wide symmetry-break.
When individual agents adapt to, or influence their neighbors localized symmetry-breaking clusters begin to emerge. But within the wider system the LLN ensures (the emergence of) diversity, because different areas/localities/regions will randomly veer off in many different directions from the “Most Probable Distribution”.
To put this another way! Any system devoid of any adaptive behavior is akin to a thermal-like system. In such thermal-like systems there is no diversity because no element makes any decision to be one thing or another; each element is always in “superposition”; each is, in itself, a little bit of everything. In a system where the elements adapt to each other however, local regions will randomly make different local decisions, and consequent different parts of the system will move in different directions; which ensures the system-wide “emergence of diversity”.
Co-Emergence and Integration
The emergence of diversity means that what comes into existence must be in some way complementary, or at least be accommodative to the presence of co-emergent entities. Over time this mutual accommodation fine-tunes the integration of these co-emergent entities. This process of the self-organizing integration of co-emergent diversity leads to the “emergence of integrated diversity”.
The Mystery of Spontaneous Complexity
From the outset, the science of complexity has wrestled with the obvious contradiction of the emergence of complexity in a universe dominated by the SLOT. Consequently, complexity has long tried to address the question “what is the source of spontaneous order?” As it turns out however, Spontaneous Order does not arise in defiance of the SLOT but with the help of it.
All fluid-like adaptive systems are capable of both spontaneous viscosity and spontaneous mixing. It is the collaborative interplay of symmetry-breaking positive reinforcement, and the integrating force of the SLOT that molds formless uniformity into complex structures of integrated co-emergent diversity, which have effectively “self-organized themselves into existence”.
Complexity occurs when internal interactions become too coarse for a system to simply synchronize and stabilize to a single fine-tuned equilibrium. Complexity is simple a form of “Coarse Damping” to uniformity – a form of coarse symmetry! It is
The emergence of symmetry-breaking resistance to the gravitational pull of a finely-tuned and symmetric, but fundamentally bland and featureless thermal equilibrium.
So ultimately complexity is in reality actually a coarse version of the SLOT, it is the SLOT in disguise – “Complexity is Coarse Entropy” driven by “Coarse Emergent Mathematics”.
Part 3.1 — Lock-In + Upheaval
The Second Law of Thermodynamics (SLOT) states that left undisturbed all systems gravitate towards “Thermal Equilibrium”. Not only that, but any system which is disturbed from equilibrium will ultimately be pulled back to equilibrium; so this means that the SLOT actually behaves like a “restoring force”.
The restoring force is a fundamental concept in coarse mathematics; and consequently there are many examples of restoring forces in nature. In self-balancing systems it is the internal feedback that acts as the restoring force. In elastic materials it is the strength of the internal bonds. In a pendulum it is gravity. And in “fluid-like systems” it is entropic force of the SLOT.
Entropic Limit + Lock-in
The restoring force of the SLOT means that all systems (of any type) are all naturally damped systems. Complex Adaptive Systems (CAS) however are “Damped-Driven Systems” – naturally damped by entropic decay of the SLOT, but also driven by “symmetry-breaking positive-feedback”.
Positive feedback, in a CAS, can persist or evaporate over time; which means that emergent structures form and dissolve all the time. In CASs we know that we know that positive feedback can cause the emergence of local imbalance, but such local imbalances are usually offset by wider balancing co-emergent diversity. Complexity relies on this co-emergence; complexity is local imbalance within global balance.
Unrestrained positive feedback however has the potential to disrupt complexity by causing wider regional and global imbalance. Excessive positive feedback within a CAS can over-amplify internal imbalances and drive the system beyond its “Entropic Limit”.
The entropic limit is the elastic limit of the restoring force of entropy; it is the point beyond which reinforcement can no longer be restrained; a point beyond which the natural entropic force of the SLOT is overpowered by emergent positive feedback.
This entropic limit can be considered a tipping point for a system-wide “phase transition” to a domain where positive reinforcement can effectively “hold” a structure far away from equilibrium – a domain where emergent structures can become virtually “Locked-in”.
Skewing the Gaussian
We know from our study of materials that when a restoring force is over-stressed (beyond its elastic limit) a permanent deformation can occur; and in a similar vein, in a CAS when the restoring force of entropy is over-stressed by excessive positive feedback, a mathematical deformation can occur in the system’s Gaussian Distribution.
In mathematics we have something called the Central Limit Theorem (CLT). The CLT deals with distributions of distributions. More specifically the CLT deals with the probable distribution of an underlying probability distribution. It is the CLT that enforces the SLOT; for it is the CLT that relies on the LLN to enforce a Gaussian Distribution.
The CLT says if we sample, a distribution of outcomes of a probabilistic event, a large enough number (LLN) of times then the sample distribution (of the underlying distribution of outcomes) will itself ultimately converge on a Gaussian Distribution – [equally balanced between higher-than-the-average-underlying-distribution-of-outcomes (RLLN+) and lower-than-the-average-underlying-distribution-of-outcomes (RLLN-)…]
As we have seen however, at every level of scale in a CAS, excessive positive feedback can reverse the balancing effect of the LLN in the CLT. The effect of the RLLN on the CLT is to initially widen out a Gaussian distribution up until it reaches the entropic limit – beyond which a “skewing” of the distribution will occur.
It is this potential (in CASs) for the skewing of the balancing Gaussian distribution that can cause the emergence of Power Law Distributions…
The Emergence of the Power Law
In nature there exists, a mathematical “universality” in the behaviour of all sorts of unrelated systems that appears blind to the underlying details of the actual system itself. This mathematical universality is the “Power Law Distribution”, and the reason for its mysterious universality, is simply because it is driven by the universality of “Positive Feedback and the RLLN”.
Power Laws are ubiquitous in Nature! In nature a “Power Law Distribution” usually describes a relationship between the size of something and how often it occurs.
Often we only become aware of this universality as a result of so-called “extreme events”. Power laws are the result of nature’s constant and ubiquitous interplay of formation and decay, growth and decline, build-up and release. Most of the time a minor reversal goes nowhere, but sometimes a minor event can randomly initiate some degree of positive feedback, which itself can sometimes amplify the disturbance into an “extreme event” – large earthquakes and massive forest fires are obvious examples! Different reversals/releases/collapses experience different amounts of positive feedback, and as a result, such events often follow a power law distribution.
This mathematical universality is often mimicked in the behavior of CAS’s. Power laws do not only describe the distribution of positive feedback driven collapse so ubiquitously at play in nature, they can also describe the distribution of positive feedback driven formation causing the emergence of the RLLN in CAS’s
Power laws in CASs, often describe a “skewed distribution of reinforcement”; effectively describing the different degrees of reinforcement (degrees of coarse damping) in different parts of the system. Consequently, the stronger the power law the more unbalanced is the system, and the greater is the potential for excessive positive feedback and System-Wide Lock-in.
Such “locked-in structures” might on the surface look stable, but there is potential for chaos to be hidden in the order….
In CAS’s structures form and dissolve all the time. In CAS’s the whole never stays the same because the parts are constantly changing. In a fast evolving system it is of no surprise therefore, when redundant things gradually disappear, but what is surprising is when something long standing and apparently stable suddenly collapses.
Often things that appear stable are in fact holding themselves away from a more natural fragmented equilibrium by sheer force of their own internal reinforcement. Many structures that have managed to lock themselves in, far away from equilibrium, don’t always stay locked-in. Many such structures can be vulnerable to collapse should their internal dynamics change.
While self-reinforcing structures may on the surface look stable, this is in reality merely short or medium term stability; for long term sustainability of a far from equilibrium structure is pretty much doomed, thanks to the entropic power of the SLOT. Ultimately the SLOT ensures that everything eventual dissolves or collapses – even those things that have had the appearance of absolute solidity for a very long time indeed.
The collapse of structures held away from equilibrium can lead to sudden and violent change. Such upheavals are usually considered shocking and often described as freak events, but in but in truth here are merely an extreme realization of the universal powers laws that govern the distribution of events in all complex systems which have the capacity for internal reinforcement….
Part 3.2 — The Evolution of Complexity
Spectrum of Equilibrium
Complexity is the constant interplay or collaboration between the LLN and the RLLN in the integration of co-emergent diversity. The “evolution of complexity” on the other hand is a constant tug-of-war between the LLN and the RLLN in the formation and upheaval of complex structures.
Fortunately most CAS’s do not swing wildly between excessive reinforcement and total collapse (except of course Asset Markets – and this is mainly because policy-markets do not seem to appreciate how much these markets are vulnerable to “herding” and the RLLN).
In nature the evolution of complexity is a gradual construction, layer upon layer, of integrated diversity; constantly fine-tuned by, co-emergent-adaptation, and entropic-upheaval. And as a result, of this constant push and pull, every CAS lies somewhere on the greyscale between the extremes of the LLN and the RLLN.
Chaos Theory has shown us that there is more to equilibrium than a finely-tuned, highly symmetric, but fundamentally bland and featureless uniformity. Chaos has shown us that there is in fact a “spectrum of equilibrium” on which lie a range of hidden “attractors”, — and each attractor is merely a different evolutionary path, a different version of a previously hidden but now emergent “coarse and complex equilibrium”.
The “Edge of Chaos”
In actuality thermal-equilibrium is itself an attractor; just not a very interesting attractor. Thermal-equilibrium is a “point attractor” which always pulls the system to the same place, regardless of where it starts from.
Positive feedback is the driving force that progress the system away from a thermal-equilibrium in a step by step fashion; and as a result the emergence of “coarse equilibrium structures” tends to be a bottom-up evolutionary process of co-emergence and upheaval leading to constant change; constant rejuvenation.
The Edge of Chaos is the system-wide entropic limit; the balancing point between the self-stabilizing forces of entropy and the self-reinforcing forces of symmetry-breaking positive feedback. The Edge of Chaos would appear to be where nature does its best work, the place where nature combines emergence and upheaval to the optimal effect…
Left to itself many a CAS is capable of self-organizing its own way to the optimal balance between formation and upheaval, between stability and instability – a point of optimal complexity – a point often referred to in complexity literature as the “Edge of Chaos”.
The Edge of Chaos is a point of “criticality”, a point midway between the bland featureless attractor of thermal-equilibrium, and the wild strange attractor of chaotic-equilibrium. It is the point on the spectrum of equilibrium where elements within a complex system tend to optimally combine the traits of being locally reinforced and globally well adapted; a point on the spectrum of equilibrium we could refer to as “The Optimalibrium”…
The Whole is Greater
All emergent (far from equilibrium) structures are self-organized from the bottom-up and as a result can follow “many evolutionary paths”, but the system as a whole has but one evolutionary path; which is an aggregation and integration of the many diverse evolutionary paths taken by the parts of the whole.
Over time these integrated co-emergent entities, constantly fine-tuned by upheaval and rejuvenation, mutually reinforce in a symbiotic relationship; and in so doing the evolutionary whole is forever becoming greater than the sum of its evolutionary parts!
Evolution is the emergent result when a coarsely-reinforcing naturally-damping system constantly tries and re-tries to stabilize to equilibrium. It is the constant interplay, in Complex Adaptive Systems, of the LLN & RLLN, of upheaval and rejuvenation, repeated over and over, at every level of scale, that drives Spontaneous System Self-Organization and the
“Progressive Emergence of Ever Greater Complexity”…