Deep Intuition

Most higher level animals  are able to identify obvious patterns of cause and effect, mankind especially so; but even we can struggle with the less obvious stuff…

Fast Thinking

Simple systems exhibit obvious cause and effect, complex systems however are much more nuanced.  Over millions of years evolution equipped us with a biological tool to navigate a world of complex nonlinear dynamics.

The brain is a nonlinear tool designed by natural selection for building abstract nonlinear models of a complex nonlinear world.  Over time the subconscious mind builds a repertoire of “nonlinear patterns”, which the conscious mind accesses by means of fast processing “Intuition”.

Some time ago Daniel Kahneman wrote a book called “Thinking, Fast and Slow” in which he essentially suggested that we cannot rely on our fast thinking intuition, that generally speaking while our intuition works well when dealing with simplicity, it tend to lets us down when we are dealing with even the smallest amount of complexity.

This however  does not always have to be the case.  In reality our intuitive fast thinking can in fact be a more valuable tool than our logical slow thinking if we actually train it correctly.  With the right type of training we can develop “Deep Intuition”…

Integration for Free

In his book “Bounce” Matthew Syed argues that “Talent” is not God-given, but must be worked at;  the ultimate result of many long hours of practice.  Virtually anyone who is any good at anything will recognise the truth in these words.  But how exactly does practice make perfect?

Think about what is involved in learning to play tennis to very high level.  People who are bad at tennis essentially play every shot more or less the same way, the forehand is almost the same shot as the backhand, even the serve is essentially just racket meets ball.   People who play tennis well however, have learned to separate one stroke from another; they have “evaluated feedback” from hours and hours of practice to fine-tune the mechanics of each unique stroke.

Strangely enough though, despite this constant focus on training the mechanics of each individual stroke, nobody ever seems to train the transition from one stroke to another — that somehow just seems to come naturally over time.  It seems that co-training a diversity of different strokes means the the integration comes for free…

It is the “integration” of a repertoire of different strokes is what makes the whole greater than the sum of its parts.  And so while “practice focused on feedback” may consolidate the technique of each individual stroke, it is the subconscious mind that combines these finely-tuned strokes into a single greater integrated whole.

Separate + Integrate

Learning to play tennis, or anything else for that matter, is simply a process of conscious separation and subconscious integration repeated over and over again.  In a similar vein, “Intuition” can be thought of as a nonlinear map, or nonlinear network, learnt from the bottom-up by the constant interplay of separation and integration.

In general as humans, we learn to map, or model, the world by trial and error (although trial and evaluation of feedback is probably a more accurate, if clumsier,  representation).  The brain makes sense of the world, the same way a child makes sense of a jigsaw puzzle.   A child will separate out all the edge pieces, separate out all the sky pieces, separate out all the castle pieces etc, etc, and then try to fit them all together.  The brain carries out a similar process of separation and integration; it appears to carries out the separation when it is awake and deeply focused, and the integration when it is asleep and deeply relaxed.  During sleep the mind is likely trying to compress the diversity of information into a single “Integrated Whole”.  [Note:  The information can only be compressed to its maximum compression but no more — this is the basic idea behind something called information entropy].

Language is probably the most obvious example of such bottom up emergence of integrated nonlinearity.  A child does not learn to speak by learning the linear rules; it is with attention, practice and sleep, that nonlinear language ultimately bubbles up to the surface…

This subconscious integrated learning, this bubbling to the surface, is the same process that drives our “Intuitive Pattern Recognition” and so if the premise of Matthew Syed’s argument extends to the nonlinear mind, then any lack of “deep pattern recognition” is simply a lack of both practice and diversity.

Nonlinear Thinking

It is often said that, chance discoveries favours the prepared mind, and so it is with nonlinear thinking, and deep pattern recognition.

Through the interplay of practice and relaxation a fully formed integrated idea can suddenly emerge into consciousness as if by a random thought.  But this is not really a random thought.  This “information structure” has probably been forming in the subconscious mind for a very long time indeed; before ultimately surfacing into consciousness as a eureka “moment of inspiration”.

Such insights about “how things fit together” can seem as if it they come out of nowhere; but this inspiration is actually the end result of the complex integration of a diversity of information, which can ultimately lead to “Deep Intuition”.

Deep Intuition - 006

10,000 Hours

So to become an expert at anything, requires practice.  Learning the rules just doesn’t cut it!   To become an expert we need to build from the bottom-up in order to generate the “emergent integrated nonlinearity”.

But how long does all this surfacing of nonlinearity take?   How long does it take to train our intuitive fast thinking to an expert or genius level?

Well according to Malcolm Gladwell most skills take about 10,000 hours to achieve mastery and this sounds about right and likely also applies for the diversity and integration necessary for deep intuition.

Now this is where it gets interesting.  To learn something to an expert level we need around 10,000 hours of feedback, but for a computer to learn something it simply needs a certain amount of processing power and a great diversity of data.

Artificial Deep Intuition

To date, so-called “expert systems” have been built by encoding top-down rules.  These systems, however, only work in the limited environments of simple rule-based systems.  If we want expert systems that can handle nonlinear complexity, then we need to build such systems from the bottom up…

To date, the only things holding back true Generalized Artificial Intelligence (GAI) were the processing power required to process a bottom-up methodology, and the diversity of data required to feed that methodology.

“Big Data” is a term bandied about a lot these days; however most of the time the people who are using this term are really just writing reports.  Big Whoop!   Very cutting edge!  But despite all of the over-hyped nonsense, there is actually some real value to be had from so-called “big data” from an AI point of view…

Big data, from the AI perspective, means lots of “diverse training data”; it means the digital equivalent of 10,000 hours of practice.  And the thing about digital practice hours is that our digital friends are not operating in the same restricted time environment as us.  With the right amount of data AI systems should be able build deep intuition on just about anything (really really fast)….

So does this mean that our digital servants, are about to become our digital overlords?  The question on the minds of many is “are we on the verge of machines taking over?”