AI has only reached adolescence – everything will change when machines learn to learn

We come into this world pure, helpless, and protected only by some basic reflexes and a survival instinct best demonstrated by an astounding scream that can hit 97 decibels. But with our mother’s first touch, the learning process begins, and in many ways consumes us for the rest of our lives. Living is learning.

Many believe that even Einstein started out with tabula rasa, a clean slate. An absence of innate, preconceived ideas or predetermined goals. Just like machines in fact. Every piece of functioning apparatus comes into the world essentially as if it was born yesterday. Dumber than Dumb and Dumber. 

But it is this state, of course, that represents the starting block for machine learning, the vital subcategory of artificial intelligence that enables computers to act without being explicitly programmed. Although its precise birthdate is hazy, the science was christened ‘machine learning’ by IBM’s Arthur Samuel in 1952. 

This ever-more ubiquitous process uses algorithms and neural network models to progressively improve the performance of computer systems. The algorithms automatically build a mathematical model using sample data – called training data – to make its own decisions.  

That phrase, ‘progressively improve the performance of computer systems’ is central to the theme of this article. The need to improve performance, to train machines faster, more efficiently and at far less cost, is a commercial imperative being felt across the world’s increasingly intelligent industries.  

Limitless potential of AI technology 

I work with more than 900 colleagues here at Cambridge Consultants – engineers, designers, scientists, technologists and consultants – all helping our clients to push boundaries across innumerable fields, from smart industrial connectivity to bioinnovation; from next generation medical devices to the digitalisation of products and services. The list goes on. AI-powered technology, dependent on machine learning, has limitless potential to propel these, and every other conceivable sector. Quite plainly, progressive gains in machine learning will bring huge rewards. 

There is a clear and obvious end game here, because when we create machines that perceive and think for themselves, everything will change. As I’m one of the lucky ones concerned with helping to advance the journey, let me share a little insight into our current technical direction and some of the research challenges involved. 

The tabular rasa state of machines is a key point and poses the main conundrum. Sophisticated systems are created around the world, usually well over-spec’d, that are capable of doing much more than is asked of them. But to achieve more, they need to be exposed to many hundreds of thousands, if not millions, of training examples for each and every task.  

Yet a human child, and indeed an animal, can utilise what they’ve learned in previous tasks. They can get shown how to do something once and immediately ‘get it’ by drawing on building blocks of learning from past experiences. In simple terms, they learn naturally. But in the technology sphere, each system is trained from scratch, on a single task or on a single data set. 

Learning more naturally  

Much of our current focus is to advance this situation by getting machines to learn more naturally. Let’s look at some of the ways. It can mean learning from limited data – and considerably less data than we’ve used hitherto. One approach that we’ve made great strides in here at Cambridge Consultants is generative adversarial networks, or GANs as they’re known. The idea here is to massively expand your training data synthetically from a core set of real world data.  This can be much cheaper and quicker than collecting a large dataset entirely from the real world. 

The ‘adversarial’ bit comes from the process of pitting one neural network against the other to generate new synthetic data in lieu of the real data. I always think of this as being very similar to the way we ordinary humans dream, or elite athletes visualise success. We generate our own training data simply by remembering, imagining or dreaming about something. And we learn from it. 

There are a host of other subsets in this area. Synthetic data rendering is all about using gaming engines or computer graphics to render useful data. Domain adaption is a complex one to explain, but the gist of it is this: if you collect most of your data in winter but want to use it in summer, there is still a huge amount of useful, transferable knowledge that can be learned from the winter data. 

Few shot learning is a really fascinating one and has a child-like feel about it. Think of it like this. If I’ve learned what a dog is and I’ve learned what a cat is, you can show me a cow and I can work out that it’s neither of the first two and is something else but similar. And trust me, that’s a much quicker way of learning what a cow is. 

Other approaches take a different route from learning from limited data. One of them, multitask, is about machines working just as humans do. Let’s imagine trying to navigate a strange house to find an apple hidden somewhere inside. In the case of AI, the system will be given the task and have to figure out each step completely from scratch.  

But that’s not what humans do. If we’re searching a large house, we already know what chairs, hallways and doors are. We know how to plan, how to walk, and what an apple is. Applying this multitask model to machine learning allows much more complex systems to be built. The achievement of the goal goes hand in hand with object recognition – so next time round a task to find a banana will be much easier. 

Learning without labels 

Most machine learning is currently supervised, involving input and target pairing labels. The system is provided with an input and a known answer – the apple and the task finding it, for example. But now, progress is being made in unsupervised, semi-supervised and self-supervised learning, all essentially about learning without having all data labelled. 

Clustering is a good example of using unsupervised learning to discover data characteristics that can prove useful down the line. An algorithm might cluster people into ten different groups with similarities that may or may not be labelled. But you can subsequently examine the clusters and effectively discover the system’s thinking.  

This trend is really significant for commercial reasons. It could be argued that the labelling process of input to output pairing is what’s holding a lot of machine learning back right now. It’s a time-consuming and expensive business – indeed, a whole industry has built up around it – and the less of it you have to do the better.  

Just as it sounds, semi-supervised learning is a mix of labelled and unlabelled – supervised and unsupervised learning at the same time. If you have, say 10 per cent of your data labelled and the rest is unlabelled, you’ve saved 90 per cent of your costs. The judgement to make, of course, is the level of performance against the 100 per cent expenditure. 

I find self-supervised is a little more interesting with its similarities to the multitask approach. Briefly, it’s about the data essentially supervising itself. If you’re detecting cat and dog images, you could also set tasks for the algorithm to learn further characteristics of the data. Then ask it to take the colour image, turn it monochrome – and then successfully put the colours back. This involves a lot of data understanding. 

The meta leap 

So much for incremental advances, now for the giant leap. Meta learning, essentially learning to learn, is the next, coming horizon for machine learning. Frankly, it has the potential to blow everything I’ve been talking about so far clean out of the water. Its approach will of course deepen our ethical, philosophical and moral debates as it takes us a step closer to the point where artificial intelligence behaves just like humans. 

As I said at the beginning, we spend an awful lot of time learning how to learn. Teach something how to learn, and it can take on any task and achieve anything. If we think of AI only now reaching adolescence, there’s a lifetime of learning still to come. And plenty to talk about in my next article. Meanwhile, please email me if you’d like to discuss any aspect of the topic in more detail.