Turning our sketches into art with machine learning

“Vincent,” a deep learning demonstration from Cambridge Consultants, builds on human input to create completed ‘works of art’

Product design and development firm Cambridge Consultants has developed “Vincent,” a breakthrough in machine learning that is capable of completing a drawing that has been started with a human sketch. Completed ‘works of art’ combine a users’ sketch with the digested sum of art since the renaissance, as if Van Gogh, Cézanne and Picasso were inside the machine, producing art to order.

Machine learning has made its way into the arts, having the ability to create an image or song based on sound or pictures it has seen before. But never before has artificial intelligence (AI) had the ability to interpret what a human is drawing and then complete the piece for them. Beyond simple machine-generated art, Vincent is an engaging, interactive system in which the output is guided and influenced by the user.

Based on multiple Generative Adversarial Networks (GANs), Vincent was shown thousands of paintings from the Renaissance period to the current day to study the works and build an understanding of where contrast, colour and texture change. Now trained, Vincent is able to interpret important edges in paintings and uses this understanding to produce a complete picture.

“What we’ve built would have been unthinkable to the original deep learning pioneers,” said Monty Barlow, director machine learning at Cambridge Consultants. “By successfully combining different machine learning approaches, such as adversarial training, perceptual loss, and end-to-end training of stacked networks, we’ve created something hugely interactive, taking the germ of a sketched idea and allowing the history of human art to run with it.”

To interact with Vincent, a user simply draws directly to a tablet. Vincent will interpret the different lines being drawn and pick up where the user left off – taking as much information as it’s given to build out the piece into a completed picture. Unlike typical machine learning approaches which simply use mathematics to generate approximations of art, Vincent is able to take the human input given and produce a relevant, finished artwork.

Potential applications for Vincent-like technology reach far beyond art, with autonomous vehicles and digital security being early use-cases. Here, the technology could be used to generate training scenarios and simulations, introducing almost limitless variation and convincing detail beyond what humans could efficiently produce.

“We’re exploring completely uncharted territory – much of what makes Vincent tick was not known to the machine learning community just a year ago” said Barlow. “We’re excited to be at the leading edge of an emerging, transformative industry and to be making the leap from the art of the possible to delivering practical machine learning solutions for our clients.”

Vincent is the latest output from Cambridge Consultants’ Digital Greenhouse – a unique experimental environment where data scientists and engineers explore and develop cutting edge machine learning techniques.

Cambridge Consultants will be demonstrating its Vincent technology at the RE•WORK Deep Learning Summit, September 21 in London.