AI Photoshop
© Screenshot NVIDIA/YouTube
No matter how much time some people spend lulled into quiet artistic contemplation by syndicated episodes of The Joy of Painting with Bob Ross, their efforts will never be better than an acrylic on canvas nightmare you might find in a thrift shop hidden under a soiled rug. The ability to transfer onto a paper what one sees either in one's mind or with one's eyes is, after all, a skill that seems to require equal parts diligence, discipline, and DNA. Some people just take to it. Others... not so much.

As with so many other first world problems, however, technology has leveled the playing field. NVIDIA Research has developed GauGAN (yes, that's an intentional reference to the post-impressionist painter Paul Gauguin): a deep learning model that allows you to turn the most basic of sketches into photorealistic masterpieces that Ansel Adams would envy. We're talking WAY beyond Bob Ross (Rest In Paint, sir) here.

So how does GauGAN take shapeless blobs of color and turn them into mountains and shimmering Alpine landscapes? Through an enhanced form of deep learning known as Generative Adversarial Networks (GANs). The best description of how a GAN works we could find was provided by skymind.ai: "One neural network, called the generator, generates new data instances, while the other, the discriminator, evaluates them for authenticity; i.e. the discriminator decides whether each instance of data that it reviews belongs to the actual training dataset or not."

In the case of GauGAN, it would seem that NVIDIA's GAN learned by creating images that were compared to real images by a discriminator network. As such, GauGAN "knows" what a field or forest would look like in whatever certain shape you provide. You make the sketch, tell GauGAN where everything should go, and then the program fills in all the details.

"It's like a coloring book picture that describes where a tree is, where the sun is, where the sky is," said Brian Catanzaro vice president of applied deep learning research at NVIDIA. "And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images."

Take a look at the video to see more.