One of the best things about Washington D.C. is its public art museums. There are about a dozen or so world class galleries where you are allowed to take photos and use the work in your own art, because after all, we the people own the paintings. Excited by the possibilities of deep learning and how well style transfer was working, the kids and I went to the National Portrait Gallery. for some inspiration.
One of the first things that occurred to us was a little inception like. What would happen if we applied style transfer to a portrait using itself as the source image. It didn't turn out that well, but here are a couple of those anyways.
While this was idea of a dead end, the next idea that came to us was a little more promising. Looking at the successes and failures of the style transfers we had already performed, we started noticing that when the context and composition of the paintings matched, the algorithm was a lot more successful artistically. This is of course obvious in hindsight, but we are still working to understand what is happening in the deep neural networks, and anything that can reveal anythign about that is interesting to us.
So the idea we had, which was fun to test out, was to try and apply the style of a painting to a photo that matched the painting's composition. We selected two famous paintings from the National Portrait Gallery to attempt this, de Kooning's JFK and Degas's Portrait of Miss Cassatt. We used JFK on a photo of Dante with a tie on. We also had my mother pose best she can to resemble how Cassatt was seated in her portrait. We then let the deep neaural net do its work. The following are the results. Photo's courtesy of the National Portrait Gallery.
Farideh likes how her portrait came out, as do we, but its interesting that this only goes to further demonstrate that there is so much more to a painting than just its style, texture, and color. So what did we learn. Well we knew it already but we need to figure out how to deal with texture and context better.