AI Creates 3D Models From Faces | Two Minute Papers #149

AI Creates 3D Models From Faces | Two Minute Papers #149


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. How cool would it be to be able to place a
character representing us in a digital film or a computer game? Of course, it would clearly be an extremely
laborious task to digitize the 3D geometry and the albedo map of our face. This albedo map means a texture, a colored
pattern that describes how our skin reflects and absorbs light. Capturing such a representation is clearly
a very lengthy and expensive process. So get this completely crazy idea: this technique
creates this full digital representation of any face from no more than one simple photograph. We can even get historical figures in our
digital universe, all we need is one photograph of them. And now, feast your eyes on these incredible
results. After taking a photograph, this technique
creates two of these albedo maps: one is a complete, low frequency map, which records
the entirety of the face, but only contains the rough details. The other albedo map contains finer details,
but in return, is incomplete. Do you remember the texture synthesis methods
that we discussed earlier in the series? The input was a tiny patch of image with a
repetitive structure, and after learning the statistical properties of these structures,
it was possible to continue them indefinitely. The key insight is that we can also do something
akin to that here as well – we take this incomplete albedo map, and try to synthesize the missing
details. Pretty amazing idea indeed! The authors of the paper invoke a classical
learning algorithm, a convolutional neural network to accomplish that. The deeper the neural network we use, the
more high-frequency details appear on the outputs, or in other words, the crisper the
image we get. In the paper, you will find a detailed description
of their crowdsourced user study that was used to validate this technique, including
the user interface and the questions being asked. There are also some comparisons against PatchMatch,
one of the landmark techniques for texture synthesis that we have also talked about in
an earlier episode. It’s pretty amazing to see this Two Minute
Papers knowledge base grow and get more and more intertwined. I hope you’re enjoying the process as much
as I do! Also, due to popular request, the Two Minute
Papers T-shirts are now available! This time, we are using a different service
for printing these shirts, please give us some feedback on how you liked it. I’ve put my e-mail address in the video description. If you attach a photo of yourself wearing
some cool Two Minute Papers merch, we’ll be even more delighted! Just open twominutepapers.com and you’ll immediately
have access to it. This link will bring you to the service that
ships to the US. The link for shipping outside the US is available
in the video description. Thanks for watching and for your generous
support, and I’ll see you next time!

53 thoughts on “AI Creates 3D Models From Faces | Two Minute Papers #149”

Leave a Reply

Your email address will not be published. Required fields are marked *