Jun 23, 2022
No image
Completed

3D head reconstruction solution

$50,000+
2-3 months
Germany
2-5
Service categories
Service Lines
IT Consulting
Artificial Intelligence
Other
Domain focus
Other
Programming language
Python
Other
Frameworks
Other
Subcategories
Artificial Intelligence
Machine Learning

Challenge

The customer’s agency had 100+ in-house 3D designers to process every image. In the past, the flow used to involve designers processing images from clients. The pictures could be poor quality, from social networks or family albums. Then designers took stock prop of heads, hands, torsos, etc., and sculpted a 3D model that suited the sent image as much as possible. The image was overlaid on top of the created model as a texture. Every image took 15 minutes to process.

Solution

We did not take data from designers because they considered the visible side of the head only, thus their 3D models were not symmetrical and had a non-constant number of vertices. We had to use something other than the state-of-the-art solutions like 2D to 3D neural networks because: ► we were missing a proper dataset; ► the neural network had a huge size and an enormously long training time; ► even the state-of-the-art solution had quite low-quality results for the provided examples.

Results

► Take any random 2D photo shot as an input; ► Detect the face with RetinaFace deep face detection library; ► Estimate the rotation with FSA-Net neural network; ► Find 2D landmarks with FAN2D network; ► Estimate the scale; ► Estimate PCA coefficient for Nvidia ICT-FaceKit head models; ► Create a full 3D object that looks similar to the original face on the photo and is properly scaled and positioned.