Jun 23, 2022
No image
Completed
3D head reconstruction solution
$50,000+
2-3 months
Germany
2-5
Service categories
Service Lines
Artificial Intelligence
IT Services
Domain focus
Other
Programming language
Python
Challenge
The customer’s agency had 100+ in-house 3D designers to process every image.
In the past, the flow used to involve designers processing images from clients. The pictures could be poor quality, from social networks or family albums.
Then designers took stock prop of heads, hands, torsos, etc., and sculpted a 3D model that suited the sent image as much as possible. The image was overlaid on top of the created model as a texture. Every image took 15 minutes to process.
The customer’s agency had 100+ in-house 3D designers to process every image.
In the past, the flow used to involve designers processing images from clients. The pictures could be poor quality, from social networks or family albums.
Then designers took stock prop of heads, hands, torsos, etc., and sculpted a 3D model that suited the sent image as much as possible. The image was overlaid on top of the created model as a texture. Every image took 15 minutes to process.
Solution
We did not take data from designers because they considered the visible side of the head only, thus their 3D models were not symmetrical and had a non-constant number of vertices.
We had to use something other than the state-of-the-art solutions like 2D to 3D neural networks because:
► we were missing a proper dataset;
► the neural network had a huge size and an enormously long training time;
► even the state-of-the-art solution had quite low-quality results for the provided examples.
We did not take data from designers because they considered the visible side of the head only, thus their 3D models were not symmetrical and had a non-constant number of vertices.
We had to use something other than the state-of-the-art solutions like 2D to 3D neural networks because:
► we were missing a proper dataset;
► the neural network had a huge size and an enormously long training time;
► even the state-of-the-art solution had quite low-quality results for the provided examples.
Results
► Take any random 2D photo shot as an input;
► Detect the face with RetinaFace deep face detection library;
► Estimate the rotation with FSA-Net neural network;
► Find 2D landmarks with FAN2D network;
► Estimate the scale;
► Estimate PCA coefficient for Nvidia ICT-FaceKit head models;
► Create a full 3D object that looks similar to the original face on the photo and is properly scaled and positioned.
► Take any random 2D photo shot as an input;
► Detect the face with RetinaFace deep face detection library;
► Estimate the rotation with FSA-Net neural network;
► Find 2D landmarks with FAN2D network;
► Estimate the scale;
► Estimate PCA coefficient for Nvidia ICT-FaceKit head models;
► Create a full 3D object that looks similar to the original face on the photo and is properly scaled and positioned.