Post-production pipelines rely heavily on image processing and computer vision tasks, processes where machine learning has yet to be disruptive. By integrating the domain statistics from a given training dataset, machine learning-based algorithms may be more efficient at solving existing tasks and enabling new solutions to be integrated into these pipelines.
This paper will present Autodesk’s experience based on developing and integrating a number of machine learning-based algorithms into software for the post-production industry. This algorithm extracts geometry information from a sequence of monocular images of faces. More specifically, the algorithm consists of a deep neural network taking as its input the RGB color image of a face and outputting the associated matte, depth, normal, and UV multi-channels maps which may be used to generate a "3D face”. Such 3D representation permits to apply many different effects to the analyzed face during color grading, including beautification or even relighting.
This algorithm will serve as a case study to review and discuss the multiple challenges posed by the implementation, integration, and deployment of such a machine learning-based algorithm into the workflow.
Technical Depth of Presentation
Technical depth of the presentation will be intermediate.
What Attendees will Benefit Most from this Presentation
Ideal audience range from engineers interested in the technological details of machine-learning to the managers interested in using this technology in their projects.
Take-Aways from this Presentation
Takeaways for the audience:
- understanding the challenges posed by the implementation, integration, and deployment of a machine-learning algorithm
- a case-study using machine-learning to extract geometry from images of faces
- advice on how to approach fitting machine-learning models to post production image datasets