Post-production pipelines rely heavily on image processing and computer vision tasks, processes where machine learning has yet to be disruptive. By integrating the domain statistics from a given training dataset, machine learning-based algorithms may be more efficient at solving existing tasks and enabling new solutions to be integrated into these pipelines.

This paper will present Autodesk’s experience based on developing and integrating a number of machine learning-based algorithms into software for the post-production industry. This algorithm extracts geometry information from a sequence of monocular images of faces. More specifically, the algorithm consists of a deep neural network taking as its input the RGB color image of a face and outputting the associated matte, depth, normal, and UV multi-channels maps which may be used to generate a "3D face”. Such 3D representation permits to apply many different effects to the analyzed face during color grading, including beautification or even relighting.

This algorithm will serve as a case study to review and discuss the multiple challenges posed by the implementation, integration, and deployment of such a machine learning-based algorithm into the workflow.