Occlusion-aware face image analysis

Analysis-by-Synthesis is a conceptually elegant and powerful approach for
image analysis. The idea is to start with a generative, probabilistic model
of the image, and to adapt the model parameters such that the generated
instances are close to the observed image.
In practical applications, the approach has proven to be difficult to
apply. One of the big challenges is missing data and outliers which often
occur in real-world scenarios.
This talk will focus on face image analysis where this challenge arises as
occlusions, caused by various objects such as facial hair, glasses or even
objects not directly related to faces. I present an occlusion-aware and
fully probabilistic approach for adaptation of a three-dimensional
statistical model of faces to single 2D images.
I will start by presenting the 3D Morphable Face Model which is a combined
Gaussian shape and color model. To generate images a camera and
illumination model is incorporated in a computer graphics pipeline. To
analyze a face image we search for model parameters rendering an image
close to the observed target image. This posterior distribution of
suitable instances is explored by a Markov chain Monte Carlo method. We
handle occlusions by integrating Markov Random Field segmentation into face
and non-face during the model adaptation process. The segmentation of face
and non-face is solved together with the Morphable Model adaptation using
an EM-like algorithm.