PTDC/EIA-CCO/108791/2008

The human face has been the subject of tremendous scrutiny in the fields of human cognition, computer vision, image processing and computer graphics, and is perhaps the most extensively researched facet of our body. This is hardly surprising given the importance of its function in social interactions. The face is the primary medium for conveying identity, cognitive state, emotive intent and disseminating affective responses. Although humans use language as the main channel for conveying facts and schemes, it does not match the face’s capacity for communicating emotions: - “The face is rich in communicative potential. It is the primary site for communicating emotional states; it reflects interpersonal attitudes...” - The study of how facial expressions transmit affective information was pioneered in behavioural psychology by Ekman and his associates. From these studies it was discovered that facial expressions are uniquely human, basic emotional expressions are consistent in exhibition and interpretation across all demographics, and emotional messages are constructed by the actions of certain facial muscles. - Due to its complex dynamic nature, the study of the face from a computational perspective has resulted in increasingly more sophisticated tools in non-rigid object modelling and tracking, object parameterisation and recognition, occlusion handling and extracting invariant features in real-world settings. Automated facial analysis has been motivated by some very desirable applications, which consider both its static nature, such as identity recognition, and its dynamic nature, such as expression recognition, visual speech recognition and the realistic animation of virtual humans. This analysis has also paved new insights and tools in medical applications, behavioural science, security, education and human computer interaction. - Much of this analysis has been conducted on static2D or 3Dimages or short2D image sequences. However there has been very little work in investigating facial dynamics in video-rate 3D data. The advantages of 3D over 2D data in pattern recognition tasks have been largely considered as a means to overcome variations in pose and illumination. However, 3D information over time also provides us with a complete description of how an object deforms in 4D spatiotemporal space without the loss of information which is incurred as part of the 2D image projection process. With respect to the human face, one important use for this is to analyse the ways in which individuals can, or are able to deform their face while performing expression or speech. We can use this analysis to explore the similarities and idiosyncrasies of facial motion across individuals, which has important applications in physiological and clinical studies. We can also use this to generate individualized dynamic facial models for highly realistic animation. Another direction of research stems from the question of whether it is possible to characterise an individual based on their facial motion. For example, how can we quantify the similarly between two peoples’ smiles? Which expressions best discriminate individuals? Is it possible to build a prototypical 4D model of how people smile? How are individual differences reflected as a deviation from this model? To the best of our knowledge, these questions have received a very small amount of attention in 2D image analysis, and none in 3D sequences. From a biometrics point of view, the concept of recognising a person based on facial motion is attractive; since facial movements comprise a complex sequence of muscle activations, it is almost impossible to imitate another person’s facial expressions and these facial motion characteristics are unique to an individual. Furthermore, the use of facial motion is in certain aspects more robust to fraudulent attacks than current static face recognition algorithms, which fail if presented with a physical model of a person’s face. Facial dynamics are also independent of lighting, pose and appearance changes (such as wearing make-up), which are apparent in real-world environments. In experimental psychology, determining the precise role of facial motion in determining identity is still largely unknown, and is actively pursued. By exploring this notion using computer vision techniques, we will be able to evaluate the strength of the dynamic cue in identity recognition.