Advancements in 3D Facial Expression Reconstruction (Talk)
Recent advances in 3D face reconstruction from in-the-wild images and videos have excelled at capturing the overall facial shape associated with a person's identity. However, they often struggle to accurately represent the perceptual realism of facial expressions, especially subtle, extreme, or rarely observed ones. In this talk, we will present two contributions focused on improving 3D facial expression reconstruction. The first part introduces SPECTRE—"Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos"—which offers a method for precise 3D reconstruction of mouth movements linked to speech articulation. This is achieved using a novel "lipread" loss function that enhances perceptual realism. The second part covers SMIRK—"3D Facial Expressions through Analysis-by-Neural-Synthesis"—where we explore how neural rendering techniques can overcome the limitations of differentiable rendering. This approach provides better gradients for 3D reconstruction and allows us to augment training data with diverse expressions for improved generalization. Together, these methods set new standards in accurately reconstructing facial expressions.
Biography: Panagiotis Filntisis is a Postdoctoral Researcher at the National Technical University of Athens and the Athena Research and Innovation Center. His doctoral thesis focused on multimodal deep learning models for emotion recognition and facial expression synthesis, with applications in human-robot interaction. Currently, he specializes in 3D computer vision, human modeling, and affective computing, aiming to enhance the interaction between humans and machines through advanced emotional understanding. George Retsinas received his Ph.D. degree from the Department of Electrical and Computer Engineering of National Technical University of Athens in collaboration with the research center “Demokritos”. The main research directions during his Ph.D. were document analysis and recognition, with emphasis on machine learning approaches, as well as compression of deep neural networks. Today he is a Post-Doctoral Researcher, focusing on vision applications in robotics, including 3D vision applications, in National Technical University of Athens, collaborating with the Athena Research Center.
