Reconstruction and Animation of Realistic Head Avatars

Digital humans, or realistic avatars, are a centerpiece of future telepresence and special effects systems, and human head modeling is one of their main components. The abovementioned applications, however, are highly demanding in terms of realism, control, and rendering speed, as the resulting 3D avatar should be personalized, rendered in high resolution, and controlled using a wide range of expressions. This talk will focus on the approaches that create controllable and detailed 3D head avatars using the data from consumer-grade devices, such as smartphones, in an uncalibrated and unconstrained capture setting. We will discuss leveraging in-the-wild internet videos and synthetic data sources to achieve the high levels of facial control and appearance personalization required by the downstream applications. We focus on two specific topics: high-fidelity image-based facial reconstruction and animation and video-based reconstruction of detailed structures such as hairstyles. We also showcase how the resulting human-centric assets can be integrated into virtual environments for real-time telepresence and entertainment applications, illustrating the future of digital communication and gaming.