Authors
Chen Cao, Hongzhi Wu, Yanlin Weng, Tianjia Shao, Kun Zhou
State Key Lab of CAD&CG; Zhejiang University
Portals
Abstract
We present a novel image-based representation for dynamic 3D avatars, which allows effective handling of various hairstyles and headwear, and can generate expressive facial animations with fine-scale details in real-time. We develop algorithms for creating an image-based avatar from a set of sparsely captured images of a user, using an off-the-shelf web camera at home. An optimization method is proposed to construct a topologically consistent morphable model that approximates the dynamic hair geometry in the captured images. We also design a real-time algorithm for synthesizing novel views of an image-based avatar, so that the avatar follows the facial motions of an arbitrary actor. Compelling results from our pipeline are demonstrated on a variety of cases.
Contribution
- We introduce a novel image-based representation for dynamic 3D avatars, which allows effective handling of various hairstyles and headwear, and can generate expressive facial animations with fine-scale details
- We develop algorithms for creating an image-based avatar from a set of sparsely captured images of a user. In particular, we propose an optimization method to construct a topologically consistent morphable model to approximate the dynamic hair geometry in the captured images
- We design a real-time algorithm for synthesizing novel views of an image-based avatar so that the avatar follows the facial motions of an arbitrary actor.
Overview
From captured images of the user that sparsely sample over head poses and expressions, we build a head blendshape and a morphable hair model to represent the corresponding coarse geometry. Other components such as eyes and teeth are expressed as billboards. During runtime, an actor can drive our avatar using a single-camera-based face tracker. The captured images are warped and blended under the guidance of the coarse geometry to generate real-time facial animation with fine-scale details.