Sparse2dense: Sparse to Dense Dynamic 3D Facial Expression Generation

This is an official repository of the paper Sparse to Dense Dynamic 3D Facial Expression Generation. https://arxiv.org/abs/2105.07463

Results

• 4D Facial expression generation

• Interpolation between 4D facial expressions

• Speech transfer

Usage

The code is divided into two folders:

Motion3DGAN: used to train a GAN generator of 3D landmarks motion;

S2D: used to train and test the decoder that deforms the 3D mesh according to a given landmarks configuration;

You will find a Readme file inside each one of these folders with the installation and the usage instructions.

Models

Please download models from the link below and include them in S2D\Models folder.

https://drive.google.com/drive/folders/1-RdBhUfP7JcxihVCT4AdLRYfSBOD9Rmq?usp=sharing

Acknowledgments

This work was supported by the French State, managed by National Agency for Research (ANR) National Agency for Research (ANR) under the Investments for the future program with reference ANR-16-IDEX-0004 ULNE and by the ANR project Human4D ANR-19-CE23-0020. This paper was also partially supported by European Union’s Horizon 2020 research and innovation program under grant number 951911 - AI4Media.

Reference

Please cite the following paper if you use the code directly or indirectly in your research/projects.

@inproceedings{otberdout2022sparse,
title = {Sparse to Dense Dynamic 3D Facial Expression Generation},
author = {Otberdout, Naima and Ferrari, Claudio and Daoudi, Mohamed and Berritti, Stefano and Del Bimbo, Alberto},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
month = jun,
year = {2022},
}