X2C: A Large-Scale Benchmark for Nuanced Humanoid Facial Expression Imitation

Peizhen Li*, Longbing Cao☆, Xiao-Ming Wu, Runze Yang, Xiaohan Yu
*Project lead, ☆Advisor
Paper Code Dataset arXiv

Demonstration of X2CNet, a framework enabling realistic imitation of nuanced facial dynamics. Unlike traditional methods that rely on sparse landmarks, our approach leverages the X2C dataset to learn a direct mapping from visual cues to continuous mechanical control values. The framework demonstrates robust cross-embodiment generalization, applicable to multiple humanoid robots with different facial appearances and actuation schemes.

Qualitative Results
Qualitative Results on Physical Robot. The system demonstrates robust cross-subject generalization, accurately transferring diverse facial configurations—including gaze shifts, eye closure, and asymmetric mouth movements—from human inputs to the Ameca humanoid robot. The results highlight the ability to capture fine-grained dynamics often lost by landmark-based methods.
Cross-Embodiment
Cross-Embodiment Generalization. Demonstration of the framework's adaptability. The motion transfer module standardizes diverse human inputs, allowing the control policy to drive different humanoid embodiments effectively despite variations in facial geometry and appearance.

Abstract

Fine-grained facial expression transfer from humans to humanoid agents presents a unique pattern recognition challenge due to the significant domain gap between biological facial dynamics and mechanical control spaces. While visual synthesis of talking heads has advanced rapidly, mapping high-dimensional visual cues to precise, physically constrained actuation signals remains an open problem, primarily due to the lack of large-scale paired data. To bridge this gap, we introduce X2C (Expression to Control), a comprehensive benchmark dataset comprising 100,000 <image, control value> pairs. Unlike existing resources, X2C features nuanced, physically grounded expressions annotated with 30 continuous control parameters, establishing a high-fidelity standard for this task. Building on this resource, we propose X2CNet, a two-stage deep learning framework that explicitly decouples visual motion features from mechanical control regression to model the correspondence between human perceptual cues and humanoid actuation. Extensive experiments, including quantitative benchmarking and real-world physical validation, demonstrate that our approach achieves superior cross-domain consistency and enables robust, in-the-wild expression imitation.

The X2C Benchmark

Dataset Samples
Representative samples from the X2C dataset. Each sample consists of: (1) a virtual robot rendered in a high-fidelity simulation environment (middle), and (2) the corresponding ground-truth control vector (bottom). In the visualization, blue bars represent the active control magnitude, while orange dots indicate the neutral resting state. The dataset covers complex, asymmetric expressions beyond basic emotion categories.

Data Collection Pipeline

Pipeline
Pipeline for dataset collection. We curate humanoid facial expression animations covering a wide spectrum of emotional dynamics. To ensure precise alignment, we analytically sample the ground-truth control values from the animation curves at the exact timestamps of the rendered video frames, eliminating temporal drift and sensor noise.

Control Space Analysis

1. Semantic Grouping

Control Mapping
Anatomical Organization. The 30-dimensional control vector is organized by anatomical region: Brows (4), Eyelids & Gaze (6), Nose (1), Mouth (14), and Head & Neck (5).

2. Statistical Distribution

Control Distribution
Value Distributions. Violin plots showing the statistical distribution of all 30 control channels across the 100,000 samples. The high variance in expressive regions (e.g., Mouth and Brows) confirms that the dataset explores the full dynamic range of the hardware rather than clustering around the neutral state.

The X2CNet Framework

X2CNet Architecture
Overview of X2CNet. The framework consists of two stages: (1) A Motion Transfer Module that warps a source robot image to match the driving human expression dynamics, and (2) A Control Mapping Network that regresses continuous actuation values from the synthesized humanoid facial expression. This decoupled design ensures that the control regressor operates on a consistent visual domain.