Head Avatars with Dynamic Explicit Hair

DynHair: Head Avatars with Dynamic Explicit Hair

Paper: Head Avatars with Dynamic Explicit Hair
Authors: Vanessa Sklyarova, Haonan Chen, Berna Kabadayi, Tobias Kirschstein, Zicong Fan, Xi Wang, Gerard Pons-Moll, Matthias Niessner, Marc Pollefeys, Michael J. Black, Justus Thies
Affiliations: ETH Zurich, MPI for Intelligent Systems, MPI for Informatics, TU Munich, TU Darmstadt, Microsoft
Links: arXiv:2607.23861 | Project Page
Code: Open source at dynhair.is.tue.mpg.de

One-Sentence Summary

DynHair learns an explicit strand-based dynamic hair model from multi-view video, using an LSTM to encode head angular velocity, acceleration, and relative gravity as temporal conditioning, and FiLM modulation of per-point strand features to predict physically plausible displacements, enabling animatable head avatars with controllable hair dynamics.

Background and Motivation

Photo-realistic 3D digital human modeling has broad applications in e-commerce, entertainment, and AR/VR telepresence. Recently, NeRF and 3D Gaussian Splatting (3DGS) have driven rapid progress in head avatar research, with advances in visual quality, runtime efficiency, few-shot reconstruction, and appearance disentanglement. Gaussian Head Avatar (GHA) conditions Gaussians on BFM parameters with a super-resolution module, GaussianAvatars binds Gaussians to a FLAME mesh for controllable animation, and FlashAvatar embeds Gaussians in FLAME UV space. However, these methods focus primarily on appearance quality, paying insufficient attention to realistic motion—especially hair dynamics.

Current head avatar methods mostly use 3D morphable models (e.g., FLAME, BFM) to handle expression-dependent deformations but treat hair implicitly as part of the head representation, ignoring dynamic hair motion. This leads to unrealistic avatars: when the head rotates, hair should follow gravity; when the head shakes rapidly, hair should lag due to inertia, swing, and exhibit physically plausible follow-through and settling behavior. Current methods move hair as a rigid whole with the head, lacking these dynamic behaviors, severely breaking immersion during animation.

Physics-based simulation can model hair dynamics but requires strand-based hair reconstruction and appropriate physical parameters (mass, stiffness, damping, friction, etc.). Recovering strand-based reconstruction from images is extremely challenging because most hair strands are occluded and only the outer layer is visible. Existing strand reconstruction methods (Im2Haircut, GaussianHair, MonoHair, DiffLocks, Vid2Haircut) assume static hair geometry and exploit learned hairstyle priors to reconstruct internal structure. They can animate with reconstructed hairstyles, but physics simulation parameters must be hand-tuned and cannot be automatically recovered from observations. Automatically finding realistic physics parameters remains an unsolved problem, as it requires efficient differentiable hair simulation, which current differentiable simulators struggle with in terms of computational efficiency and numerical stability.

On the other hand, learning-based hair dynamics methods (e.g., GroomGen, DGH, Quaffure) train neural networks on data generated by physics simulators to approximate simulation behavior. While avoiding online simulation, these methods still depend on simulator-generated training data, indirectly inheriting its physical model assumptions. Learning hair dynamics directly from image observations is a less explored direction. DynHair proposes a different path: learning hair dynamics directly from multi-view video data, without relying on physics simulation or simulator-generated data.

DynHair captures appearance through strand-aligned 3D Gaussian Splatting, initializes the canonical hairstyle from the Im2Haircut prior, and uses a temporal motion model to predict per-frame strand deformations based on head motion history. The key component is a temporal motion model conditioned on gravity, head velocity, and acceleration—extracting angular velocity, acceleration, and relative gravity, encoding a sliding window through an LSTM to produce a motion embedding, then using FiLM to modulate per-point strand features, and finally a strand MLP to predict Gaussian displacements. This approach combines the rendering quality of Gaussian-based avatars with the geometric fidelity and physical plausibility of explicit strand representations, requiring neither a physics simulator nor hand-tuned parameters.

Method Details

DynHair disentangles the dynamic head avatar into two regions: hair and upper body (including face), each with different motion modeling and appearance representation. Hair is represented with explicit strand polylines and strand-aligned Gaussian primitives, while the upper body follows Gaussian Head Avatar (GHA) with unstructured 3D Gaussians. At each frame, deformations for both regions are predicted separately, all Gaussians are concatenated, and jointly rendered via differentiable Gaussian Splatting to obtain RGB images, segmentation masks, and hair orientation maps, enabling joint supervision through photometric, hair-geometric, and physics-based regularization.

Figure 1: DynHair method overview. Hair and upper body are disentangled; hair uses explicit strands with strand-aligned Gaussians, and an LSTM encodes head motion history to predict non-rigid strand deformation.

Canonical Hairstyle Representation. Hair modeling has two stages: static hairstyle reconstruction at the initial timestep, and learning a head-motion-conditioned dynamic deformation network. The canonical hairstyle $\mathcal{H}$ is defined as a set of $N$ polylines originating from a scalp mesh: $\mathcal{H}=\{S_i\}_{i=1}^N$, where each strand $S_i=\{p_{ij}\}_{j=1}^L$ consists of $L=40$ points. Im2Haircut (a PCA-based hair prior with coarse-to-fine encoders) predicts PCA coefficients from multi-view images, decoded into strand geometry, upsampled, filtered with a scalp silhouette mask, and resampled to 40 points per strand to obtain the canonical hairstyle $\mathcal{H}_0$. The static optimization objective is:

$$\mathcal{L}_{\text{static}}=\mathcal{L}_{\text{im2haircut}}+\lambda_{\text{pca}}\mathcal{L}_{\text{pca}}+\lambda_{\text{length}}\mathcal{L}_{\text{length}}+\lambda_{\text{smooth}}\mathcal{L}_{\text{smooth}}+\lambda_{\text{consist}}\mathcal{L}_{\text{consist}}$$

where $\mathcal{L}_{\text{pca}}$ regularizes PCA coefficients to prevent deviation from the prior distribution, $\mathcal{L}_{\text{length}}$ and $\mathcal{L}_{\text{smooth}}$ encourage neighboring strands to have similar lengths and shapes via root-based kNN, and $\mathcal{L}_{\text{consist}}$ enforces that the PCA distribution of back-view strands matches front-view strands, improving geometry in occluded regions. This strategy of starting from a prior and then optimizing ensures canonical hairstyle plausibility.

Motion Conditioning Extraction. Hair dynamics arise from rigid head motion and non-rigid deformations caused by inertia and gravity. At each frame, the global pose $P_t=\{R_t, T_t, S_t\}$ is extracted from the parametric face model, and angular velocity and acceleration are computed using central differences:

$$\boldsymbol{\omega}^{\text{world}}_{t}=\frac{\mathrm{Log}(R_{t+1}R_{t-1}^{\top})}{2\Delta t},\quad \boldsymbol{\alpha}^{\text{world}}_{t}=\frac{\boldsymbol{\omega}^{\text{world}}_{t+1}-\boldsymbol{\omega}^{\text{world}}_{t-1}}{2\Delta t}$$

where $\mathrm{Log}(\cdot)$ maps an SO(3) rotation to $\mathbb{R}^3$. Both are rotated into the head-local frame for pose-invariant conditioning: $\boldsymbol{\omega}_t=R_t^{\top}\boldsymbol{\omega}^{\text{world}}_t$, $\boldsymbol{\alpha}_t=R_t^{\top}\boldsymbol{\alpha}^{\text{world}}_t$. The relative gravity direction is the canonical gravity vector rotated into the head-local frame:

$$\mathbf{g}_t=\mathrm{normalize}(R_t^{\top}[0,-1,0]^{\top})$$

These signals are normalized per-axis and clamped to $[-1,1]$ for training and inference stability. Choosing angular velocity, acceleration, and relative gravity rather than absolute pose is a key design validated by ablation—using absolute pose reduces generalization, removing gravity prevents strands from falling properly, and removing acceleration reduces motion expressiveness. Rotating to the head-local frame ensures pose invariance, enabling cross-subject generalization.

Hair Deformation Network. The deformation network $\mathcal{D}_{\text{hair}}$ predicts per-point non-rigid displacements for the canonical hairstyle, conditioned on the past $T=5$ frames of head motion history. For each frame $\tau$ in the sliding window $\{t-T,\ldots,t\}$, a conditioning vector $\mathbf{c}_\tau=[\boldsymbol{\omega}_\tau, \boldsymbol{\alpha}_\tau, \mathbf{g}_\tau]\in\mathbb{R}^9$ is formed. Each $\mathbf{c}_\tau$ is positionally encoded $\gamma(\mathbf{c}_\tau)$ and fed through an LSTM (with learnable initial hidden states $h_0, c_0$) to produce a temporal motion embedding:

$$\mathbf{z}_t = \mathrm{LSTM}(\{\gamma(\mathbf{c}_\tau)\}_{\tau=t-T}^{t};\;h_0,c_0)$$

Using LSTM rather than an MLP encoder is a key design—ablation shows MLP encoders produce stiffer motion (negative tLPIPS_ex). The LSTM's recurrent structure captures cumulative effects of motion history: the current hair state depends not only on current motion but also on past trajectory (e.g., hair still swinging after rapid rotation stops). The motion embedding $\mathbf{z}_t$ modulates per-point strand features via FiLM (Feature-wise Linear Modulation). A linear layer maps $\mathbf{z}_t$ to scale and shift parameters $(\boldsymbol{\gamma}_{\text{film}}, \boldsymbol{\beta}_{\text{film}})$ that modulate the positional encoding of strand points:

$$\mathrm{FiLM}(p_{ij}) = \gamma(p_{ij})\odot(1+\boldsymbol{\gamma}_{\text{film}})+\boldsymbol{\beta}_{\text{film}}$$

FiLM modulation allows the motion embedding to influence each strand point's features continuously, more flexibly than simple concatenation. Ablation shows removing FiLM increases VER from 26.88 to 40.46. The modulated features are concatenated with a strand position embedding $\gamma(j/L)$ (encoding the normalized position $j/L$ along the strand) and passed through a strand-wise MLP to predict displacements:

$$\Delta p_{ij}=\mathrm{MLP}(\mathrm{FiLM}(p_{ij}),\gamma(j/L))\cdot\rho_j$$

where $\rho_j$ is a learnable root attenuation factor initialized as a linear ramp from small values near the root to 1.0 at the tip—encoding the physical prior that strand roots anchored to the scalp move less than free-moving tips. The final MLP layer is initialized near zero to ensure small initial deformations and stable early training. The deformed hairstyle is $\mathcal{H}_t=\mathcal{H}_0+\Delta\mathcal{H}_t$, then rigidly transformed to world coordinates using the head's global pose.

graph LR
  A[Multi-view Video] --> B[BFM Head Tracking]
  B --> C[Angular Vel + Accel + Relative Gravity]
  C --> D[LSTM Encoder
5-frame Sliding Window]
  D --> E[Motion Embedding z_t]
  E --> F[FiLM Modulation
Per-point Strand Features]
  F --> G[Strand MLP
Predict Displacement]
  G --> H[Root Attenuation
Less at Root/More at Tip]
  H --> I[Deformed Hairstyle H_t]
  I --> J[Strand-aligned Gaussian Rendering]
  style D fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
  style F fill:#fff3cd,stroke:#856404,stroke-width:2px

Figure 2: DynHair strand deformation pipeline. LSTM encodes motion history (blue), FiLM modulates per-point features (yellow), root attenuation encodes physical priors.

Strand-aligned Gaussian Rendering. 3D Gaussian primitives are constrained to lie on strand polylines: each Gaussian mean is placed at the midpoint of a line segment, the primary scale axis is aligned with the segment direction (with magnitude proportional to segment length), and the two cross-sectional scale axes are set to a fixed strand width. The rotation quaternion is computed via parallel transport from a reference axis to the segment direction. Gaussians near strand roots are tapered to reduce visual artifacts at the scalp boundary. This design combines Gaussian rendering quality with strand geometry precision—Gaussian appearance attributes (color, opacity) are learnable, but spatial positions are strictly constrained by strand geometry.

Joint Training and Losses. Training has two stages: (1) static initialization optimizing the canonical hairstyle $\mathcal{H}_0$ using the first frame across all views; (2) dynamic training jointly training the hair deformation network, $\mathcal{H}_0$ (with small learning rate), head deformation networks, and hair appearance. The dynamic objective combines photometric, hair-geometric, physics-based, and color regularization losses:

$$\mathcal{L}_{\text{dynamic}}=\mathcal{L}_{\text{photo}}+\mathcal{L}_{\text{hair}}+\mathcal{L}_{\text{color\_reg}}$$

Photometric losses include RGB L1, SSIM, and VGG perceptual loss. Hair geometric losses include silhouette (IoU), orientation, and penetration losses. The penetration loss prevents strands from penetrating the head mesh—10,000 strand points are randomly sampled, and points inside the head mesh are penalized with squared distance to the nearest triangle:

$$\mathcal{L}_{\text{penetr}}=\frac{1}{|\mathcal{P}_{\text{in}}|}\sum_{p\in\mathcal{P}_{\text{in}}}d^2(p,\mathcal{M})$$

The elastic regularization penalizes stretching and compression of hair segments during deformation. For consecutive strand points $p_j$ and $p_{j+1}$, segment vector lengths in rest and posed configurations are compared:

$$\mathcal{L}_{\text{elastic}}=\frac{1}{NL}\sum_{i=1}^{N}\sum_{j=1}^{L-1}(\|d_{ij}^{\text{posed}}\|-\|d_{ij}^{\text{rest}}\|)^2$$

Ablation shows removing elastic loss causes catastrophic degradation—VER spikes from 26.88 to 283.75, P95 from 0.071 to 1.159, strands severely stretch and lose physical plausibility. Color regularization includes three terms: color gradient loss $\mathcal{L}_{\text{cg}}$ penalizes abrupt color changes between consecutive points along strands, spatial color smoothness loss $\mathcal{L}_{\text{cs}}$ via k=8 nearest neighbor matching encourages nearby Gaussians to share similar colors, and color consistency loss $\mathcal{L}_{\text{cc}}$ penalizes deviation from per-strand mean color.

The model trains for 320,000 iterations on a single A100, with increased VGG perceptual loss weight for the last 80,000 iterations to enhance fine-grained appearance. Using 1024x1024 resolution, 5-frame window history, 40 points per strand, and approximately 11,000 strands.

Experimental Results

The paper evaluates on two multi-view datasets: HHAvatar (4 cameras, ~3 min/scene, 2048x2048, diverse hair motion and expressions) and a newly captured dataset (15 cameras, 4K/72FPS, 22 predefined actions including 10 hair dynamics actions: slow to rapid acceleration, nodding, tilting, rotation, lateral movement, sudden stops). Baselines include Gaussian Head Avatar (GHA), GaussianAvatars (GA), and Maya physics simulator (geometry only, with DynHair's estimated colors driving Gaussians).

Quantitative Comparison. Under self-reenactment, DynHair achieves slightly lower PSNR/SSIM than GHA (21.60 vs 22.33 dB)—expected behavior, as enforcing strand-level geometric consistency prevents arbitrary smoothing of high-frequency structures, introducing small pixel-wise deviations. However, DynHair achieves FID of 30.06, significantly better than GHA (36.25) and GA (45.73), with FID measuring perceptual realism better reflecting hair dynamics learning goals. For temporal stability metric tLPIPS_ex, DynHair achieves 0.0045 (positive means more stable than ground truth), far better than GHA's -0.0127 and GA's -0.0230 (negative means temporal flickering).

Method	PSNR ↑	SSIM ↑	LPIPS ↓	FID ↓	tLPIPS_ex
GA	20.17	0.779	0.254	45.73	-0.0230
GHA	22.33	0.793	0.206	36.25	-0.0127
Maya*	19.39	0.661	0.247	62.64	0.0183
DynHair (Ours)	21.60	0.764	0.201	30.06	0.0045

Table 1: Self-reenactment quantitative comparison (3 subjects average). DynHair achieves best FID and temporal stability.

Physical Realism Evaluation. The paper introduces four motion metrics to evaluate hair dynamics physical plausibility: Velocity Explosion Ratio (VER), Curvature Temporal Smoothness (CTS), Displacement Spike (P95-Disp), and Angular Momentum Drift (AMD). DynHair significantly outperforms the variant without elastic loss (VER 26.88 vs 283.75), proving the effectiveness of physics regularization. While Maya simulation achieves better physics metrics on some measures (CTS 0.00010 vs 0.00022), its IoU (0.789 vs 0.883) and FID are worse, showing hand-tuned parameters cannot match data-driven visual quality.

Variant	IoU_hair ↑	tLPIPS_ex	VER ↓	P95 ↓	CTS ↓
DynHair Full	0.883	0.0024	26.88	0.071	0.00022
w/ absolute pose	0.885	0.0028	36.27	0.071	0.00019
w/o acceleration	0.878	0.0026	29.05	0.077	0.00021
w/o gravity	0.852	0.0058	23.13	0.070	0.00022
w/o elastic loss	0.873	0.0047	283.75	1.159	0.00156
w/o FiLM	0.877	0.0009	40.46	0.104	0.00038

Table 2: Ablation study. Removing elastic loss causes catastrophic degradation (VER from 26.88 to 283.75).

Qualitative Comparison. Under both self-reenactment and cross-reenactment, DynHair produces sharper strand-level details and more temporally coherent strand motion. Baselines produce over-smoothed and flickering hair regions. Maya simulation deviates from true hair motion due to hand-tuned parameters and limited physical model accuracy, and directly attaching learned appearance to simulator-driven strand positions degrades rendering quality. Cross-reenactment experiments show DynHair's explicit strand representation conditioned on relative motion produces geometry-consistent hair animation for novel driving sequences—hair correctly follows gravity and inertia even when driven by different subjects.

Figure 3: Self-reenactment comparison. Left to right: ground truth, Maya simulation, GaussianAvatars, GHA, DynHair. DynHair produces sharper strand-level detail and more temporally coherent motion.

Figure 4: Results on diverse hair motions. DynHair recovers per-strand temporal motion. Ablation without deformation module shows rigid hair motion.

Limitations

First, DynHair models only hair-head interactions; hair-body collisions are handled implicitly through data-driven priors rather than explicitly resolved. This means long hair collisions with shoulders or body are not precisely modeled, potentially causing interpenetration under extreme motion. Explicit collision modeling requires differentiable collision detection and response mechanisms, which are computationally expensive and training-unstable—an important future direction.

Second, the method depends on hair silhouette estimates from pre-trained segmentation models, which can be inaccurate for complex hairstyles or under heavy occlusion. Rendering quality at hair-head boundaries may exhibit artifacts due to the discrete transition between the two Gaussian representations (strand-aligned vs. unstructured). Additionally, reliance on landmark detectors and trackers can lead to incorrect reconstructions when hair occludes the face, affecting pose estimation and consequently motion conditioning extraction.

Conclusion and Future Work

DynHair presents a dynamic head avatar framework learned end-to-end from multi-view video, representing hair as strand-aligned 3D Gaussians whose deformations are predicted by a temporal motion model conditioned on head angular velocity, acceleration, and relative gravity. By combining an LSTM head pose encoder, FiLM modulation, and physics-based regularization (elastic constraints, root attenuation, penetration penalties), the approach produces temporally coherent hair dynamics with strand-level geometric fidelity. Experiments demonstrate improved visual quality (FID) and motion plausibility (tLPIPS_ex) compared to SOTA head avatar methods.

Explicitly modeling hair structure and dynamics provides a foundation for future large-scale dynamic hair learning, where richer datasets and explicit collision modeling can be naturally integrated. The disentangled head-hair modeling also supports direct hair editing applications—adjusting Gaussian brightness or trimming hairstyle length while preserving realistic hair motion dynamics.

Golden Quotes

What current head avatars lost is not rendering quality, but the ability of hair to follow physical laws—DynHair aims to relearn this from data.

Physics simulation needs hand-tuned parameters; data-driven learning needs physics regularization—DynHair chooses the latter, encoding physical priors into the network through elastic constraints and root attenuation.