World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation

1. Overview

This paper tackles the fundamental challenge in sim-to-real transfer: the same observation-action pair can produce different state transitions under different hidden variables. Existing learned dynamics models assume these hidden variables are recoverable from observation history, but this assumption fails whenever the history is uninformative — such as a sudden contact event with no prior warning.

World Translation exploits a complementary strength of simulators and learned dynamics. Simulators are deterministic but physically imperfect; learned models are accurate but underdetermined under partial observability. The authors propose backward dynamics extraction: encoding the unobservable dynamics information backward from an already-observed transition, then translating this feature across simulation and reality as an unpaired domain-translation problem that preserves dynamics content while transferring domain style using CycleGAN.

Experiments span humanoid (G1), quadruped (Go2), and manipulator (R5) platforms, with the largest gains when observation history is insufficient to recover hidden variables, and real-robot deployment on Go2 confirms improved policy transfer.

2. Problem Formulation

2.1 System Dynamics and Hidden Variables

Full state $\mathbf{s}_{t}\in\mathcal{S}$, observation $\mathbf{o}_{t}\in\mathcal{O}$, action $\mathbf{a}_{t}\in\mathcal{A}$. Dynamics are deterministic in state space: $\mathbf{s}_{t+1}=g(\mathbf{s}_{t},\mathbf{a}_{t})$. Due to the information gap between $\mathbf{s}_{t}$ and $\mathbf{o}_{t}$, identical $(\mathbf{o}_{t},\mathbf{a}_{t})$ can produce different outcomes. Introducing hidden variables $\mathbf{h}_{t}\in\mathcal{H}$ restores determinism:

$$\mathbf{o}_{t+1}=f(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{h}_{t})$$

No assumptions are made about $p(\mathbf{h}_{t+1}|\mathbf{h}_{t},\mathbf{o}_{t},\mathbf{a}_{t})$; only that $\mathbf{h}_{t}$'s effect is identifiable from the observed transition.

2.2 Backward Dynamics Extraction

A dynamics feature $\mathbf{z}_{t}=\omega(\mathbf{h}_{t})$ represents the hidden variable in latent space:

$$(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{o}_{t+1}) \xrightarrow{\;\text{encode}\;} \mathbf{z}_{t}=\omega(\mathbf{h}_{t})$$

$$(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{z}_{t}) \xrightarrow{\;\text{decode}\;} \mathbf{o}_{t+1}$$

"Backward" because it uses future information $\mathbf{o}_{t+1}$ to infer $\mathbf{z}_{t}$ at time $t$. Extraction alone cannot predict — an alternative source (simulator) is needed, from which features are extracted and translated to the target system.

2.3 Unpaired Domain Translation

Across domains, the same $\mathbf{h}_{t}$ produces different transitions, so $\mathbf{z}_{t}$ depends on both $\mathbf{h}_{t}$ and domain characteristics $\mathbf{c}$:

$$\mathbf{o}_{t+1}=f(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{h}_{t},\mathbf{c})$$

$$G_{A\rightarrow B}(\omega(\mathbf{h}_{t},\mathbf{c}^{A}))\approx\omega(\mathbf{h}_{t},\mathbf{c}^{B})$$

This is unpaired: transitions collected across domains cannot guarantee identical underlying $\mathbf{h}_{t}$.

3. Method

World Translation instantiates backward extraction with a VAE (plus auxiliary regularization) and unpaired translation with CycleGAN.

3.1 Backward Dynamics Extraction (VAE)

Encoder $E_{\phi}$ maps transition to latent distribution: $(\boldsymbol{\mu},\log\boldsymbol{\sigma}^{2})=E_{\phi}(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{o}_{t+1})$, sampled $\mathbf{z}_{t}\sim\mathcal{N}(\boldsymbol{\mu},\text{diag}(\boldsymbol{\sigma}^{2}))$. Decoder: $\hat{\mathbf{o}}_{t+1}=D_{\theta}(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{z}_{t})$.

Challenge 1: Direct encoding of $\mathbf{o}_{t+1}$. A blind decoder $D_{\psi}$ tries predicting $\mathbf{o}_{t+1}$ from $\mathbf{z}_{t}$ alone, creating an adversarial objective:

$$\mathcal{L}_{\text{blind}}=\lambda_{b}\exp\left(-\frac{\|\mathbf{o}_{t+1}-\tilde{\mathbf{o}}_{t+1}\|^{2}}{\sigma_{b}^{2}}\right)$$

Challenge 2: Information leakage in $(\mathbf{o}_{t},\mathbf{a}_{t})$. A domain classifier $C_{\xi}$ forces $\mathbf{z}_{t}$ to encode domain info:

$$\mathcal{L}_{\text{cls}}=\lambda_{c}\cdot\text{BCE}(C_{\xi}(\mathbf{z}_{t}),d)$$

FiLM conditions each decoder layer on $\mathbf{z}_{t}$:

$$D_{\theta}(\mathbf{o}_{t},\mathbf{a}_{t},\mathbf{z}_{t})=\boldsymbol{\gamma}(\mathbf{z}_{t})\odot f(\mathbf{o}_{t},\mathbf{a}_{t})+\boldsymbol{\beta}(\mathbf{z}_{t})$$

Full objective:

$$\mathcal{L}_{\text{VAE}}=\|\mathbf{o}_{t+1}-\hat{\mathbf{o}}_{t+1}\|^{2}+\beta\cdot\text{KL}(q_{\phi}\|p)+\lambda_{b}\exp\left(-\frac{\|\mathbf{o}_{t+1}-\tilde{\mathbf{o}}_{t+1}\|^{2}}{\sigma_{b}^{2}}\right)+\lambda_{c}\cdot\text{BCE}(C_{\xi}(\mathbf{z}_{t}),d)$$

3.2 Unpaired Domain Translation (CycleGAN)

Generators $G_{S\to R}$, $G_{R\to S}$ (residual MLPs) and discriminators $D_{S}$, $D_{R}$ trained with adversarial loss, cycle consistency ($\mathbf{z}^{S}\approx G_{R\to S}(G_{S\to R}(\mathbf{z}^{S}))$), and optional identity loss.

3.3 Training and Deployment

VAE and CycleGAN are jointly trained with shared encoder-decoder across domains. At deployment: encode a simulator transition, translate $G_{S\to R}$, decode to obtain aligned observation, and overwrite the simulator state (not just transform observations).

4. Architecture

flowchart LR
    subgraph Sim Domain S
        OS[Observation o_t] --> AS[Action a_t]
        AS --> SS[Sim Transition o_t+1^S]
    end
    subgraph Backward Extraction VAE
        SS --> ENC[Encoder E_φ]
        OS --> ENC
        AS --> ENC
        ENC --> ZS[Dynamics Feature z_t^S]
        ZS --> DEC[Decoder D_θ]
        DEC --> OS2[Aligned Obs o_t+1^S→R]
    end
    subgraph Unpaired Translation CycleGAN
        ZS --> G1[Generator G_S→R]
        G1 --> ZR[Translated Feature z_t^R]
        ZR --> G2[Generator G_R→S]
        G2 --> ZS2[Cycle Recon z_t^S]
    end
    ZR --> DEC
    OS2 --> POLICY[Policy Training]
    style ENC fill:#4f9eff,color:#fff
    style G1 fill:#ff6b6b,color:#fff
    style DEC fill:#51cf66,color:#fff

5. Experimental Results

5.1 Dynamics Modeling Accuracy

Source and target domains created with different physics parameters (stiffness, damping, mass, friction), split into low-$\mathbf{h}_{t}$ and high-$\mathbf{h}_{t}$ conditions.

In high-$\mathbf{h}_{t}$ tasks, history-based methods (RSSM) remain substantially above World Translation (Go2 Payload: 0.298 vs 0.148; R5 Force: 0.515 vs 0.401), showing the gap reflects a limitation of history-based inference, not model capacity.

5.2 Multi-Step Rollout

Autoregressive rollouts on Go2 high-$\mathbf{h}_{t}$ task. Methods without $\mathbf{h}_{t}$ inference diverge rapidly; RSSM doesn't diverge but MSE stays above RawSim; World Translation remains stable through horizon 30 with only ~2% divergence at horizon 50.

5.3 Hidden Variable Information

Lightweight decoders predict ground-truth $\mathbf{h}_{t}$ (normalized force) from different representations, measured by $R^{2}$. $(\mathbf{o}_{t},\mathbf{a}_{t})$ alone explains 66–73% variance; adding $\mathbf{z}_{t}$ matches full transition (68–81%). After translation, predictive power is largely preserved (67–78%).

5.4 Domain Characteristic Information

Without auxiliary components, $\mathbf{z}_{t}$ achieves only 64–67% domain classification; translated $\mathbf{z}^{S\to R}$ only 43%. With auxiliary components: $\mathbf{z}_{t}$ reaches 94%, $\mathbf{z}^{S\to R}$ reaches 91% — the translated feature successfully overrides domain signal in inputs.

5.5 Ablation Study

Observation-level translation collapses immediately — discriminators trivially distinguish domains from $(\mathbf{o}_{t},\mathbf{a}_{t})$. Without cycle consistency, cycle error increases 8× (0.034→0.273).

5.6 Real-Robot Downstream Policy

Deployed on Go2 quadruped. Policies trained with World Translation outperform RawSim and DirectPred in real-world command tracking, validating that improved dynamics modeling directly translates to better sim-to-real policy transfer.

6. Discussion and Future Directions

6.1 Design Tradeoffs

Unpaired vs direct supervision: Indirect optimization may sacrifice accuracy, but paired sim-real transitions with matching hidden variables are generally impossible to obtain.

Implicit representation: $\mathbf{z}_{t}$ captures unmodeled dynamics beyond system identification, at the cost of interpretability.

6.2 Practical Challenges

Tight coupling between $\mathbf{h}_{t}$ and $\mathbf{c}$ (R5 force-stiffness interaction). State overwriting may cause physics inconsistency. Joint training involves competing objectives but transfers across platforms without tuning.

6.3 Future Directions

• Multi-step training: Train on trajectory segments with autoregressive objectives.
• Online adaptation: Update translation model during deployment; unpaired formulation naturally supports this.
• Visual observations: Extend to image-based observations, requiring disentanglement of visual appearance from dynamics content.

7. Conclusion

World Translation reformulates dynamics alignment as unpaired domain translation over learned features. Backward dynamics extraction captures hidden variable effects from observed transitions without assuming history sufficiency; cycle-consistent translation maps these features across domains while preserving dynamics information. Experiments on humanoid, quadruped, and manipulator platforms demonstrate improved dynamics modeling with the largest gains when history is insufficient, and Go2 real-robot deployment confirms improved command tracking.