Emergent Compositional Skills in Mixture-of-Experts VLAs

Background and Motivation

Vision-Language-Action models (VLAs) have shown excellent generalization across broad robotic manipulation tasks, with models like RT-2, OpenVLA, Octo, and $\pi_0$. However, existing VLAs are typically trained and deployed as monolithic policies, making it difficult to identify reusable skills, compose behaviors hierarchically, or adapt parts of the policy without modifying the entire model. In contrast, many robotic tasks naturally decompose into reusable behavioral modes—such as reaching, grasping, and placing. A modular policy that explicitly learns such skill structures could improve interpretability, compositionality, and robustness while retaining the broad task competence of VLAs. The core question this paper investigates is: can this modular structure emerge directly from data, without relying on pre-specified decomposition or manually constructed skill libraries?

Notably, MoE has been widely applied in NLP (e.g., Mixtral, GShard), but migrating it to robot VLAs presents unique challenges. NLP MoE typically routes per token—each token independently selects experts, as different tokens may need different processing. However, robot actions have temporal coherence—multiple timesteps of a grasping motion should be handled by the same expert to maintain behavioral consistency. This paper's "whole forward pass routing" is the key design addressing this difference: a single routing decision spans the entire action chunk generation, ensuring skill coherence. Additionally, NLP experts are typically trained from scratch, while robot VLAs already have powerful pretrained backbones; this paper uses LoRA deltas to add experts on top of the backbone rather than training from scratch, preserving pretrained knowledge while achieving specialization.

Existing hierarchical VLA systems typically impose a fixed planner-controller split, requiring predefined skill libraries or sub-task labels. This paper proposes a different approach: constructing a Mixture-of-Experts (MoE) architecture on the VLA's action expert, learning compositional policies end-to-end from expert demonstrations without imposing any task decomposition or hierarchical priors. The router selects experts based on current observations and language instructions, implicitly acting as a high-level controller, while experts specialize into lower-level behavioral modes, forming an emergent hierarchy. The key finding is that even the simplest MoE action expert architecture produces distinguishable, interpretable, cross-task reusable expert skills while matching the task performance of monolithic baselines.

Core Method: LoRA Mixture-of-Experts Architecture

MoE Architecture Design

The method builds on two pretrained VLA backbones—$\pi_0$ and SmolVLA—both using flow matching for action generation. Each backbone consists of a vision-language module (encoding camera views and language into context tokens) and an action expert (transformer decoder processing proprioceptive state tokens and noised action chunks). The MoE replaces only the FFN sublayer of each action expert layer, with self-attention shared across all experts:

$$\text{FFN}_\ell(x) = \text{FFN}_\ell^{\text{base}}(x) + \sum_{e \in \mathcal{R}} w_e \left(\text{FFN}_\ell^{(e)}(x) - \text{FFN}_\ell^{\text{base}}(x)\right)$$

where $\mathcal{R}$ is the routed expert set (only top-$k$ selected experts have non-zero weights $w_e$), and each expert differs from the base only through rank-$r=16$ LoRA deltas: $$\Delta W = \frac{\alpha}{r} BA$$ In the delta matrix,. Zero-initialized $B$ matrices ensure the policy exactly reproduces the pretrained backbone at training step zero, so specialization emerges around a strong prior rather than from scratch. This design choice is crucial—it ensures MoE training does not degrade the backbone's learned general manipulation capabilities, but adds composable skill variants on top of it.

From technical details, the LoRA delta design involves several key considerations. In the delta matrix $\Delta W = (lpha/r) BA$, $A \in \mathbb{R}^{r imes d}$ and $B \in \mathbb{R}^{d imes r}$ are low-rank decomposition matrices, $d$ is the FFN hidden dimension, and $r=16$ is the rank. The scaling factor $lpha/r$ controls the delta's strength relative to the base. Zero-initializing $B$ means $\Delta W = 0$ at training start, i.e., $ ext{FFN}^{(e)}(x) = ext{FFN}^{ ext{base}}(x)$, with all experts degenerating to the base—this ensures training starts from the pretrained backbone's optimum, avoiding randomly initialized experts disrupting existing capabilities. As training proceeds, $A$ and $B$ gradually learn deltas that specialize each expert. Compared to training complete FFNs as experts, LoRA deltas have only $2rd$ parameters (far less than $d^2$), significantly reducing multi-expert parameter overhead, while the low-rank constraint serves as regularization against overfitting.

Whole Forward Pass Routing

Routing is performed once per policy forward pass and shared across all $L$ action expert layers, unlike standard MoE transformers that route per token per layer. Sharing a single (indices, weights) pair makes each expert a coherent end-to-end behavior (a "skill") rather than $L$ independent layer-wise routing decisions. The MLP router $g_\phi$ receives a context vector encoding the agent's visual, linguistic, and proprioceptive state:

$$c = \left[\text{MeanPool}\left(\text{VLM}(I, \ell, s)\right) \| W_s s\right]$$

where $I, \ell, s$ are camera observations, instruction, and proprioceptive state, with $W_s$ shared with the action expert. The router emits logits over $E$ experts, from which top-$k$ are selected and their softmax weights renormalized, applied uniformly to every MoE FFN. At inference, the router fires once per action chunk. This "route once, execute throughout" design ensures skill coherence—a grasping expert remains active throughout the entire grasping motion, rather than making independent decisions at each layer that could produce incoherent behavior.

The router's top-$k$ selection mechanism also deserves attention. Given $E$ expert logits $z_1, \ldots, z_E$, standard softmax assigns non-zero weights to all experts, but in MoE only top-$k$ experts are activated to control computation. Let the selected top-$k$ set be $\mathcal{R}$, then renormalized weights are $w_e = ext{softmax}(z_e) / \sum_{e' \in \mathcal{R}} ext{softmax}(z_{e'})$ ($e \in \mathcal{R}$), $w_e = 0$ ($e otin \mathcal{R}$).

The renormalized weight computation can be formalized as:

$$w_e = \frac{\exp(z_e)}{\sum_{e' \in \mathcal{R}} \exp(z_{e'})}, \quad e \in \mathcal{R}; \quad w_e = 0, \quad e \notin \mathcal{R}$$

This sparse activation means each forward pass computes only $k$ expert FFNs, reducing complexity from $O(Ed^2)$ to $O(kd^2)$. More importantly, sharing routing decisions across all $L$ layers further reduces routing overhead from $O(LE)$ to $O(E + Lk)$, making the router a lightweight high-level decision module.

flowchart TD
    I["Camera obs I"] --> VLM["Vision-Language Module VLM"]
    L["Language instr ℓ"] --> VLM
    S["Proprioception s"] --> VLM
    VLM --> MP["MeanPool context"]
    S --> WS["Projection W_s·s"]
    MP --> C["Context vector c"]
    WS --> C
    C --> R["MLP Router g_φ"]
    R -->|top-k select| E1["Expert A: Place"]
    R -->|top-k select| E2["Expert B: Release retract"]
    R -->|top-k select"| E3["Expert C: Approach grasp"]
    E1 --> FFN["FFN sublayer
base + LoRA delta"]
    E2 --> FFN
    E3 --> FFN
    FFN --> AC["Action chunk â_{t:t+H}
K-step flow matching"]

Flow matching is the core action generation mechanism of $\pi_0$ and SmolVLA; understanding it is crucial for grasping this method. Flow matching learns a velocity field that transforms a noise distribution into an action distribution: given a noised action chunk $a_ au$ and flow-matching timestep $ au$, the action expert predicts velocity $v_ heta(a_ au, au, c)$, generating the final action chunk $\hat{a}_{t:t+H}$ through $K$ denoising steps.

The flow matching velocity prediction and action generation process can be formalized as:

$$\hat{a}_{t:t+H} = a_K = a_0 + \frac{1}{K} \sum_{j=1}^{K} v_\theta(a_{j-1}, \tau_j, c), \quad a_0 \sim \mathcal{N}(0, I)$$

The MoE replaces the FFN sublayer in the action expert, so different experts essentially learn different velocity field corrections—Expert A's velocity field tends to guide actions toward placement trajectories, Expert C's toward approach-grasp trajectories. Embedding skill specialization in flow-matching velocity fields is smoother than directly outputting discrete actions, as flow matching's iterative denoising naturally provides temporal action coherence.

Training Objective

The full objective combines the backbone's flow-matching behavior cloning loss with a load-balancing auxiliary term:

$$\mathcal{L} = \mathcal{L}_{\text{FM}} + \lambda_{\text{LB}} \mathcal{L}_{\text{LB}}$$

where $\mathcal{L}_{\text{FM}}$ is the backbone's flow-matching velocity prediction loss on noised action chunks, and $\mathcal{L}_{\text{LB}}$ is the standard load-balancing term (Fedus et al., 2022) that discourages routing collapse to a single expert. $\lambda_{\text{LB}}$ is the load-balancing weight.

The standard form of the load-balancing loss is:

$$\mathcal{L}_{\text{LB}} = E \sum_{e=1}^{E} f_e \cdot P_e, \quad f_e = \frac{1}{N}\sum_{i=1}^{N} \mathbb{1}[e \in \mathcal{R}_i], \quad P_e = \frac{1}{N}\sum_{i=1}^{N} \frac{\exp(z_e^{(i)})}{\sum_{e'} \exp(z_{e'}^{(i)})}$$

where $f_e$ is the batch frequency of expert $e$ being selected, $P_e$ is the average routing probability, and $E$ is the total number of experts. This loss encourages uniform usage of all experts, preventing the router from concentrating all inputs on a few experts. When an expert's $f_e$ and $P_e$ are both high, the product $f_e P_e$ increases the loss, driving the router to disperse selections.

The load-balancing mechanism is crucial—without it, the router might route all inputs to a single expert, leaving other experts untrained and degraded. Experiments observe that the model autonomously discards capacity beyond what the task distribution demands: 6 out of 16 experts are largely unused, demonstrating that the MoE architecture adaptively allocates capacity to behavioral modes that genuinely need diversification.

Experimental Results

Qualitative Analysis of Expert Skills

After training on the LIBERO-10 benchmark, the router selects three representative experts, each corresponding to qualitatively distinct low-level behaviors: Expert A handles the final transport phase of manipulation—activating after an object is grasped to place it at its final position; Expert B releases an item and retracts the arm upward, typically triggered after placement; Expert C approaches and grasps thin-handled objects (e.g., pot handles, cup handles), responsible for initial approach and grasping. These expert behaviors emerge through self-supervision without any predefined skill labels or sub-task annotations.

Figure 1: Expert A places a grasped object at its final position

Figure 2: Expert B releases an item and retracts upward

Figure 3: Expert C approaches and grasps thin-handled objects

Reusable vs Task-Specific Skills

Examining each active expert's selection frequency across LIBERO-10 reveals two clearly different roles: Reusable experts (e.g., Experts A and B) each fire at 35-45% on five different tasks and near-zero on the rest. Both cover nearly the same task set (T0, T1, T4, T6, T7), and inspecting individual rollouts shows the router alternates between them within a single trajectory. Task-specific experts (e.g., Experts C and D) concentrate on single tasks—Expert C at 66% on T5, Expert D at 66% on T2. These experts plausibly absorb idiosyncratic behaviors that only one or two tasks demand, keeping reusable experts crisp by sparing them edge cases.

Skill Composition for Long-Horizon Tasks

Trajectory analysis shows the trained MoE policy exhibits clear compositions of primitive skills into sequences on LIBERO-10 tasks T2, T4, T5. For instance, on task T5, Expert C is first used to grasp the book, then Expert A places it at the final position; on task T2, Expert C grasps the stove dial, then Expert A moves the arm onto the handle. Task T4 shows a different phenomenon: when the policy fails, it manifests as repeated application of known primitives rather than degenerate behavior—e.g., repeatedly executing grasp-release cycles attempting recovery. This "interpretable failure" property is a significant advantage of modular policies. The router acts as a high-level sequencer, solving long-horizon tasks by stitching the same primitives in different orders, such as the "grasp-then-place" pattern reused across moka pot, mug, and book tasks.

Figure 4: Example trajectories labeled by top expert used at each step, showing skill composition

Manual Routing Experiments

To verify that experts truly cause behavioral differences (rather than merely due to router placement), the authors conducted manual routing experiments: forcing specific expert selection at inference. Results show manual routing achieved better skill stitching and suggest out-of-distribution generalization—forcing a specific expert in new contexts produces that expert's characteristic behavior, proving experts learned causal behaviors rather than incidental correlations.

LIBERO Performance Comparison

Both MoE and baseline are initialized from the same pretrained checkpoint and finetuned for 20K steps with identical optimization, differing only in whether the action expert FFN is replaced by MoE. The MoE matches the dense baseline's task performance while learning meaningfully specialized experts, demonstrating that a simple MoE architecture drives skill specialization at no cost to task competence.

From a broader perspective, this work represents the trend of robot learning shifting from "end-to-end black boxes" to "emergent modularization." Traditional monolithic VLAs are powerful but uninterpretable—it's impossible to know how the model internally decomposes tasks or which parameters are responsible for which behaviors. The MoE architecture decomposes policies into identifiable expert modules through routing decisions, where each expert's behavior can be individually inspected through forced routing experiments, giving the policy a degree of "interpretability." More importantly, this modularity is emergent rather than designed—the model discovers natural task decomposition from data through imitation learning plus load balancing, suggesting that manipulation tasks may possess inherent compositional structure. For long-horizon planning, emergent modular policies naturally support skill reuse and recombination—new tasks may only need new sequences of existing skills rather than entirely new learning. This finding has profound implications for future robot learning system design: perhaps explicit skill libraries are unnecessary, and appropriate architectural induction can let models autonomously discover and compose skills.

Limitations and Future Work

Limitation 1: Fixed expert count with partial unused capacity. 6 out of 16 experts are largely unused, indicating wasted capacity in fixed allocation. Future work could explore dynamic expert counts or expert generation/pruning mechanisms to adaptively adjust capacity based on task distribution complexity.

Limitation 2: Simulation-only validation. Experiments are conducted on the LIBERO-10 simulation benchmark without real robot validation of emergent skill generalization. Real-world perceptual noise and dynamics complexity may affect routing decisions and expert behavioral stability. Additionally, expert semantic labels are assigned post-hoc; whether router-learned skill boundaries align with human intuition requires more systematic quantitative verification.

Summary and Insights

This paper demonstrates that VLAs trained with an MoE action expert can decompose manipulation tasks into a small set of modular skills reused across tasks without predefined hierarchy, skill libraries, or sub-task labels. Key technical contributions include: LoRA delta experts $\Delta W = (\alpha/r)BA$ ($r=16$, $B$ zero-initialized) building specialization around the pretrained backbone's strong prior; whole forward pass routing $c = [\text{MeanPool}(\text{VLM}(I,\ell,s)) \| W_s s]$ making each expert a coherent end-to-end skill rather than independent per-layer decisions; load-balancing loss $\mathcal{L} = \mathcal{L}_{\text{FM}} + \lambda_{\text{LB}}\mathcal{L}_{\text{LB}}$ preventing routing collapse while allowing autonomous discarding of redundant capacity. On both $\pi_0$ and SmolVLA backbones, MoE matches dense baseline performance while emergently producing distinguishable expert skills—placement, release retract, approach grasp—with reusable experts shared across 5 tasks at 35-45% frequency and task-specific experts concentrated at 66% on single tasks. The router as a high-level sequencer stitches the same primitives in different orders to solve long-horizon tasks, with even failures manifesting as repeated known skills rather than degenerate behavior. This work takes an important step toward modular, interpretable robot policies emerging from data, with implications for long-horizon planning, skill transfer, and policy interpretability.