FloAff-Kitchen: Bridging Navigation and Manipulation via Canonical and Progressive Floor Affordance Learning

1. Overview: Bridging Navigation and Manipulation via Canonical and Progressive Floor Affordance Learning

FloAff-Kitchen, proposed by Ping Zhong, Manling Teng, Tao Wu, Bolei Chen, Jiazhi Xia, and Jianxin Wang at Central South University (July 2026), is a unified framework for predicting FloAff (Floor Affordance) from egocentric multimodal perception. Mobile manipulation (MoMa) requires robots to determine base placements that maximize downstream manipulation success rather than merely ensuring navigation feasibility. FloAff prediction is a target-conditioned local spatial reasoning problem, but existing methods suffer from representation ambiguity—irrelevant spatial context and arbitrary object orientations cause inconsistent point cloud representations, and shared vs. task-specific knowledge is entangled across heterogeneous manipulation skills. This work proposes CFAR (Canonical Floor Affordance Representation) to learn canonical interaction geometry, preserving affordance-relevant local structure while eliminating nuisance spatial variations; PFAL (Progressive Floor Affordance Learning) to learn transferable FloAff priors from a foundation task and progressively adapt to heterogeneous downstream skills. It also establishes the first cross-scene, multi-view FloAff-Kitchen benchmark. Experiments consistently outperform strong baselines across three settings.

2. Core Problem: Representation Ambiguity and Transfer Dilemma in FloAff Prediction

Mobile manipulation requires robots to determine base placement regions that support successful manipulation. FloAff prediction estimates such regions from egocentric observations, a fundamental capability bridging navigation and manipulation. Existing learning-based methods directly predict FloAff from complete scene observations, with two limitations: (1) FloAff is primarily determined by local interaction geometry around the manipulation target, while distant structures introduce irrelevant spatial context; (2) affordance-equivalent scenes under different target orientations produce inconsistent point cloud representations, causing representation ambiguity. The model must simultaneously identify affordance-relevant geometry and learn orientation invariance. Moreover, heterogeneous manipulation skills (hinged doors, drawers, pick-and-place) have distinct interaction dynamics; multi-task learning entangles transferable spatial reasoning with task-specific patterns, limiting cross-skill generalization.

3. Method: CFAR and PFAL

3.1 Canonical Floor Affordance Representation (CFAR)

CFAR is a target-conditioned scene representation preserving affordance-relevant local geometry while invariant to irrelevant spatial context and target orientation. Instead of directly learning $f_\theta(\mathcal{P}, \mathcal{S})$, it first applies a canonical transformation:

$$\hat{\mathcal{A}} = f_\theta\!\left(\mathcal{T}(\mathcal{P}, \mathcal{S})\right) \tag{3}$$

where $\mathcal{T}(\cdot)$ is the canonical transformation, $\mathcal{P}$ the point cloud, $\mathcal{S}$ the target specification. CFAR satisfies two properties: Affordance Relevance (preserve FloAff-relevant geometry, remove irrelevant context) and Orientation Consistency (map affordance-equivalent scenes to consistent representations).

Eliminating Irrelevant Spatial Context: FloAff primarily depends on local interaction geometry around the target, so a target-centered spherical point set is constructed:

$$\mathcal{P}_{\text{local}} = \left\{\mathbf{p}_i \in \mathcal{P} \,\middle|\, \|\mathbf{p}_i - \mathbf{t}\|_2 \leq r\right\} \tag{4}$$

where $\mathbf{t}=(x_t, y_t, z_t)$ and $r$ is the crop radius. Eliminating Orientation Ambiguity: Although local geometry is extracted, the same affordance pattern may correspond to different point clouds due to target orientation. CFAR uses AxisNet to transform arbitrary-oriented interaction geometry to a canonical orientation, mapping geometrically equivalent interactions to consistent representations.

3.2 Progressive Floor Affordance Learning (PFAL)

Heterogeneous MoMa tasks share common spatial requirements for base placement (reachability, collision-free execution, visibility, manipulability)—shared FloAff priors—while different skills introduce task-specific constraints (hinge directions, sliding directions). Hypothesis 1: across heterogeneous skills, FloAff prediction benefits from a transferable latent representation capturing task-independent spatial reasoning, while task-specific knowledge can be progressively learned via representation adaptation. PFAL conceptually decomposes the latent as:

$$\mathbf{z} = \Phi(\mathbf{z}_s, \mathbf{z}_t), \quad \theta_{\text{pnp}} \to \theta_{\text{door,drawer}}$$

where $\mathbf{z}_s$ encodes shared priors and $\mathbf{z}_t$ task-specific constraints. Through a curriculum of increasing FloAff complexity, it first establishes transferable spatial reasoning on pick-and-place (foundation task), then adapts to hinge- and sliding-dependent patterns. Shared Prior Learning on $\mathcal{D}_{\text{pnp}}$:

$$\theta_{\text{pnp}} = \arg\min_\theta \frac{1}{|\mathcal{D}_{\text{pnp}}|} \sum_{(\hat{\mathcal{P}}, \mathcal{S}, \mathcal{A}) \in \mathcal{D}_{\text{pnp}}} \mathcal{L}_{\text{aff}}\!\left(f_\theta(\hat{\mathcal{P}}, \mathcal{S}), \mathcal{A}\right) \tag{6}$$

Progressive Adaptation: initialized from $\theta_{\text{pnp}}$, fine-tuned on door/drawer tasks, separating transferable spatial reasoning from task-specific patterns. The adaptation objective is:

$$\theta_{\text{door,drawer}} = \arg\min_\theta \frac{1}{|\mathcal{D}_{\text{task}}|} \sum \mathcal{L}_{\text{aff}}\!\left(f_\theta(\hat{\mathcal{P}}, \mathcal{S}), \mathcal{A}\right), \quad \theta_0 = \theta_{\text{pnp}}$$

4. FloAff-Kitchen Benchmark

Establishes the first cross-scene, multi-view FloAff-Kitchen benchmark covering diverse manipulation skills, scene layouts, furniture styles, and viewpoints. Data collection has three phases: scene generation, candidate base pose generation, collision-aware filtering, rollout-based affordance annotation, and multi-view observation collection.

5. Experiments

5.1 FloAff Prediction Comparison

Method	RMSE↓	logMSE↓	PCC↑	SIM↑
PointNet++	0.164	0.0142	0.565	0.589
VoteNet	0.167	0.0143	0.543	0.570
H3DNet	0.174	0.0156	0.503	0.522
NavAff	0.147	0.0115	0.680	0.696
Ours (MoMa-Kitchen)	0.073	0.0028	0.796	0.794
Ours (FloAff-Kitchen Styles)	0.029	0.0004	0.863	0.862
Ours (FloAff-Kitchen Layouts)	0.035	0.0006	0.738	0.721

On MoMa-Kitchen, RMSE drops from 0.147 to 0.073, PCC rises from 0.680 to 0.796. On FloAff-Kitchen Styles PCC reaches 0.863, Layouts 0.738—Styles changes only appearance, Layouts alters interaction geometry (harder), yet the method remains robust.

5.2 Downstream MoMa Performance

MoMa success rates on FloAff-Kitchen: $SR_{\text{all}}$ significantly exceeds NavAff baseline on both Styles and Layouts. NavAff achieves $SR_{\text{all}}=0.37$ on Styles and $0.225$ on Layouts.

Benchmark	NavAff SR_all	Ours SR_all	Improvement
FloAff-Kitchen Styles	0.37	higher	significant
FloAff-Kitchen Layouts	0.225	higher	significant

6. Limitations

• Crop radius $r$ sensitivity: CFAR's local spherical crop radius is preset; too small loses interaction geometry, too large introduces irrelevant context.
• AxisNet dependency for canonical transform: Orientation normalization accuracy depends on AxisNet; generalization to unseen object shapes is limited.
• Fixed curriculum order: PFAL's pnp→door/drawer curriculum is preset; optimal order for other heterogeneous skills remains to explore.
• Simulation-only evaluation: FloAff-Kitchen is a simulation benchmark; real-world MoMa transfer capability is unverified.
• Limited task types: Currently covers doors/drawers/pick-and-place; adaptation to more skills (pouring, folding) awaits extension.

7. Conclusion

This paper proposes a unified FloAff prediction framework: CFAR learns canonical interaction geometry via target-conditioned representation transformation, eliminating irrelevant context and orientation ambiguity; PFAL progressively acquires transferable FloAff priors from a foundation task and adapts to heterogeneous skills. The first cross-scene, multi-view FloAff-Kitchen benchmark is established. On MoMa-Kitchen and FloAff-Kitchen, both FloAff prediction and downstream MoMa performance consistently improve, demonstrating effectiveness and generalization. The core insight: affordance prediction accuracy depends on representation canonicity and transfer progression—first "straighten" then "crop" local interaction geometry, then "solidify" transferable spatial reasoning before "adapting"—ambiguity and entanglement dissolve naturally.

flowchart TD
    A["Egocentric multimodal obs P,S"] --> B["CFAR canonical transform T"]
    B --> C["Eliminate irrelevant context: spherical crop P_local"]
    B --> D["Eliminate orientation ambiguity: AxisNet normalization"]
    C --> E["Canonical representation P̂"]
    D --> E
    E --> F["PFAL progressive learning"]
    F --> G["Stage 1: pick-and-place foundation
learn shared FloAff priors θ_pnp"]
    G --> H["Stage 2: adapt to doors/drawers
task-specific constraints"]
    H --> I["FloAff prediction Â"]
    I --> J["Base placement decision"]
    J --> K["Navigation + manipulation execution"]