FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
Existing occupancy prediction relies on large-scale voxel annotations and generalizes poorly across environments, collapsing to near-zero per-class scores on unseen scenes. FreeOcc is the first training-free framework for open-vocabulary occupancy prediction: a SLAM backbone supplies poses and sparse geometry, geometrically anchored Gaussian updates build a dense 3DGS map (ray-aligned anisotropic initialization with s_parallel = gamma * s_perp and Gaussian centers fixed to SLAM points), an off-the-shelf vision-language model attaches open-vocabulary semantics to Gaussian primitives, and a probabilistic Gaussian-to-occupancy projection yields a voxel field queryable online, with no learnable parameter serving the occupancy task. On EmbodiedOcc-ScanNet it reaches 31.29/13.86 monocular and 34.40/15.84 RGB-D IoU/mIoU, more than double the self-supervised baselines; geometric IoU improves 76.1% over the next-best DROID-Splat. The paper also introduces ReplicaOcc, where learned baselines collapse to 0.00-0.01 while FreeOcc transfers zero-shot to 55.65/20.90 and runs online at 25 FPS on a live RealSense D435i stream.
