Skip to content
RobotWorld
Back to Papers

PAPER DEEP DIVE

Paper3D Gaussian Splatting3DGS

Exploration Matters for Escaping the Blur Trap in 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) employs Gaussian primitives for explicit scene representation, facilitating real-time, high-fidelity reconstruction and novel view synthesis of complex scenes. However, the explicit modeling inherent in 3DGS introduces a gradient bias during optimization, rendering its non-convex optimization process highly susceptible to convergence toward local suboptimal solutions. This constitutes a fundamental limitation in 3DGS optimization, which we term the Blur Trap. To address this limitation, we integrate simple explicit exploration into the 3DGS optimization framework. First, through rigorous mathematical analysis of the 3DGS optimization formulation, we identify the underlying optimization bias responsible for the Blur Trap and categorize it into two distinct subtypes: the Far-Side Blur Trap and the Near-Side Blur Trap. Subsequently, we propose two highly straightforward exploration strategies (Random Seeding and Random Splitting) to mitigate the far-side and near-side blur traps, respectively. Experimental validation demonstrates that the incorporation of these exploration operators effectively and complementarily overcome the Blur Trap, achieving high-quality rendering performance across multiple datasets. Project page: https://chengbo-wang.github.io/ExploreGS/

Chengbo Wang, Guozheng Ma, Jinhong Wu, Tie Ji, Yizhen LaoJuly 20, 202612 min read
中文
Title:Exploration Matters for Escaping the Blur Trap in 3D Gaussian Splatting
Authors:Chengbo Wang, Guozheng Ma, Jinhong Wu, Tie Ji, Yizhen Lao
Affiliations:Hunan University, Nanyang Technological University
Project Page:chengbo-wang.github.io/ExploreGS
Keywords:3D Gaussian Splatting, Blur Trap, Exploration-Exploitation Balance, Differentiable Rendering, Novel View Synthesis
One-line summary:This paper mathematically proves that 3DGS optimization suffers from a systematic defect called the "Blur Trap," rooted in gradient orthogonality to the viewing direction and alpha-blending-induced gradient attenuation. The authors escape this trap using just two minimal stochastic exploration operators (Random Seeding + Random Splitting), achieving SOTA across five benchmarks.
Blur Trap overview
Figure 1: Escaping the Blur Trap via stochastic exploration. Far-Side Blur Trap (left) from depth-gradient deficiency; Near-Side Blur Trap (right) from 2D-gradient attenuation. Random Seeding and Random Splitting address each subtype.

1. Background and Motivation

3D Gaussian Splatting (3DGS) represents scenes explicitly through anisotropic Gaussian primitives, achieving breakthrough performance in real-time rendering and photorealistic novel view synthesis. Unlike implicit neural representations such as NeRF, each Gaussian in 3DGS is updated by gradients derived directly from the camera projection model, with no intermediate neural network. This physically grounded design brings efficiency but also plants a fundamental hidden danger.

In practice, researchers and users repeatedly observe a systematic failure pattern: certain regions in rendered scenes remain persistently blurry regardless of how many training views are provided. These blurry regions are not randomly distributed but concentrate in two types of locations — distant background content (mountains, skylines) and occluded near-field regions (grass behind a rack). The persistence of blur under abundant supervision indicates the problem lies not in insufficient data but in the optimization process itself. The authors name this systematic failure the Blur Trap.

Existing 3DGS improvement efforts fall into three lines, none touching the root cause: the first refines the densification criterion (AbsGS, Pixel-GS) but still depends on the same biased gradient signal; the second (HoGS) reparameterizes positions in homogeneous coordinates to better represent far-field content but introduces no new depth-direction optimization signal; the third (3DGS-MCMC) injects Langevin noise into positions, closest in spirit to this work, but applies noise uniformly across the scene without diagnosing the two distinct geometric failure origins.

2. Mathematical Roots of the Blur Trap

2.1 The 2D Gradient Dominance Effect

In 3DGS, Gaussian position updates comprise three gradient branches: the 2D projected position gradient, the 2D covariance gradient, and the spherical harmonics coefficient gradient. The authors empirically find that the 2D position component dominates the other two by two to three orders of magnitude throughout training, so the 3D position update is effectively governed entirely by 2D reprojection error. This means the optimizer is essentially performing greedy screen-space exploitation.

2.2 The Gradient Orthogonality Theorem

The core finding: through rigorous derivation of the camera projection pipeline, the authors prove that the 3D position gradient derived from the 2D gradient is always orthogonal to the viewing ray from the camera center to the Gaussian primitive. In other words, the optimizer can only move primitives within the plane perpendicular to the viewing ray and never along the depth direction. This is the mathematical root of the Far-Side Blur Trap.

Specifically, let $\mathbf{P}_{\mathrm{3D}}$ be the world-space position and $\mathbf{P}_{\mathrm{cam}}$ the camera position. The rendering pipeline passes through view transform $\mathbf{V}$, projection $\mathbf{P}$, perspective division, and viewport transformation. The 2D position gradient decomposes into a product of three Jacobian matrices:

$$ \mathbf{g}_{\mathrm{2d}}=\frac{\partial L^{2D}}{\partial\mathbf{P}_{\mathrm{3D}}}=\frac{\partial L}{\partial\mathbf{p}_{\mathrm{2D}}}\frac{\partial\mathbf{p}_{\mathrm{2D}}}{\partial\mathbf{p}_{\mathrm{ndc}}}\frac{\partial\mathbf{p}_{\mathrm{ndc}}}{\partial{}^{h}\mathbf{p}}\frac{\partial{}^{h}\mathbf{p}}{\partial\mathbf{P}_{\mathrm{3D}}} $$

The viewport Jacobian $\frac{\partial\mathbf{p}_{\mathrm{2D}}}{\partial\mathbf{p}_{\mathrm{ndc}}}$ has its third column identically zero, and the null space of the perspective division Jacobian contains the homogeneous coordinate vector ${}^{h}\mathbf{p}$. These two geometric facts combined force the inner product of the gradient with the viewing direction to be exactly zero:

$$ \frac{\partial L^{2D}}{\partial\mathbf{P}_{\mathrm{3D}}}\cdot\mathbf{D}_{\mathrm{C2G}}=\frac{\partial L^{2D}}{\partial\mathbf{P}_{\mathrm{3D}}}(\mathbf{P}_{\mathrm{3D}}-\mathbf{P}_{\mathrm{cam}})=0 $$

This theorem profoundly reveals that the differentiable rendering formulation of 3DGS provides no optimization signal along the depth direction. Primitives cannot move to their correct depth through gradient descent, causing distant geometry to remain permanently blurred.

2.3 Alpha-Blending-Induced Gradient Attenuation

The second failure mechanism comes from the alpha-blending pipeline. In forward rendering, pixel color is accumulated from depth-sorted Gaussians:

$$ C_{p}=\sum_{i=1}^{N}c_{i}\alpha_{i}T_{i},\quad T_{i}=\prod_{j=1}^{i-1}(1-\alpha_{j}) $$

where $T_i$ is the accumulated transmittance of the first $i-1$ primitives. Since $T_i$ decreases monotonically with index $i$, later occluded primitives contribute minimally to the final pixel color and their backpropagated gradients are severely attenuated. The authors further prove an upper bound on the 2D position gradient magnitude:

$$ \left|\frac{\partial\hat{\mathbf{c}}_{c}}{\partial\alpha_{m}}\right|\leq T_{m-1}=\prod_{j=1}^{m-1}(1-\alpha_{j}) $$

Since each $\alpha_j \in [0,1]$, the transmittance sequence is monotonically non-increasing: $T_0=1\geq T_1\geq T_2\geq\cdots\geq T_N\geq 0$. Rear Gaussians receive progressively weaker gradient signals, and their periodic average gradient magnitude consistently fails to reach the densification threshold $\tau_{\text{split}}$, causing densification failure. This is the formation mechanism of the Near-Side Blur Trap — occluded primitives cannot obtain densification opportunities because their gradients are attenuated by alpha blending.

Gradient magnitude vs depth
Figure 5: 2D gradient magnitude and camera-space depth of primitives contributing to a central pixel. Earlier primitives exhibit substantially larger gradients, spanning three distinct depth intervals.

2.4 Formation of the Blur Trap

Combining screen-space gradient dominance, depth-gradient deficiency, and blending-induced densification failure, the 3DGS optimization forms an exploitation-only dynamic. Deterministic gradient descent cannot escape this local optimum basin because the required depth-probing signals are mathematically suppressed by the rendering formulation itself. This systematic stagnation is not a transient optimization phase but an inherent consequence of the differentiable rendering pipeline.

Blur trap under sufficient supervision
Figure 6: Despite abundant training views, distant regions (mountains) and occluded areas (beneath the bench) show persistent blur. Dashed boxes denote dense supervision views for the blurred areas.

3. Method: Two Minimal Exploration Operators

3.1 Design Principles

The root cause of the Blur Trap lies in the physics-based formulation of the differentiable rendering pipeline. Both parameter updates and adaptive densification rely exclusively on backpropagated gradients, inheriting a systematic bias. The core design principle to break this: bypass purely gradient-driven exploitation by injecting explicit exploration that operates independently of 2D gradient magnitude.

The authors deliberately adopt the simplest possible operators to validate the intrinsic efficacy of exploration. For the Far-Side Blur Trap, exploration must establish update pathways along the depth axis; for the Near-Side Blur Trap, it must proactively trigger densification for primitives whose gradients fall below $\tau_{\text{split}}$. This minimalist philosophy aims to prove that what 3DGS optimization is missing is the exploration principle itself, not sophisticated engineering.

3.2 Random Seeding

The Random Seeding operator uniformly samples $N_{\text{seed}}=20$ candidate positions within the minimum bounding box of all existing Gaussians at each densification iteration, injecting new seed Gaussians into the primitive collection. These seeds directly probe depth intervals inaccessible to gradient updates, bypassing the viewing-gradient orthogonality constraint.

The elegance lies in its self-cleaning mechanism: seeds in invalid or empty regions fail to reduce photometric loss and are automatically removed by native opacity pruning; seeds in geometrically plausible regions are guided by the standard 2D reprojection loss, refining their positions within the plane orthogonal to the viewing ray, eventually triggering adaptive densification. This forms a complementary optimization loop: Random Seeding provides global depth exploration while deterministic gradients drive local planar exploitation.

$$ \mathbf{P}_{\text{seed}}^{(k)} \sim \mathcal{U}\big(\mathbf{B}_{\min},\,\mathbf{B}_{\max}\big),\quad k=1,\ldots,N_{\text{seed}},\quad N_{\text{seed}}=20 $$

3.3 Random Splitting

The Random Splitting operator bypasses the gradient-dependent densification criterion by proactively targeting large-scale Gaussians across the entire scene for exploratory splitting. Specifically, it samples primitives based on their mean scale and splits only a small subset ($N_{\text{split}}=20$) per iteration to preserve training stability. This allows occluded regions to circumvent 2D gradient suppression and actively participate in densification.

$$ \mathcal{S}_{\text{split}}=\text{RandomSample}\big(\{G_m:\text{scale}_m>\bar{s}\},\,N_{\text{split}}\big),\quad N_{\text{split}}=20 $$

Once these regions are populated with finer primitives, they rapidly converge to optimal configurations under unoccluded training views. Random Splitting directly addresses the fundamental limitation: in occluded areas, alpha-blending attenuation dilutes 2D position signals, preventing primitives from reaching the statistical splitting threshold. Random Splitting restores this capability by decoupling structural refinement from gradient magnitude.

Stochastic exploration operators
Figure 7: Stochastic exploration for escaping the Blur Trap. Top: physically-derived 3D position gradients are orthogonal to viewing direction; Random Seeding probes unexplored depths. Bottom: alpha-blending attenuates gradients below threshold; Random Splitting restores detail via probabilistic splitting.

4. Experimental Results

4.1 Cross-Dataset Reconstruction Quality

The authors comprehensively evaluated on five benchmarks: Mip-NeRF 360, Tanks & Temples, Deep Blending, OMMO, and DL3DV. Both Seed Exploration and Split Exploration consistently improve reconstruction fidelity, with their combination achieving optimal performance on most benchmarks.

MethodMip-360 PSNR↑Mip-360 LPIPS↓T&T PSNR↑T&T LPIPS↓DB PSNR↑OMMO PSNR↑DL3DV PSNR↑
3DGS27.520.21523.730.16929.8030.4927.16
HoGS (50K)27.550.20124.230.16029.2130.6628.16
Seed Exp.27.680.21424.300.16329.7830.8527.74
Split Exp.27.950.19224.300.13930.0131.2928.47
Seed & Split27.960.19524.370.13929.9831.2728.43

Several important patterns emerge: Seed Exploration improves unbounded scenes (T&T) by effectively probing distant depths; Split Exploration excels in occluded regions (OMMO), boosting PSNR from 30.49 to 31.29 and reducing LPIPS from 0.142 to 0.121. Their combination reaches optimum in most scenarios, validating that both Blur Trap variants can be simultaneously resolved.

4.2 Gaussian Primitive Allocation

MethodMip-360T&TDBOMMODL3DV
3DGS2.72M1.57M2.48M1.78M1.14M
HoGS (50K)4.40M2.16M2.43M2.01M1.61M
Seed Exp.2.62M1.51M2.33M1.68M1.11M
Split Exp.2.58M2.13M0.86M1.77M2.13M
Seed & Split2.53M2.11M0.79M1.77M2.11M

This table reveals a key insight: on Deep Blending, Split Exploration reduces Gaussian count by 65% (2.48M to 0.86M) while maintaining superior geometric fidelity. In complex scenes (T&T, DL3DV), the strategy adaptively increases primitive allocation to resolve fine geometric details. This adaptive allocation reduces redundancy in simple areas while refining complex structures, embodying the principle of allocating computation where it most improves fidelity.

4.3 The Blur Trap in 4D Gaussian Splatting

Due to the shared optimization pipeline, the Blur Trap naturally propagates from 3DGS to 4D Gaussian Splatting. On the Neu3D dataset, despite extensive multi-view coverage, occlusion-susceptible regions remain severely blurred — distant buildings through windows, text on indoor bottles, steak textures dynamically obscured by flames. The authors integrated only $N_{\text{split}}=5$ extremely sparse Random Splitting into 4DGS and successfully bypassed the Blur Trap.

MethodPSNR↑SSIM↑MS-SSIM↑LPIPS-vgg↓LPIPS-alex↓
4DGS30.5750.93140.96590.15070.0602
Split Exp.30.8200.93690.96890.13800.0488
Split & Seed31.0850.93790.97020.13770.0483

Gains are most pronounced in perceptual fidelity (LPIPS), dropping from 0.1507 to 0.1377. That such minimal intervention resolves severe blur confirms that standard splitting mechanisms, which strictly rely on 2D positional gradients, are fundamentally bottlenecked by signal attenuation.

4.4 Ablation: Depth Supervision vs. Stochastic Exploration

To validate the critical role of depth-directed signals, the authors used Depth Anything V2 to generate pseudo-depth maps and integrated a depth regularization term $\mathcal{L}_{\text{Depth}}$ into standard 3DGS. While pseudo-depth supervision consistently improved fidelity (confirming depth supervision absence as the primary driver of the Far-Side Blur Trap), its gains remained inferior to sparse Random Seeding. More critically, combining $\mathcal{L}_{\text{Depth}}$ with stochastic exploration yielded diminishing returns — inaccurate depth signals deterministically constrain the optimizer, actively suppressing stochastic exploration dynamics.

Another ablation compared Random Splitting against simply lowering the threshold $\tau_{\text{split}}$. Lowering the threshold fails to yield meaningful fidelity improvements and instead introduces excessive redundant primitives. In contrast, even sparse Random Splitting efficiently enhances reconstruction quality while maintaining a significantly leaner primitive allocation, confirming that performance gains originate from targeted structural exploration rather than brute-force densification.

5. Method Workflow

flowchart TD A["Standard 3DGS Optimization Loop"] --> B{"Gradient Analysis"} B -->|"2D gradient orthogonal to viewing ray"| C["Far-Side Blur Trap\nDepth gradient deficiency"] B -->|"Alpha-blending attenuates transmittance"| D["Near-Side Blur Trap\nDensification failure"] C --> E["Random Seeding\nUniform sampling N_seed=20 depth positions"] D --> F["Random Splitting\nRandom selection N_split=20 large-scale primitives"] E --> G["Invalid seeds → opacity pruning removes"] E --> H["Valid seeds → 2D gradient refinement → densification"] F --> I["Occluded primitives gain densification opportunity"] H --> J["Escape Blur Trap\nSOTA fidelity"] I --> J style C fill:#ffe0b2 style D fill:#ffcdd2 style J fill:#c8e6c9

6. Limitations and Discussion

Limitation 1: Spatial distribution of seed sampling. Current Random Seeding uses uniform distribution within the minimum bounding box. For extremely non-uniform scenes (e.g., outdoor scenes with mostly empty sky), uniform sampling may cause many seeds to land in invalid regions and be pruned, wasting computation. A more heuristic sampling strategy (e.g., depth-prior-based non-uniform distribution) could improve efficiency, but this would deviate from the minimalist design philosophy.
Limitation 2: Hyperparameter sensitivity. While defaults $N_{\text{seed}}=20$ and $N_{\text{split}}=20$ perform well on most scenes, the paper does not systematically explore their impact on extreme scenarios (tiny indoor scenes or very large outdoor scenes). Additionally, the 4DGS experiment uses only $N_{\text{split}}=5$, whose generalization to other dynamic scene datasets remains unverified.
Limitation 3: Far-Side and Near-Side co-optimization. On Deep Blending, the Split & Seed combination's LPIPS is slightly worse than Split Exp. alone (0.249 vs 0.248), suggesting mild mutual interference between the two operators in certain scenes. The paper does not deeply analyze this marginal degradation or propose adaptive scheduling to dynamically balance the two exploration intensities.

7. Conclusion and Implications

The most important contribution is not the two specific operators themselves but the diagnosis: it mathematically proves that 3DGS's Blur Trap stems from the physics-based rendering formulation — viewing-gradient orthogonality and alpha-blending attenuation systematically suppress depth-directed signals. The success of two minimal stochastic exploration operators proves a profound conclusion: reconstruction bottlenecks stem from optimization dynamics, not model capacity.

This finding carries methodological significance for the entire differentiable rendering field: it introduces the classic exploration-exploitation balance concept from reinforcement learning into 3D reconstruction optimization, showing that deterministic gradient descent has systematic blind spots in non-convex optimization. Explicit exploration is a necessary component of differentiable rendering. Future work can design more sophisticated exploration strategies on this foundation, but this paper has already demonstrated that even the simplest random exploration suffices to break the fundamental deadlock in 3DGS optimization.

"Reconstruction bottlenecks stem from optimization dynamics, not model capacity. Explicit exploration is a necessary component for differentiable rendering, advocating a shift toward balanced exploration-exploitation dynamics."

Related Papers

Pre-training Visual Dexterity in Simulation

Pre-training Visual Dexterity in Simulation

Large-scale pre-training has made robot policy fine-tuning increasingly data-efficient, but this progress has largely been driven by datasets and embodiments built around simple parallel-jaw grippers. Dexterous, multi-fingered hands remain comparatively data-starved because real teleoperation is costly to scale, while human hand video is off-embodiment and requires lossy pose estimation and retargeting. We introduce Simulation Pre-training for Dexterity (SPD), a pre-training framework for dexterous manipulation that uses data entirely collected in simulation. In SPD, humans manipulate virtual objects inside a VR headset, enabling on-embodiment trajectories and robot-free collection. With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective. We study the benefits of simulation pre-training on real-world tasks by fine-tuning on 1-2 hours of physical demonstrations on a 56-DoF bimanual dexterous setup. We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation. We perform ablation studies, measuring the benefits of history conditioning and short action chunks for reactive control.

灵巧操作灵巧手预训练Aug 16, 2026
FA-RDP: A Frequency-Adaptive Reactive Diffusion Policy for Contact-Rich Manipulation

FA-RDP: A Frequency-Adaptive Reactive Diffusion Policy for Contact-Rich Manipulation

In contact-rich manipulation, action multimodality and reactivity dominate different stages of a single episode. Before contact, multiple trajectories might be equally valid, making it important to preserve diverse action modes. After contact, geometric constraints and force limits narrow the solution space, while successful execution demands rapid responses to force feedback. However, standard diffusion policies use a fixed inference frequency and sampling steps throughout the episode, forcing a fundamental compromise: low-frequency, multi-step sampling better preserves pre-contact multimodality but responds slowly to force feedback, whereas high-frequency sampling improves reactivity but tends to collapse distinct pre-contact modes. To resolve this tradeoff, we present FA-RDP, a frequency-adaptive reactive diffusion policy. A shared multi-frequency visual-force Transformer predicts action chunks at both low and high frequencies, while a learned multimodality indicator dynamically selects multi-step low-frequency sampling before contact and one-step high-frequency sampling as action ambiguity decreases. We further introduce Manifold Consistency Distillation (MCD), which reparameterizes the diffusion network to predict actions on the robot action manifold while retaining DDPM-based residual supervision. Experiments on three contact-rich manipulation tasks show that FA-RDP achieves the highest success rate while preserving diverse pre-contact trajectory modes. Code and videos are available at https://fa-rdp.github.io.

PaperJul 30, 2026
PhiZero: A World Model Built Around Physical Language

PhiZero: A World Model Built Around Physical Language

We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.

PaperJul 30, 2026
Booster Lab: A Data-Centric Pipeline for Learning Deployable Humanoid Locomotion Policies

Booster Lab: A Data-Centric Pipeline for Learning Deployable Humanoid Locomotion Policies

Humanoid robot motion learning requires not only task-oriented control policies but also physically feasible and natural behaviors that can be transferred to real robots. However, robot-feasible motion data are often scarce: raw human demonstrations may be incompatible with the robot morphology, open-source clips vary in quality, and simulation-collected robot trajectories still require feasibility checking. To address these challenges, we propose a data-centric training and deployment pipeline that integrates motion data curation, real-to-sim model adaptation, AMP-based reinforcement learning, and sim-to-real deployment. We validate the framework on the Booster T1 robot and further provide preliminary cross-platform validation on Booster K1.

步态优化人形机器人AMPJun 26, 2026