SubSplat: High-Resolution Pixel-aligned 3DGS via Sub-pixel Gaussian Reparameterization

SubSplat: High-Resolution Pixel-aligned 3DGS via Sub-pixel Gaussian Reparameterization

Paper: SubSplat: High-Resolution Pixel-aligned 3DGS via Sub-pixel Gaussian Reparameterization
Link: arXiv:2607.20813　|　Benchmarks: RealEstate10K / ACID　|　Basis: MVSplat + Deformable Attention

One-line summary: Introduces a Sub-pixel Gaussian Reparameterizer (SPGR) that subdivides primary Gaussians from low-resolution features into fine-grained primitives, restoring structural density while maintaining stable backbone latency, achieving real-time high-fidelity rendering at 42ms/frame.

Background and Motivation

Pixel-aligned Gaussian splatting enables efficient and generalizable novel-view synthesis — feed-forward networks train across diverse scenes predicting Gaussian primitives on a fixed image grid without per-scene optimization. But high-resolution rendering faces a critical trade-off: increasing input resolution improves detail at quadratic network computational cost; maintaining low-resolution inputs stabilizes cost but yields insufficient Gaussian density and artifacts.

The core contradiction is the grid-anchored design — total primitive count is tied to input resolution. Directly increasing input resolution makes the backbone process a denser grid, and since total latency is dominated by the backbone forward pass, doubling resolution requires 4× computation. Using low-resolution inputs to render sparse Gaussians on high-resolution screens leads to under-parameterization — blur and halo artifacts. Existing methods are trapped between (i) quadratic backbone cost or (ii) quality degradation from coarse input grids.

SubSplat's core idea: instead of increasing backbone input resolution (avoiding quadratic cost), introduce SPGR to subdivide primary Gaussians into $K$ sub-pixel primitives, restoring structural density directly from low-resolution features. $K$ is the squared ratio of target to input resolution ($K=4$ for $2\times$, $K=16$ for $4\times$).

Method Details

1. Feature Aggregation

SubSplat is an end-to-end pipeline: a pixel-aligned backbone (MVSplat) initializes primary Gaussians, a deformable-attention module aggregates multi-view features to enrich primitive descriptions, and SPGR subdivides primary Gaussians into sub-pixel primitives. Feature aggregation has geometry and appearance paths.

Geometry feature aggregation maps primary Gaussian attributes (position, scale, rotation, opacity) through small MLPs to a geometry query, projected onto reference views to aggregate cross-view geometric context via deformable attention. Three stacked deformable-attention layers progressively aggregate local-to-global cross-view features, yielding the geometry descriptor $\mathbf{f}^g_p$. Appearance feature aggregation maps spherical harmonic coefficients to an appearance query, conditioned on the geometry descriptor via deformable attention — geometry determines "where to look," appearance determines "what to amplify."

2. Geometry Reparameterization

SPGR's geometry head $g_\theta$ predicts spatial properties for each sub-pixel primitive: sub-pixel offsets $(\Delta u_k, \Delta v_k)$, depth residual $\Delta z_k$, log-scale residuals, and axis-angle rotation. Position is back-projected through camera intrinsics and adjusted along the viewing ray:

$$\mathbf{\mu}_{p,k}=\mathbf{\mu}_{p}+R_{c2w}\begin{bmatrix}z_{\text{sel}}\Delta u_{k}/f_{x}\\z_{\text{sel}}\Delta v_{k}/f_{y}\\0\end{bmatrix}+\Delta\hat{z}_{k}\,R_{c2w}\begin{bmatrix}0\\0\\1\end{bmatrix}$$

Depth residual is bounded via $\tanh$: $\Delta\hat{z}_k=\tanh(\Delta z_k)$. Anisotropic scale inherits and locally refines: $\mathbf{s}_{p,k}=\mathbf{s}_p\odot\exp(\Delta\log\mathbf{s}_k)$. Rotation residual converts to quaternion, composes with primary quaternion, and normalizes. Covariance: $\Sigma_{p,k}=R(q_{p,k})\text{diag}(\mathbf{s}_{p,k}^2)R(q_{p,k})^\top$.

Figure 1: SubSplat generates sub-pixel reparameterized Gaussians from low-resolution features, avoiding quadratic cost of high-resolution inputs.

3. Opacity Redistribution and Color Modulation

The appearance head predicts per-primitive opacity weights and color gains. Opacity is softmax-normalized and adjusted by projected area; final opacity is distributed from the primary Gaussian:

$$\alpha_{k}=\alpha_{p}\cdot\frac{\tilde{w}_{k}}{\sum_{j}\tilde{w}_{j}}$$

This enforces distributional integrity from primary to sub-pixel primitives, prioritizing density on intricate scene details. Color is modulated by a bounded gain: $\mathbf{h}_{p,k}=\gamma_k\mathbf{h}_p$. Anti-aliasing enforces a minimum projected area $A_{\min}$ to mitigate flickering.

4. Training Objective

The network is trained with ground-truth RGB images using weighted $\ell_2$ and LPIPS losses:

$$\mathcal{L}=\mathcal{L}_{\ell_2}+\lambda\,\mathcal{L}_{\text{LPIPS}}$$

with $\lambda=0.05$, trained for 300K iterations on a single NVIDIA A100, batch size 12, Adam optimizer with learning rate $2\times 10^{-4}$.

graph TD
  A[Low-res input 256x256] --> B[MVSplat backbone: primary Gaussians]
  B --> C[Deformable Attention: geometry+appearance aggregation]
  C --> D[SPGR: sub-pixel reparameterization]
  D --> E[Geometry head: position/depth/scale/rotation]
  D --> F[Appearance head: opacity/color]
  E --> G[K sub-pixel Gaussians]
  F --> G
  G --> H[High-res rendering 512x512 / 1024x1024]
  style D fill:#f5a623,stroke:#b97316,color:#fff
  style C fill:#4a90d9,stroke:#2c5f8a,color:#fff
  style H fill:#7ed321,stroke:#4a8a14,color:#fff

Experimental Results

Resolution Scalability

Reconstructing from $256\times 256$ inputs and evaluating at $512\times 512$ ($\times 2$) and $1024\times 1024$ ($\times 4$), SubSplat achieves SOTA across all metrics on RealEstate10K and ACID, with minimal degradation as target resolution scales. Baselines couldn't be evaluated at $1024\times 1024$ inputs due to OOM.

Method	Input Res	Output Res	Latency(ms)	Peak Mem(GB)	PSNR
PixelSplat	512	512	86	15.2	24.35
MVSplat	512	512	58	12.1	25.21
HiSplat	512	512	1960	28.5	25.40
SubSplat	256	512	42	6.8	25.52

Efficiency and Cost-Performance

At a fixed $512\times 512$ target, SubSplat achieves 25.52 PSNR and 0.850 SSIM at only 42ms/frame (24 FPS) — the only real-time method among evaluated baselines. HiSplat achieves competitive quality but 1960ms latency from its three-stage feed-forward design. SubSplat's low-resolution feature recovery significantly reduces compute and memory budgets.

Figure 2: Comparison with image-space upsamplers. SubSplat densifies geometry in 3D space, recovering structural details that image-space methods cannot.

Comparison with Image-Space Upsamplers

Method	PSNR	SSIM	LPIPS	Latency(s)
MVSplat + Bilinear	24.01	0.801	0.237	0.042
MVSplat + HiT-SR	23.88	0.807	0.203	0.215
SubSplat	25.52	0.850	0.167	0.042

Bilinear interpolation remains blurry; SR head introduces ringing and cross-view inconsistencies. SubSplat optimizes Gaussian distributions via sub-pixel reparameterization, capturing geometry-aware details that 2D upsamplers cannot recover — they cannot resolve grid-anchored primitive limitations.

Figure 3: Ablation. SPGR provides the primary performance gain; feature aggregation ensures structural consistency at larger scales.

Ablation Study

Module	K=4 PSNR	K=16 PSNR	Contribution
Baseline (MVSplat)	19.46	16.98	—
+ Sub-pixel Reparam.	25.15	19.49	+5.69 / +2.51 dB
+ Feature Aggregation (Full)	25.52	22.65	+0.37 / +3.16 dB

SPGR provides +5.69 dB at $K=4$ as the primary recovery mechanism. Feature aggregation's contribution grows with task difficulty — +3.16 dB at $K=16$, as larger scale ratios need richer cross-view context. Three-stage deformable attention yields 1.58 dB over one-stage, as iterative feature sampling better captures spatial details.

高斯属性向量

$$ \mathbf{a}_{p}=[\,\boldsymbol{\mu}_{p},\mathbf{s}_{p},q_{p},\alpha_{p}\,]\in\mathbb{R}^{11} $$

几何特征编码

$$ \mathbf{z}^{g}_{p}=W_{g}\!\big(\phi_{x}(\boldsymbol{\mu}_{p})+\phi_{s}(\mathbf{s}_{p})+\phi_{q}(q_{p})+\phi_{\alpha}(\alpha_{p})\big) $$

Limitations

Author-stated: Rendering overhead increases with primitive count. Future work will explore content-aware primitive density control to further optimize the rendering throughput vs. structural fidelity trade-off.

Analysis: While backbone latency is stable, the 16× primitive count at $K=16$ significantly increases differentiable rasterization rendering overhead. Bounded sub-pixel offsets and depth residuals ($\tanh$) stabilize but may limit extreme geometric detail expression. Feature aggregation relies on deformable attention cross-view projections that may degrade with sparse input views or wide baselines. The training objective uses only $\ell_2$ + LPIPS without explicit geometry consistency or depth optimization. The $\times 4$ model trained at $128\to 512$ on 720p is directly applied to $256\to 1024$, relying on scale-ratio invariance — absolute scale differences may introduce unverified generalization risks. SPGR's $K$ is fixed rather than content-adaptive, potentially producing redundant primitives in texture-sparse regions.

Conclusion and Outlook

SubSplat achieves high-fidelity rendering by generating fine-grained sub-pixel primitives directly from low-resolution features, mitigating quadratic backbone cost while maintaining stable network latency across extended output scales. The core innovation is shifting detail enhancement from backbone computation to the subdivision stage — SPGR reparameterizes primary Gaussians into sub-pixel primitives, with deformable attention feature aggregation capturing cross-view high-frequency details. The 42ms/frame real-time performance and 6.8GB peak memory represent a significant quality-efficiency advantage over full-resolution baselines. Ablations reveal SPGR provides the primary gain while feature aggregation becomes increasingly important at larger scale ratios — this synergy provides insight into modular design for multi-scale rendering. SubSplat demonstrates that 3D-space geometry densification outperforms image-space upsampling, which cannot resolve the fundamental grid-anchored primitive limitation.

The secret of high resolution lies not in larger inputs but in smarter subdivision — transforming one coarse Gaussian into multiple fine sub-pixel primitives, recovering detail in 3D space rather than image space. This is the core of SubSplat's real-time high-fidelity from low-resolution inputs.