Humanoid robots keep falling over inside gaussian splats
Figure: Pham et al., VinMotion / University of Southern California · Research
Aug 17, 2026, 7:30 AM ETResearch
Quan-Dung Pham and colleagues at VinMotion and the University of Southern California have built HumanoidVLN, a physics-grounded simulator whose environments are drawn partly from gaussian splatting reconstructions of real interiors — and whose headline result is how badly current navigation models cope with having legs.
Why it matters: Vision-language navigation benchmarks have mostly assumed a wheeled robot gliding along a floor plan. A bipedal robot has to stay upright, its camera pitches and rolls with every step, and no two humanoid platforms are shaped alike.
The test environments are real interiors, captured as gaussian splats and filtered down to those with more than 100 square metres of navigable floor. The capture is not the demonstration — it is the ground the robot has to walk on, and it has to hold up under a physics engine rather than a camera.
The result: Across four models and four robots, the best combination — JanusVLN — reaches a mean success rate of 43.55%. That is the ceiling, not the average.
The fall rates are more striking than the success rates. Unitree's H1 goes over in 27% of episodes under the steadiest model and 71% under the least steady, while the other three platforms stay between roughly 2.7% and 10%. Same instructions, same scenes — the body decides the outcome.
Zoom in:
- Built on NVIDIA Isaac Sim, with four humanoids spanning 10–12 lower-body degrees of freedom and heights from 1.17m to 1.80m.
- Control is a reinforcement-learning locomotion policy under interchangeable PD or MPC path trackers, so the controller can be varied independently of the navigation model.
- 933 collision-aware reference episodes across 87 scenes and 17 indoor classes, each with one precise instruction plus formal, natural and casual paraphrases.
- A 20-episode sim-to-real pilot on a real Unitree G1 found navigation error in simulation correlated with reality at r=0.935, mean absolute difference 0.68m.
The geometry problem: Buried in an ablation is a finding worth pulling out: vanilla gsplat produces noisy, discontinuous surface normals, particularly around textureless walls, furniture boundaries and thin structures — which makes the resulting collision meshes unusable for physics.
Their pipeline adds depth-normal consistency and unbiased depth rendering to get surfaces coherent enough to fuse into a mesh a simulator will accept. Pretty renders were never the bottleneck; walkable geometry was.
Yes, but: The sim-to-real pilot covers two scenes and one model checkpoint. The authors are explicit that it is episode-level evidence, not a claim that reconstructed scenes generalise.
They also name human verification as a scaling bottleneck — every instruction in the benchmark passed through a human, which is what makes 933 episodes a respectable number rather than a small one.
