A driving world model that only predicts the parts that move
Figure: Ma et al., CUHK / Fudan / Shanghai Academy of AI for Science · Research
Aug 28, 2026, 11:50 AM ETResearch
World action models — the systems that predict what a scene will look like a few frames after an agent acts — almost all work in 2D video, which means redrawing every pixel of every frame, including the buildings that have not moved since the drive began. Yueen Ma and colleagues at the Chinese University of Hong Kong, Fudan, and the Shanghai Academy of AI for Science argue that this is a strange way to spend a prediction budget, and propose keeping the background.
Why it matters: A video world model treats the future as an image-generation problem. It has no explicit notion that the parked car on the left is an object, that the object has a pose, or that the terrace behind it was fully observed ninety frames ago and has not changed since. Every frame is regenerated from scratch, and the model spends most of its capacity reproducing content it already had.
4DGS-WAM makes the decomposition explicit. The scene is a 4D Gaussian splat split into a static bank and a set of dynamic object splats. A policy model predicts the actions of the other actors as rigid transforms; a world model applies those transforms to the objects' Gaussians. The background is not predicted at all — it is retrieved.
The consequence is that future prediction reduces to modelling the evolution of the things that actually evolve, which is a much smaller problem than synthesising a street.
By the numbers:
- On KITTI-MOT future prediction with the camera trajectory supplied, 4DGS-WAM scores 18.80 dB against Epona's 17.46 and DriveDreamer-2's 13.73. LPIPS is 0.161 against 0.203 and 0.426.
- On past reconstruction of the observed frames, dynamic-region PSNR is 25.53 dB against MonoGS at 15.51 and Flow4DGS-SLAM at 12.91 — a ten-decibel margin, and the figure to trust, because it is measured without the fusion underlay discussed below.
- Dynamic-region LPIPS is 0.025 against MonoGS's 0.224, an order of magnitude.
- The point-cloud baselines simply fail on this benchmark: VGGT-Ω and VGGT-SLAM score 6.66 and 6.64 dB, which is not a competitive reconstruction but a broken one.
- The paper is explicitly labelled a work in progress, and evaluates only short horizons — h=1 and h=3.
Yes, but: The headline full-frame number needs an asterisk that the authors supply themselves, twice. 4DGS-WAM's 27.63 dB against MonoGS's 20.42 dB is not measured on comparable outputs: the 4DGS-WAM renders fill uncovered pixels with a frozen NeoVerse fusion underlay, while every baseline is scored on its native output. A seven-decibel lead where one competitor has an extra generative component is not a seven-decibel lead.
To their credit the dynamic-region scores — the ones the argument actually rests on — come from the no-fusion evaluation, and the limitations section restates the caveat in plain language rather than leaving it in a table caption.
The second qualification is that the advantage narrows sharply when the model has to predict the camera as well as the objects. Given the camera, it leads by 1.34 dB. Predicting it, the score drops to 16.47 dB, below Epona's given-camera result, and the standard deviation more than doubles to 3.05. The static bank is only useful if you know where you will be standing.
And the architecture inherits a hard limit from its own premise. The static bank preserves what was observed but cannot synthesise background that becomes newly visible, there is no shape completion for the unobserved side of a car, and objects that appear after the prediction starts cannot be instantiated at all. The authors also note it assumes objects do not collide — fine for the non-contact driving they test, and disqualifying for the robot manipulation they name as the goal.
The big picture: The interesting claim here is not the benchmark position, which is qualified enough to be arguable. It is the architectural bet: that a world model should be a persistent scene plus a small dynamic delta, rather than a video generator conditioned on history.
That bet is cheap to state and expensive to honour. Every limitation in this paper is downstream of the same thing — a retrieved background is only correct while nothing new comes into view. The version of this idea that handles occlusion, contact and deformation is a much larger piece of work, and the authors say so.
