1 big thing

Splatting, but the light is sound and the camera is a transducer

Two rows of photoacoustic reconstructions of a branching vascular phantom, shown for SlingBAG, for PAGS, and as the ground-truth digital phantom. The SlingBAG panels carry a mottled noise floor around the vessels; the PAGS panels are cleaner, with the vessel network closer to the crisp white tracery of the phantom

Figure: Ge et al., Shanghai Jiao Tong University · Research

Photoacoustic tomography fires a laser pulse into tissue, waits for the absorbed light to heat and expand it, and listens to the ultrasound that comes back. Reconstructing where the sound came from requires knowing how fast it travelled — and tissue is not uniform, so the assumption of a single sound speed smears the result. A group at Shanghai Jiao Tong and collaborators have attacked that with machinery borrowed wholesale from Gaussian splatting, in which the camera is a transducer and the spherical harmonics encode acoustic propagation instead of colour.

Why it matters: The speed of sound in tissue varies by scene, and getting it wrong changes acoustic time-of-flight, which defocuses everything. The two existing answers are both awkward: calibrate the acoustic properties in advance, or optimise a dense physical model of the medium, which is expensive and scales badly in three dimensions.

PAGS declines to recover the medium at all. It keeps the initial pressure field as sparse Gaussian sources — the direct analogue of splats — and replaces the medium model with a compact field of what the authors call anisotropic path-averaged sound speed, parameterised by spherical harmonic probes. For a given source and a given transducer direction, that field returns the one number the reconstruction actually needs: the effective speed along that path.

This is a genuinely economical piece of reasoning. The full medium is a hard, high-dimensional inverse problem. The arrival time is not. PAGS solves only for what changes the answer, and the spherical harmonics — which in ordinary splatting encode how colour varies with viewing angle — here encode how sound speed varies with direction.

A schematic of a scene divided into a wireframe grid of cells against black. One cell is outlined in yellow and holds a sharp green cylinder; a blurred blue slab sits behind it and a red slab in front, standing in for the frozen regions flattened into single background and foreground images

Their trick makes VRAM independent of scene size. The test scenes were too small to show it.

6 hours ago

Six holographic reconstructions of laboratory equipment photographed against black through a HoloLens: a Bunsen burner and a rack of capped test tubes above, and below them a shredded, torn reconstruction of a mortar and pestle beside two further mortar-and-pestle models whose pestles are visibly deformed

PSNR said Gaussian splatting won. Seventeen people said it didn't.

3 days ago

A fairground drop-tower ride rendered twice: on the left from a degraded reconstruction, where the tower and foliage dissolve into white streaks and smears, and on the right after refinement, sharp and photographic against a clear sky

Give it scattered keypoints and it matches a full splat reconstruction

3 days ago

Key art for the plugin: a Gaussian-splat capture of a derelict stone barn with a corrugated roof, sitting on a white tile and surrounded by scattered blue and purple splat points, with the wireframe box of its tile bounds drawn around it

To put splats on the globe, he replaced the renderer

4 days ago

Theo Lindqvist

4 days agoTools

Babylon.js gave splats a hard ceiling

The base of a Gaussian-splat capture of a pasta box shown twice. Above, the shadow beside it breaks into a hard blocky wedge, circled in red by the developer. Below, after the fix, the same shadow falls away as a smooth gradient

Babylon.js shipped 9.23.0 on Wednesday with four Gaussian splat changes, two of which are substantial. The first admits something the field has been quiet about: distance-based level of detail does not actually bound how much you draw. The second fixes a shadow bug with an unusually clean diagnosis — the randomness was in the wrong place.

Why it matters: Streamed splat scenes can render an unbounded number of primitives. On a workstation that is a feature. On a laptop it is dropped frames, and on a phone it is a coin toss, because the only lever available was to reduce detail with distance — which lowers the count without capping it.

Go deeper (4 min. read) ⟶

Wen Jiang

4 days agoResearch

738 MB to 3.2 MB, without touching the training loop

The same view of a white bicycle leaning against a black bench on grass, rendered twice side by side — once from the uncompressed reconstruction and once from the compressed one — with no visible difference between them

The best 3D Gaussian splat compression systems have generally been the ones that reach into training and change how the scene is built, which makes them effective and almost impossible to take apart. A group at Fraunhofer HHI and Humboldt-Universität zu Berlin has published the opposite proposition: leave training alone entirely, take a finished vanilla 3DGS scene, and compress it afterwards in two independent stages.

Why it matters: Splat files are enormous, and the field's answer has been to fold size reduction into reconstruction — learned entropy models, anchor structures, quantisation-aware training. Those systems work, but they combine several strategies at once, which makes it hard to see where the gains actually come from and harder still to reuse a component in someone else's pipeline.

Go deeper (3 min. read) ⟶

Priya Raghunathan

4 days agoResearch

A driving world model that only predicts the parts that move

Three pairs of suburban street scenes from a car's forward camera, each pair showing the ground-truth future frame beside 4DGS-WAM's prediction of it at a different horizon

World action models — the systems that predict what a scene will look like a few frames after an agent acts — almost all work in 2D video, which means redrawing every pixel of every frame, including the buildings that have not moved since the drive began. Yueen Ma and colleagues at the Chinese University of Hong Kong, Fudan, and the Shanghai Academy of AI for Science argue that this is a strange way to spend a prediction budget, and propose keeping the background.

Why it matters: A video world model treats the future as an image-generation problem. It has no explicit notion that the parked car on the left is an object, that the object has a pose, or that the terrace behind it was fully observed ninety frames ago and has not changed since. Every frame is regenerated from scratch, and the model spends most of its capacity reproducing content it already had.

Go deeper (4 min. read) ⟶

Wen Jiang

5 days agoResearch

The water got murkier and the scores went up

A figure summarising the study: five systems and four water regimes across the top, a row of murky underwater renderings of a submerged structure, and beneath them two Gaussian point clouds of a sunken car — one coherent and car-shaped labelled 3DGS, one scattered and diffuse labelled SeaSplat, each captioned with its PSNR and chamfer error

Two underwater Gaussian splatting papers appeared on Monday, each declaring state of the art on an evaluation set it had assembled itself, neither citing the other. Olaya Álvarez-Tuñón and Stella Graßhof have now done the thing that was missing: taken five underwater splatting systems with public code, run them under a single protocol with shared poses, initialisation, compute budget and evaluator, and pushed them across four water regimes. The conclusion is that the architecture matters less than the setup, and that the numbers the field reports do not show it.

Why it matters: Underwater splatting has a benchmark culture problem that is easy to state and hard to see: the standard photometric metrics reward a model for fitting the water rather than the scene behind it. Haze is smooth, low-frequency and highly predictable, so a system that commits opacity to the veil scores well on held-out views while placing its geometry in the wrong place.

Go deeper (5 min. read) ⟶

Tessa Marchetti

6 days agoIndustry

Arrival.Space gave away the browser version of what it sells

The Tanks and Temples Truck scene — a pale blue vintage flatbed pickup parked on a pavement — rendered sharply inside the Splat.js browser interface, with a readout showing 579,748 splats and a Train button in the toolbar

Arrival.Space sells hosted reconstruction: you upload video, their servers turn it into a splat, you get it back. A week ago the company published Splat.js under an MIT licence — a library that does the whole job in a browser tab, on the viewer's own GPU, without the photographs ever leaving the device. As of this afternoon it scores better on the standard benchmark than the 2024 papers it was presumably meant to be a toy beside.

Why it matters: The division of labour in splatting has been stable for two years: capture on a phone, solve and train on a server or a workstation with CUDA, view in a browser. The browser was the last mile, and everything upstream of it was somebody's compute bill.

Go deeper (4 min. read) ⟶

Wen Jiang

5 days agoResearch

Your phone records three viewpoints per shot. The pipeline throws two away.

Three columns comparing a reconstruction of a hand moving across a carpet — ground truth, a monocular reconstruction in which the hand dissolves into a vertical smear, and an iPhone multi-camera reconstruction in which the closed fist is legible — each shown as a wide view above a zoomed crop

The three rear cameras on an iPhone see the same scene from viewpoints about five degrees apart, simultaneously, every time the shutter fires. An Apple Vision Pro's stereo pair sits roughly fifteen degrees apart. A Lytro Illum records a thirteen-by-thirteen grid of them at once. Shamus Li and colleagues point out that essentially every reconstruction pipeline takes one of those streams and discards the rest, then spends considerable effort inventing the parallax it just threw out.

Why it matters: The received wisdom is that consumer camera baselines are too small to matter — five degrees of separation between phone lenses is nothing next to walking around an object. So the field standardised on a moving monocular camera, and when the camera cannot move enough, on learned priors that hallucinate the missing views.

Go deeper (4 min. read) ⟶

View all stories ⟶