Gaussian Splatting

Gaussian splatting (formally, 3D Gaussian Splatting) is a scene representation and rendering technique that models a 3D environment as millions of small, colored, semi-transparent 3D Gaussian ellipsoids. Introduced in a landmark 2023 paper by Kerbl et al., it rapidly became one of the most significant advances in real-time 3D capture and rendering.

The approach works by representing a scene not as a mesh of triangles or a neural network, but as a cloud of 3D Gaussians — each defined by a position, covariance (shape/orientation), opacity, and color (represented as spherical harmonics for view-dependent appearance). These Gaussians are initialized from a sparse point cloud (typically from Structure-from-Motion) and then optimized through differentiable rendering to match a set of input photographs.

The key breakthrough is rendering speed. Unlike Neural Radiance Fields (NeRF), which require expensive per-ray neural network evaluation, Gaussian splatting renders by projecting ("splatting") the 3D Gaussians onto the 2D image plane and alpha-compositing them. This maps naturally to GPU rasterization pipelines, achieving real-time frame rates (30-200+ FPS) at resolutions where NeRF methods struggle to deliver interactive performance.

The quality-speed tradeoff is remarkably favorable. Gaussian splatting produces photorealistic novel views with sharp details, handling challenging elements like reflections, translucency, and fine geometry (hair, foliage, fences) that trip up mesh-based reconstruction methods. Training is also fast — typically 5-30 minutes on a single GPU versus hours for NeRF approaches.

Applications are expanding rapidly across domains. In mixed reality and spatial computing, Gaussian splats enable captured real-world environments to be rendered in headsets at the frame rates VR demands. In gaming and film, they offer a path to photorealistic environments derived from real locations. For e-commerce, product visualization from phone-captured photos becomes feasible.

The technique is evolving quickly. Extensions include dynamic Gaussian splatting (for video/motion), compressed representations for streaming, physics-aware splats for simulation, and generative models that produce Gaussian splat scenes from text or single images. Combined with text-to-3D generation, Gaussian splatting may become the default representation for real-time 3D content — particularly as WebGPU enables browser-based splat rendering, making captured 3D scenes as accessible as photographs on the web.