Photogrammetry

Photogrammetry is the science and technology of extracting 3D geometry, texture, and spatial measurements from photographs. By analyzing the parallax between overlapping images taken from different angles, photogrammetry software reconstructs the shape and appearance of real-world objects and environments as detailed 3D models.

The pipeline typically involves several stages. Structure from Motion (SfM) estimates camera positions and orientations from feature correspondences across images. Multi-View Stereo (MVS) generates dense point clouds by matching pixels across views. Surface reconstruction converts the point cloud into a mesh. Texture mapping projects the original photograph colors onto the mesh surface. The result is a photorealistic 3D model derived entirely from real-world imagery.

Photogrammetry has become a cornerstone of modern game and film production. Studios like Quixel (acquired by Epic Games) built massive libraries of photogrammetric assets — rocks, trees, surfaces, buildings — scanned from real environments. Unreal Engine 5's Nanite system was designed specifically to handle the high polygon counts that photogrammetry produces, eliminating the manual LOD creation that previously made scanned assets impractical for real-time use.

The technology has democratized significantly. Smartphone-based photogrammetry apps (like Polycam, RealityScan, and Luma AI) enable anyone to capture 3D objects with a phone camera. LiDAR sensors on recent iPhones and iPads improve depth estimation, producing better results in challenging conditions. Cloud processing services handle the computationally intensive reconstruction.

Photogrammetry intersects with newer 3D capture methods in important ways. NeRF and Gaussian splatting both typically use photogrammetry's SfM stage for initial camera estimation before applying neural reconstruction. These newer methods often produce superior results for view synthesis (rendering novel viewpoints) while photogrammetry excels at producing editable meshes that integrate with traditional 3D workflows.

For spatial computing and mixed reality, photogrammetry provides a path to digitizing real environments for virtual experiences. Architectural visualization, cultural heritage preservation, forensic documentation, and e-commerce product scanning all rely on photogrammetric techniques. As AI continues to improve reconstruction quality from fewer images and enable real-time capture, the boundary between the physical and digital world blurs further.