Foveated Rendering

Foveated rendering exploits the non-uniform resolution of human vision to dramatically reduce the computational cost of rendering for VR and AR displays. The human eye only perceives sharp detail in the fovea — a tiny region of the retina covering roughly 2° of visual angle. Peripheral vision, while sensitive to motion and contrast, perceives far less spatial detail. Foveated rendering leverages this by rendering full resolution only in the gaze region, progressively reducing quality in the periphery where the eye can't detect the difference.

The performance implications are substantial. A VR headset might need to render at 4K×4K per eye at 90 Hz — roughly 2.9 billion pixels per second. With foveated rendering, the high-resolution foveal region might cover only 5-10% of the frame, while the remaining 90-95% renders at 1/4 to 1/16 the pixel density. Net GPU savings of 50-70% are achievable, which can be reinvested in higher visual quality, longer battery life, or both.

There are two main approaches. Fixed foveated rendering renders highest quality in the center of the display and lower quality at the edges, based on the assumption that users mostly look straight ahead. This requires no eye tracking but wastes performance when users look away from center. Gaze-contingent foveated rendering dynamically adjusts the high-resolution region based on real-time eye tracking, providing optimal quality wherever the user looks. Apple Vision Pro, Meta Quest Pro, and PlayStation VR2 all implement gaze-contingent foveated rendering.

The technique introduces engineering challenges. Latency is critical: if the high-res region lags behind eye movement, users perceive flicker or blurriness at the gaze point. The motion-to-photon pipeline from eye tracker to rendered frame must be under 10-20 milliseconds. Transition boundaries between quality zones must be imperceptible — gradual blending prevents visible seams. Temporal stability requires that the quality transitions don't cause flickering as the eye moves.

For future lightweight AR glasses, foveated rendering may be essential rather than optional. Glasses form factors impose severe power and thermal constraints — there simply isn't room for the GPU horsepower needed to render high-resolution content across the full field of view. Foveated rendering, combined with efficient waveguide displays and MicroLED light engines, is part of the technology stack required to deliver compelling AR experiences in eyeglass-like form factors.

AI is enhancing foveated rendering through neural upsampling. Rather than simply rendering the periphery at lower resolution, AI models can intelligently reconstruct peripheral detail from sparse samples, maintaining higher perceptual quality at the same computational budget. This convergence of real-time rendering, eye tracking, and neural inference exemplifies how AI is being woven into every layer of the graphics pipeline.