Eye Tracking

Eye tracking is the technology that measures the position and movement of a user's eyes in real time, determining precisely where they're looking (gaze direction), how their pupils respond (dilation), and how their eyes move (saccades, fixations, smooth pursuit). In spatial computing, eye tracking has evolved from a research tool into a core input modality and rendering optimization technology.

Modern eye tracking in headsets typically uses near-infrared illumination and camera-based detection. Infrared LEDs project light patterns onto the eye, and cameras capture the resulting reflections from the cornea and pupil. Computer vision algorithms process these images to estimate gaze direction at rates of 60-240 Hz. Some systems use glint tracking (tracking corneal reflections) while others use pupil center corneal reflection (PCCR) methods.

The technology serves multiple critical functions in mixed reality and spatial computing:

Foveated rendering is perhaps the most impactful application. The human eye only perceives high detail in the foveal region (about 2° of visual angle). Foveated rendering uses eye tracking to render full detail only where the user is looking, reducing peripheral resolution dramatically. This can cut rendering workload by 50-70%, enabling higher visual quality within the same GPU budget — or the same quality at much lower power, critical for lightweight AR glasses.

Gaze-based interaction uses eye direction as an input method. Apple Vision Pro made eye tracking its primary selection mechanism: look at an element and pinch to select. This is faster and more natural than head-pointing or controller-based selection for many tasks. Meta Quest Pro and PlayStation VR2 also integrate eye tracking for both interaction and rendering optimization.

Social presence in VR depends on eye contact and gaze behavior. Eye tracking enables avatars that look at what their human counterpart is looking at, dramatically improving the sense of connection in virtual meetings and social experiences. Without eye tracking, avatar interactions feel hollow because humans are extraordinarily sensitive to gaze direction.

The technology is also advancing into emotional and cognitive sensing. Pupil dilation correlates with cognitive load, arousal, and attention. Blink patterns indicate fatigue. Microsaccade behavior reflects focus and interest. These signals feed into adaptive systems that can adjust content difficulty, pacing, or notification behavior based on the user's cognitive state — raising both exciting possibilities and significant privacy considerations.

Eye tracking accuracy continues to improve, with current systems achieving sub-degree accuracy in headset form factors. The challenge for lightweight AR glasses is miniaturizing eye tracking hardware while maintaining performance — a problem being addressed through integration with waveguide optics and novel sensor architectures.