Multicam and Audio Sync

Multicam and audio sync is the process of placing recordings of the same event, from several cameras, microphones or call participants, on one common clock so they can be edited together. Productions once relied on slates, clapperboards and shared timecode. Most editors now also sync by sound: software compares the audio each device recorded and finds the time offset at which they match. Audio sync is what makes a multicam interview, a double-system shoot (camera plus separate recorder) or a remote podcast editable at all, and it is a prerequisite for any automated cut that switches between angles, a core feature of AI video editing.

The Three Ways to Sync

• Timecode. Devices share or are jammed to the same timecode, and the editor aligns by it. It's precise when it's set up correctly, but consumer cameras, phones and laptops usually don't have it.
• A marker. A clapperboard or a hand clap gives a sharp transient that is easy to find in every recording. It's reliable if someone remembered to do it.
• Waveform matching. The software compares the recordings' audio directly. It needs no preparation, which is why it became the default in desktop editors.

All the major editors now sync by sound. Adobe Premiere can build a multi-camera source sequence using audio as the synchronization point, and can synchronize individual clips by their waveforms. DaVinci Resolve offers Auto Sync Audio based on waveform in its Media Pool, and multicam angles can be synced by sound. Final Cut Pro's "Use audio for synchronization" option refines the alignment of multicam angles and synchronized clips from their waveforms. Red Giant's PluralEyes, for years the standalone tool for the job, entered limited maintenance in January 2023, as the capability had moved into the editors themselves.

How Waveform Sync Works

The core technique is cross-correlation: slide one signal past the other and measure how similar they are at each offset; the offset with the highest similarity is the alignment. Computing it directly is slow for long recordings, so implementations use the fast Fourier transform, which turns the sliding comparison into a multiplication in the frequency domain. A common refinement, generalized cross-correlation with phase transform (GCC-PHAT), whitens the spectrum so that sharp, broadband events dominate and room reverberation matters less. The method goes back to Knapp and Carter's 1976 paper on time-delay estimation, and aligning many recordings of one event is a well-studied problem: a 2014 Google paper, for example, aligned sets of recordings by pairwise cross-correlation of audio features and found spectral flatness to be the most reliable feature.

Raw waveforms aren't always the best thing to compare. Two microphones in the same room hear the same speech with different tone, level and reverb. Many systems therefore correlate a simplified feature instead, such as an onset envelope, a signal that rises wherever the sound gets suddenly louder (the start of a syllable, a laugh, a door). Onsets line up across very different microphones, and they are cheap to compute at a low sample rate, so a search over several minutes of possible offset stays fast. The height of the winning peak relative to the rest of the correlation gives a natural confidence score.

Clock Drift

Finding one offset assumes the devices' clocks run at exactly the same rate. They don't. Every recorder's sample clock is derived from its own oscillator, and no two oscillators are identical. The differences are tiny but they accumulate: if two clocks differ by 50 parts per million, the recordings drift 180 milliseconds apart over an hour, enough for lips and voice to visibly part. The fix is to measure the offset in many short windows across the recording, fit a line through those offsets, and treat its slope as the drift; the later recording is then resampled or played at a slightly adjusted rate. Precise sync also has to work below the frame: rounding each recording's start to the nearest frame can leave voices more than 15 milliseconds apart at 30 frames per second, which is audible when two people's tracks are mixed.

The Double Peak in Call Recordings

Remote recordings add a twist. On a video call, each participant often records their own microphone locally for quality, a common setup for remote podcasts and interviews in the creator economy. If they aren't wearing headphones, each local recording also contains a faint copy of the other person's voice, played from their speakers, and that copy arrived over the network, a little late. Correlating two participants' local tracks then produces two peaks rather than one: one where speaker A's voice matches its delayed copy in B's track, and one where B's voice matches its delayed copy in A's. The peaks sit on either side of the true alignment, each offset by the network delay, so the correct answer is their midpoint, and the distance between them measures the delay. A naive tool that picks the higher peak is wrong by the latency of the call. When both participants wore headphones there is no shared sound at all, and sync has to fall back on the platform's own mixed recording, file start times, or a clap.

Why It Matters for AI Editing

Automated conversation editing, from transcript-based editing to agentic video editing, in which a system cuts between speakers, sets each person's level (with ducking for music underneath) and builds split-screen layouts, depends on all of this. A cut that switches to the speaker's camera on the first syllable of their answer is only right if the tracks were aligned to within a few milliseconds. Once recordings share a clock, speaker diarization and speech recognition can say who spoke when across every track, and the edit can place J- and L-cuts in the pauses. The alignment itself should be stored as data, as in video as code, with the method and confidence that produced it, so a person can check or nudge it, and so it survives into timeline interchange when the cut moves to a finishing editor.