Short Version: If you haven’t already heard, not all cameras that shoot video (for example, DSLRs like the Canon 5D mkII) record their own audio in sync with the video. This causes problems whether you’re using camera audio, dual system audio, or have a multi-camera shoot. Therefore, it’s essential to test your camera (particularly if it’s not primarily a “real” video camera) and to compensate for this potential issue. If this is news to you (or if you’ve been ignoring it), read on.
The Long Version:
We use a Canon 5D mkII for on-camera interviews. We’re not pleased with its built-in audio (even when using a high quality mic and preamp), so we’ve been recording “dual system” sound where the mic feeds a separate audio recorder. We then synchronize this recorder’s capture with the camera’s audio during editing by aligning the audio waveforms between the two. Auto-sync software performs a similar task.
To make synchronization easy, we use the old-fashioned clapper approach, which creates a nice spike in the audio waveform as the clapper arm hits the board. Failing a clapper, we just do a series of hand claps on camera before starting to talk.
The image below shows such an alignment: The camera’s audio is on top; the audio recorder is below. No, the spikes are not perfectly aligned, but they’re within one frame of video (indicated by the lighter gray box extending to the right of the red line of the current time indicator). This is an important side issue: Audio events happen a lot faster than video events, making perfect sync difficult to ascertain; it’s good to have multiple events to look at, and then average them out. (We also temporarily set the timeline to a higher frame rate to get more temporal resolution when sliding the audio tracks.)
The problem was that even after carefully aligning waveforms, the audio and video did not appear to be in sync during playback – the lip movements we saw didn’t line up perfectly with what our ears heard. This caused us to go back and study exactly what our 5D was doing.
The image below is four consecutive video frames (at 30 fps) of Chris clapping at the head of an introduction to one of our training videos:
We then looked at the audio waveform for the 5D’s internal audio for this sequence. The work area bar above the audio waveform below represents the same four frames as pictured above; the current time indicator is parked at the third frame of the above sequence:
Clearly, the 5D records its own audio roughly one frame earlier than its own video. If we’re aligning our clean audio to the camera’s onboard audio, there’s no way we can be in sync, as the camera itself isn’t in sync! The same will be true for the results of auto-sync software.
Now that we know that there’s a problem, we know to compensate for it each time we use the 5D where audio/video sync is important. Once you’ve done some tests with your own cameras, you should be able to build a table of time offsets to always apply – whether you are running dual system audio, editing a multicam shoot, or even using the camera’s output as is, and whether you’re syncing waveforms visually or with software.
Sure, some of you know this already. But a quick Google search turned up enough articles and videos by People Who Should Know Better that didn’t mention this problem, that we thought it was worth sharing.
(By the way, if you’re relying on the camera’s audio to perform synchronization, also keep the speed of sound in mind: It travels at roughly 340 meters/second, which works out to ~11 meters per frame at 29.97 fps. If the camera is, say, 70 feet from the stage, the sound recorded by the camera is going to be about 2 frames late compared to the video – and that needs to be taken into account. Indeed, Canon engineers may very well have built the one frame advance into the 5D assuming it would be some distance away from the action, and the audio it captured would otherwise be delayed in relationship to the video.)