05 — Sound, in sync

The last rung: add audio, and keep the picture in step with it. dasVideo streams the decoded MP2 audio into dasAudio and then paces the video to the audio, because the audio device is the one clock you can’t speed up or slow down. The video follows it.

(That clip is Emma — her voice decoded and played by dasVideo, her lips paced to it. If it’s in sync, the model works.)

Audio is opt-in

After opening, call video_enable_audio (it returns false if there’s no audio stream — the video-only paths never pay for audio). Then create a PCM stream and feed it the borrowed samples; append_to_pcm takes ownership, so clone the borrow first:

        // Keep the PCM channel fed: a dasAudio PCM stream dies on underrun, so the
        // queue must never empty. When the audio stream ends we rewind and feed from the
        // top right away; the loop's audio plays after everything fed so far, so anchoring
        // the video at `total_fed` keeps picture and sound in sync across the boundary.
        if (!muted && !ended) {
            var que = read_que(status_box)
            while (que < AUDIO_TARGET_CHUNKS) {
                if (!video_decode_audio(player)) {
                    if (cfg.noloop) {
                        ended = true
                    } else {
                        video_rewind(player)
                        loop_offset = double(total_fed) / rate
                        shown_pts = -frame_dur
                    }
                    break
                }
                let count = video_audio_count(player)
                var chunk : array<float>
                // append_to_pcm takes ownership, so clone the borrowed batch first.
                player |> get_audio_data() $(s : array<float>#) {
                    chunk := s
                }
                append_to_pcm(sid, chunk)
                total_fed += uint64(count)
                que++
            }
        }

The master clock is the stream’s consumed_position — the count of PCM frames the device has actually played. Divide by the sample rate and you have media-time; show the video frame whose timestamp is due. A muted clip falls back to wall-clock.

The looping subtlety

A dasAudio PCM stream finishes on underrun — if its queue ever empties, the channel dies and further append_to_pcm does nothing. So to loop you must never let it drain: when the audio stream ends, rewind and keep feeding from the top, and anchor the video at the cumulative samples fed (total_fed) so picture and sound restart together. That’s the whole reason the loop is shaped the way it is.

Self-verifying

A headless test confirms the audio decode + borrow path (no audio device needed — the real playback is what you watch above):

require video
require daslib/defer
require math

[export]
def main() {
    var p = video_open("tutorials/05_audio_sync/sample.mpg")
    verify(p != null, "the sample opens")
    defer() { video_close(p) }
    verify(video_has_audio(p), "the sample has an audio track")
    verify(video_enable_audio(p), "enabling audio succeeds")
    verify(video_audio_channels(p) == 2, "pl_mpeg audio is interleaved stereo")
    var batches = 0
    var peak = 0.0
    while (video_decode_audio(p)) {
        let count = video_audio_count(p)
        verify(count == 1152, "each MP2 batch is 1152 stereo frames")
        p |> get_audio_data() $(s : array<float>#) {
            verify(length(s) == count * 2, "borrowed buffer is count*2 interleaved floats")
            for (v in s) {
                peak = max(peak, abs(v))
            }
        }
        batches++
    }
    verify(batches >= 60, "decodes the whole ~2s audio track")
    verify(peak > 0.1, "decoded audio is real (non-trivial amplitude)")
    print("tutorial 05 ok: {batches} audio batches, peak {peak}\n")
}

Run it

cd <dasVideo>
daslang -load_module . examples/play_audio_gl.das

With no arguments it plays Emma introducing dasVideo, with sound, A/V synced — the out-of-the-box demo. That’s the end of the ladder: open → decode → borrow → screen → GPU → sound.