Instead of specifying the sample rate, channel count/map, etc. to the
PlaybackStream, we'll now use the output device's sample specification
whenever possible. If necessary, the stream will fall back to sane
default.
This hugely simplifies AudioMixingSink, since it no longer has to take
care of reinitializing the stream with a new sample specification when
it encounters a track with a higher sample rate or more channels. We
wouldn't be likely to benefit from this anyway, since it turns out that
at least Windows's virtual surround doesn't work through WASAPI at all,
and WASAPI likely wouldn't support downmixing.
This commit breaks playback of audio files that don't match the system
default audio device's sample rate and channel count. The next commit
introduces a converter into the pipeline to allow mixing of any sample
specification.
We were already assuming that our streams were using floats, we may as
well hardcode this. If we ever encounter a platform API that doesn't
support or convert from float, we can always bring this back.
Also, since we don't support big-endian systems, remove that check in
PulseAudioWrappers.
Without removing the current block, we end up playing no audio until
its timestamp is reached, after we shift the next block forward if its
timestamp is less than the previous block's end time. This would mean
that re-enabling an audio track after a seek backwards would result in
a delay in its audio resuming.
This prevents PlaybackManager's seek while enabling an audio track from
causing the AudioMixingSink to push audio blocks forward unnecessarily.
Previously, the seek would cause the initial block or blocks to repeat
from the perspective of AudioMixingSink, so it would think that it
needs to shift the first block after the seek forward by a few samples.
By moving this to the AudioDataProvider, we can clear the last sample
index every time the decoder is flushed, ensuring that the block
shifting always makes sense.
By doing this in AudioMixingSink instead of the Decoder
implementations, we avoid having to duplicate this shifting logic
across multiple implementations.
This also fixes an issue where multiple audio blocks occupying the same
timestamp would be skipped while seeking, causing a significant break
in audio.
With the previous setup setting the time directly on the main thread,
the following could occur:
- HTMLMediaElement temporarily pauses playback to begin a seek.
- AudioMixingSink starts an audio task to drain audio and suspend.
- HTMLMediaElement starts PlaybackManager seeking to a new position.
- AudioDataProvider completes the seek to the new position.
- The PlaybackManager tells AudioMixingSink to set the media time,
which it does so synchronously on the main thread.
- At this point, the time provider corresponds to the new position.
- The pause completes, and a deferred invocation sets media time to
its old position again.
This would result in the timeline showing the wrong position after a
seek on rare occasions.
Instead, always queue up a drain and suspend when setting the sink's
media time. This ensures that updating the time always occurs after the
pause has completed.
Also, since setting the time is asynchronous now, we need to store the
target time until the seeking drain completes. Otherwise, we still
briefly see the previous playback position after a seek.
This commit implements the functionality to play back audio through
PlaybackManager.
To decode the audio data, AudioDataProviders are created for each track
in the provided media data. These providers will fill their audio block
queue, then sit idle until their corresponding tracks are enabled.
In order to output the audio, one AudioMixingSink is created which
manages a PlaybackStream which requests audio blocks from multiple
AudioDataProviders and mixes them into one buffer with sample-perfect
precision.
