Media Source Extensions™

2.5.1 Attaching to a media element

A MediaSource object can be attached to a media element by assigning a MediaSource object URL to the media element src attribute or the src attribute of a <source> inside a media element. A MediaSource object URL is created by passing a MediaSource object to createObjectURL () .

If the resource fetch algorithm was invoked with a media provider object that is a MediaSource object or a URL record whose object is a MediaSource object, then let mode be local, skip the first step in the resource fetch algorithm (which may otherwise set mode to remote) and add the steps and clarifications below to the " Otherwise (mode is local) " section of the resource fetch algorithm .

If readyState is NOT set to " closed "

Run the " If the media data cannot be fetched at all, due to network errors, causing the user agent to give up trying to fetch the resource " steps of the resource fetch algorithm 's media data processing steps list .

Otherwise

1. Set the media element's delaying-the-load-event-flag to false.

2. If the MediaSource was constructed in a DedicatedWorkerGlobalScope , then setup worker attachment communication and open the MediaSource :

   1. Set [[channel with worker]] to be a new MessageChannel .
   2. Set [[port to worker]] to the port1 value of [[channel with worker]] .
   3. Execute StructuredSerializeWithTransfer with the port2 of [[channel with worker]] as both the value and the sole member of the transferList , and let the result be serialized port2 .
   4. Queue a task on the MediaSource 's DedicatedWorkerGlobalScope that will
      1. Execute StructuredDeserializeWithTransfer with serialized port2 and DedicatedWorkerGlobalScope 's realm , and set [[port to main]] to be the resulting deserialized clone of the transferred port2 value of [[channel with worker]] .
      2. Set the readyState attribute to " open ".
      3. Queue a task to fire an event named sourceopen at the MediaSource .

   Otherwise, the MediaSource was constructed in a Window :

   1. Set [[channel with worker]] null.
   2. Set [[port to worker]] null.
   3. Set [[port to main]] null.
   4. Set the readyState attribute to " open ".
   5. Queue a task to fire an event named sourceopen at the MediaSource .

3. Continue the resource fetch algorithm by running the remaining " Otherwise (mode is local) " steps, with these clarifications:
   1. Text in the resource fetch algorithm or the media data processing steps list that refers to "the download", "bytes received", or "whenever new data for the current media resource becomes available" refers to data passed in via appendBuffer () or appendEncodedChunks () .
   2. References to HTTP in the resource fetch algorithm and the media data processing steps list do not apply because the HTMLMediaElement does not fetch media data via HTTP when a MediaSource is attached.

2.5.2 Detaching from a media element

The following steps are run in any case where the media element is going to transition to NETWORK_EMPTY and queue a task to fire an event named emptied at the media element. These steps SHOULD be run right before the transition.

1. If the MediaSource was constructed in a DedicatedWorkerGlobalScope :

   1. Notify the MediaSource using an internal detach message posted to [[port to worker]] .
   2. Set [[port to worker]] null.
   3. Set [[channel with worker]] null.
   4. The implicit message handler for this detach notification runs the remainder of these steps in the DedicatedWorkerGlobalScope MediaSource .

   Otherwise, the MediaSource was constructed in a Window :

   Continue the remainder of these steps on the Window MediaSource .

2. Set [[port to main]] null.
3. Set the readyState attribute to " closed ".
4. Update duration to NaN.
5. Remove all the SourceBuffer objects from activeSourceBuffers .
6. Queue a task to fire an event named removesourcebuffer at activeSourceBuffers .
7. Remove all the SourceBuffer objects from sourceBuffers .
8. Queue a task to fire an event named removesourcebuffer at sourceBuffers .
9. Queue a task to fire an event named sourceclose at the MediaSource .

2.5.3 Seeking

Run the following steps as part of the " Wait until the user agent has established whether or not the media data for the new playback position is available, and, if it is, until it has decoded enough data to play back that position" step of the seek algorithm :

1. Note

   The media element looks for media segments containing the new playback position in each SourceBuffer object in activeSourceBuffers . Any position within a TimeRanges in the current value of the HTMLMediaElement . buffered attribute has all necessary media segments buffered for that position.

   If new playback position is not in any TimeRanges of HTMLMediaElement . buffered

   1. If the HTMLMediaElement . readyState attribute is greater than HAVE_METADATA , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA .
      Note

      Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

   2. The media element waits until an appendBuffer () or appendEncodedChunks () call causes the coded frame processing algorithm to set the HTMLMediaElement . readyState attribute to a value greater than HAVE_METADATA .
      Note

      The web application can use buffered and HTMLMediaElement . buffered to determine what the media element needs to resume playback.

   Otherwise

   Continue

   Note

   If the readyState attribute is " ended " and the new playback position is within a TimeRanges currently in HTMLMediaElement . buffered , then the seek operation must continue to completion here even if one or more currently selected or enabled track buffers' largest range end timestamp is less than new playback position . This condition should only occur due to logic in buffered when readyState is " ended ".

2. The media element resets all decoders and initializes each one with data from the appropriate initialization segment .
3. The media element feeds coded frames from the active track buffers into the decoders starting with the closest random access point before the new playback position .
4. Resume the seek algorithm at the " Await a stable state " step.

2.5.4 SourceBuffer Monitoring

The following steps are periodically run during playback to make sure that all of the SourceBuffer objects in activeSourceBuffers have enough data to ensure uninterrupted playback . Changes to activeSourceBuffers also cause these steps to run because they affect the conditions that trigger state transitions.

Having enough data to ensure uninterrupted playback is an implementation specific condition where the user agent determines that it currently has enough data to play the presentation without stalling for a meaningful period of time. This condition is constantly evaluated to determine when to transition the media element into and out of the HAVE_ENOUGH_DATA ready state. These transitions indicate when the user agent believes it has enough data buffered or it needs more data respectively.

If the HTMLMediaElement . readyState attribute equals HAVE_NOTHING :

1. Abort these steps.

If HTMLMediaElement . buffered does not contain a TimeRanges for the current playback position:

1. Set the HTMLMediaElement . readyState attribute to HAVE_METADATA .
   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

2. Abort these steps.

If HTMLMediaElement . buffered contains a TimeRanges that includes the current playback position and enough data to ensure uninterrupted playback :

1. Set the HTMLMediaElement . readyState attribute to HAVE_ENOUGH_DATA .
   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

2. Playback may resume at this point if it was previously suspended by a transition to HAVE_CURRENT_DATA .
3. Abort these steps.

If HTMLMediaElement . buffered contains a TimeRanges that includes the current playback position and some time beyond the current playback position, then run the following steps:

1. Set the HTMLMediaElement . readyState attribute to HAVE_FUTURE_DATA .
   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

2. Playback may resume at this point if it was previously suspended by a transition to HAVE_CURRENT_DATA .
3. Abort these steps.

If HTMLMediaElement . buffered contains a TimeRanges that ends at the current playback position and does not have a range covering the time immediately after the current position:

1. Set the HTMLMediaElement . readyState attribute to HAVE_CURRENT_DATA .
   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

2. Playback is suspended at this point since the media element doesn't have enough data to advance the media timeline .
3. Abort these steps.

2.5.5 Changes to selected/enabled track state

During playback activeSourceBuffers needs to be updated if the selected video track, the enabled audio track(s), or a text track mode changes. When one or more of these changes occur the following steps need to be followed. Also, when MediaSource was constructed in a DedicatedWorkerGlobalScope , then each change that occurs to a Window mirror of a track created previously by the implicit handler for the internal create track mirror message MUST also be made to the corresponding DedicatedWorkerGlobalScope track using an internal update track state message posted to [[port to worker]] whose implicit handler makes the change and runs the following steps. Likewise, each change that occurs to a DedicatedWorkerGlobalScope track MUST also be made to the corresponding Window mirror of the track using an internal update track state message posted to [[port to main]] whose implicit handler makes the change to the mirror.

If the selected video track changes, then run the following steps:

1. If the SourceBuffer associated with the previously selected video track is not associated with any other enabled tracks, run the following steps:
   1. Remove the SourceBuffer from activeSourceBuffers .
   2. Queue a task to fire an event named removesourcebuffer at activeSourceBuffers
2. If the SourceBuffer associated with the newly selected video track is not already in activeSourceBuffers , run the following steps:
   1. Add the SourceBuffer to activeSourceBuffers .
   2. Queue a task to fire an event named addsourcebuffer at activeSourceBuffers

If an audio track becomes disabled and the SourceBuffer associated with this track is not associated with any other enabled or selected track, then run the following steps:

1. Remove the SourceBuffer associated with the audio track from activeSourceBuffers
2. Queue a task to fire an event named removesourcebuffer at activeSourceBuffers

If an audio track becomes enabled and the SourceBuffer associated with this track is not already in activeSourceBuffers , then run the following steps:

1. Add the SourceBuffer associated with the audio track to activeSourceBuffers
2. Queue a task to fire an event named addsourcebuffer at activeSourceBuffers

If a text track mode becomes "disabled" and the SourceBuffer associated with this track is not associated with any other enabled or selected track, then run the following steps:

1. Remove the SourceBuffer associated with the text track from activeSourceBuffers
2. Queue a task to fire an event named removesourcebuffer at activeSourceBuffers

If a text track mode becomes "showing" or "hidden" and the SourceBuffer associated with this track is not already in activeSourceBuffers , then run the following steps:

1. Add the SourceBuffer associated with the text track to activeSourceBuffers
2. Queue a task to fire an event named addsourcebuffer at activeSourceBuffers

2.5.6 Duration change

Follow these steps when duration needs to change to a new duration .

1. If the current value of duration is equal to new duration , then return.
2. If new duration is less than the highest presentation timestamp of any buffered coded frames for all SourceBuffer objects in sourceBuffers , then throw an InvalidStateError exception and abort these steps.
   Note

   Duration reductions that would truncate currently buffered media are disallowed. When truncation is necessary, use remove () to reduce the buffered range before updating duration .

3. Let highest end time be the largest track buffer ranges end time across all the track buffers across all SourceBuffer objects in sourceBuffers .
4. If new duration is less than highest end time , then
   Note

   This condition can occur because the coded frame removal algorithm preserves coded frames that start before the start of the removal range.

   1. Update new duration to equal highest end time .
5. Update duration to new duration .
6. Update the duration to new duration .

7. If the MediaSource was constructed in a DedicatedWorkerGlobalScope :

   Post an internal duration change message to [[port to main]] whose implicit handler in Window runs the HTMLMediaElement duration change algorithm .

   Otherwise:

   Run the HTMLMediaElement duration change algorithm .

2.5.7 End of stream

This algorithm gets called when the application signals the end of stream via an endOfStream () call or an algorithm needs to signal a decode error. This algorithm takes an error parameter that indicates whether an error will be signalled.

1. Change the readyState attribute value to " ended ".
2. Queue a task to fire an event named sourceended at the MediaSource .

3. If error is not set

   1. Run the duration change algorithm with new duration set to the largest track buffer ranges end time across all the track buffers across all SourceBuffer objects in sourceBuffers .
      Note

      This allows the duration to properly reflect the end of the appended media segments. For example, if the duration was explicitly set to 10 seconds and only media segments for 0 to 5 seconds were appended before endOfStream() was called, then the duration will get updated to 5 seconds.

   2. Notify the media element that it now has all of the media data.

   If error is set to " network "

   If the MediaSource was constructed in a DedicatedWorkerGlobalScope :

   Post an internal network error message to [[port to main]] whose implicit handler in Window runs the steps in the following "otherwise" case.

   Otherwise:

   If the HTMLMediaElement . readyState attribute equals HAVE_NOTHING

   Run the " If the media data cannot be fetched at all, due to network errors, causing the user agent to give up trying to fetch the resource " steps of the resource fetch algorithm 's media data processing steps list .

   If the HTMLMediaElement . readyState attribute is greater than HAVE_NOTHING

   Run the " If the connection is interrupted after some media data has been received, causing the user agent to give up trying to fetch the resource " steps of the resource fetch algorithm 's media data processing steps list .

   If error is set to " decode "

   If the MediaSource was constructed in a DedicatedWorkerGlobalScope :

   Post an internal decode error message to [[port to main]] whose implicit handler in Window runs the steps in the following "otherwise" case.

   Otherwise:

   If the HTMLMediaElement . readyState attribute equals HAVE_NOTHING

   Run the " If the media data can be fetched but is found by inspection to be in an unsupported format, or can otherwise not be rendered at all " steps of the resource fetch algorithm 's media data processing steps list .

   If the HTMLMediaElement . readyState attribute is greater than HAVE_NOTHING

   Run the media data is corrupted steps of the resource fetch algorithm 's media data processing steps list .

3.5.1 Segment Parser Loop

Each SourceBuffer object has an [[append state]] internal slot that keeps track of the high-level segment parsing state. It is initially set to WAITING_FOR_SEGMENT and can transition to the following states as data is appended.

Each SourceBuffer object has an [[input buffer]] internal slot that is a byte buffer that holds unparsed bytes across appendBuffer () calls. The buffer is empty when the SourceBuffer object is created.

Each SourceBuffer object has a [[buffer full flag]] internal slot that keeps track of whether appendBuffer () is allowed to accept more bytes. bytes or appendEncodedChunks () is allowed to accept more encoded chunks. It is set to false when the SourceBuffer object is created and gets updated as data is appended and removed.

Each SourceBuffer object has a [[group start timestamp]] internal slot that keeps track of the starting timestamp for a new coded frame group in the " sequence " mode. It is unset when the SourceBuffer object is created and gets updated when the mode attribute equals " sequence " and the timestampOffset attribute is set, or the coded frame processing algorithm runs.

Each SourceBuffer object has a [[group end timestamp]] internal slot that stores the highest coded frame end timestamp across all coded frames in the current coded frame group . It is set to 0 when the SourceBuffer object is created and gets updated by the coded frame processing algorithm.

Each SourceBuffer object has a [[generate timestamps flag]] internal slot that is a boolean that keeps track of whether timestamps need to be generated for the coded frames passed to the coded frame processing algorithm. This flag is set by addSourceBuffer () when the SourceBuffer object is created and is updated by changeType () .

When the segment parser loop algorithm is invoked, run the following steps:

1. Loop Top: If the [[input buffer]] is empty, then jump to the need more data step below.
2. If the [[input buffer]] contains bytes that violate the SourceBuffer byte stream format specification , then run the append error algorithm and abort this algorithm.
3. Remove any bytes that the byte stream format specifications say MUST be ignored from the start of the [[input buffer]] .

4. If the [[append state]] equals WAITING_FOR_SEGMENT , then run the following steps:

   1. If the beginning of the [[input buffer]] indicates the start of an initialization segment , set the [[append state]] to PARSING_INIT_SEGMENT .
   2. If the beginning of the [[input buffer]] indicates the start of a media segment , set [[append state]] to PARSING_MEDIA_SEGMENT .
   3. Jump to the loop top step above.

5. If the [[append state]] equals PARSING_INIT_SEGMENT , then run the following steps:

   1. If the [[input buffer]] does not contain a complete initialization segment yet, then jump to the need more data step below.
   2. Run the initialization segment received algorithm.
   3. Remove the initialization segment bytes from the beginning of the [[input buffer]] .
   4. Set [[append state]] to WAITING_FOR_SEGMENT .
   5. Jump to the loop top step above.

6. If the [[append state]] equals PARSING_MEDIA_SEGMENT , then run the following steps:

   1. If the [[first initialization segment received flag]] is false or the [[pending initialization segment for changeType flag]] is true, then run the append error algorithm and abort this algorithm.
   2. If the [[input buffer]] contains one or more complete coded frames , then run the coded frame processing algorithm.
      Note

      The frequency at which the coded frame processing algorithm is run is implementation-specific. The coded frame processing algorithm MAY be called when the input buffer contains the complete media segment or it MAY be called multiple times as complete coded frames are added to the input buffer.

   3. If this SourceBuffer is full and cannot accept more media data, then set the [[buffer full flag]] to true.
   4. If the [[input buffer]] does not contain a complete media segment , then jump to the need more data step below.
   5. Remove the media segment bytes from the beginning of the [[input buffer]] .
   6. Set [[append state]] to WAITING_FOR_SEGMENT .
   7. Jump to the loop top step above.
7. Need more data: Return control to the calling algorithm.

3.5.2 Reset Parser State

When the parser state needs to be reset, run the following steps:

1. If the [[append state]] equals PARSING_MEDIA_SEGMENT and the [[input buffer]] contains some complete coded frames , then run the coded frame processing algorithm until all of these complete coded frames have been processed.
2. Unset the last decode timestamp on all track buffers .
3. Unset the last frame duration on all track buffers .
4. Unset the highest end timestamp on all track buffers .
5. Set the need random access point flag on all track buffers to true.
6. If the mode attribute equals " sequence ", then set the [[group start timestamp]] to the [[group end timestamp]]
7. Remove all bytes from the [[input buffer]] .
8. If [[input webcodecs configs and chunks]] contains any EncodedChunks , remove them from [[input webcodecs configs and chunks]] , but retain any AudioDecoderConfig or VideoDecoderConfig that may be in that internal slot.
   Note

   Keeping an unprocessed WebCodecs config in this internal slot lets future appendEncodedChunks () reuse the config in the chunks append algorithm because there is no other way to retain such an unprocessed config, except perhaps by the app calling changeType () with the config again.
9. Set [[append state]] to WAITING_FOR_SEGMENT .

3.5.3 Append Error

This algorithm is called when an error occurs during an append.

1. Run the reset parser state algorithm.
2. Set the updating attribute to false.

3. If this SourceBuffer has a Promise in [[pending append chunks promise]] :

   Reject that Promise with an AbortError DOMException and unset [[pending append chunks promise]] .

   Otherwise:

   1. Queue a task to fire an event named error at this SourceBuffer object.
   2. Queue a task to fire an event named updateend at this SourceBuffer object.

4. Run the end of stream algorithm with the error parameter set to " decode ".

3.5.4 Prepare Append

When an append operation begins, the following steps are run to validate and prepare the SourceBuffer .

1. If the SourceBuffer has been removed from the sourceBuffers attribute of the parent media source then throw an InvalidStateError exception and abort these steps.
2. If the updating attribute equals true, then throw an InvalidStateError exception and abort these steps.
3. Let recent element error be determined as follows:

   If the MediaSource was constructed in a Window

   Let recent element error be true if the HTMLMediaElement . error attribute is not null. If that attribute is null, then let recent element error be false.

   Otherwise

   Let recent element error be the value resulting from the steps for the Window case, but run on the Window HTMLMediaElement on any change to its error attribute and communicated by using [[port to worker]] implicit messages. If such a message has not yet been received, then let recent element error be false.

4. If recent element error is true, then throw an InvalidStateError exception and abort these steps.

5. If the readyState attribute of the parent media source is in the " ended " state then run the following steps:

   1. Set the readyState attribute of the parent media source to " open "
   2. Queue a task to fire an event named sourceopen at the parent media source .
6. Run the coded frame eviction algorithm.

7. If the [[buffer full flag]] equals true, then throw a QuotaExceededError exception and abort these steps.

   Note

   This is the signal that the implementation was unable to evict enough data to accommodate the append or the append is too big. The web application SHOULD use remove () to explicitly free up space and/or reduce the size of the append.

3.5.5 Buffer Append

When appendBuffer () is called, the following steps are run to process the appended data.

1. If this SourceBuffer is currently configured to expect processing of WebCodecs EncodedChunks , then run the append error algorithm and abort this algorithm.
   Note

   The expectation of processing an appended bytestream versus appended EncodedChunks is based on the most recently successful execution of the initial addSourceBuffer () that created this SourceBuffer or potentially a more recent changeType () call that may have changed the expectation.
2. Run the segment parser loop algorithm.
3. If the segment parser loop algorithm in the previous step was aborted, then abort this algorithm.
4. Set the updating attribute to false.
5. Queue a task to fire an event named update at this SourceBuffer object.
6. Queue a task to fire an event named updateend at this SourceBuffer object.

3.5.6 Chunks Append

Each SourceBuffer object has a [[pending append chunks promise]] internal slot that stores the promise necessary to communicate the completion of asynchronous steps begun by a call to appendEncodedChunks () . If there is no asynchronous appendEncodedChunks () operation in progress, then this slot is unset.

Each SourceBuffer object has an [[input webcodecs configs and chunks]] internal slot that stores a queue of unprocessed EncodedChunks and at most one SourceBufferConfig . The SourceBufferConfig is added if the initial call to addSourceBuffer () that created this object was provided a SourceBufferConfig , or if there is a more recent successful call to changeType () on this SourceBuffer that provided a SourceBufferConfig . The contents of this slot are processed by the chunks append algorithm to buffer the necessary decoder configuration and coded frames into the underlying track buffer .

When the chunks append algorithm is invoked, run the following steps to process the appended EncodedChunks relative to the current SourceBufferConfig :

1. If this SourceBuffer is currently configured to expect processing of an appended bytestream, then run the append error algorithm and abort this algorithm.
   Note

   The expectation of processing an appended bytestream versus appended EncodedChunks is based on the most recently successful execution of the initial addSourceBuffer () that created this SourceBuffer or potentially a more recent changeType () call that may have changed the expectation.
2. If there is a SourceBufferConfig in this SourceBuffer 's [[input webcodecs configs and chunks]] , then run the following steps:
   1. Let config be the SourceBufferConfig from [[input webcodecs configs and chunks]] .
      Note

      There must be precisely one AudioDecoderConfig or VideoDecoderConfig in config ; this condition was already enforced by the addSourceBuffer () or changeType () call that placed config into [[input webcodecs configs and chunks]] .
   2. Remove the SourceBufferConfig from [[input webcodecs configs and chunks]] .
   3. Run the initialization segment received algorithm with config denoting a single track configuration. If that algorithm aborts, then abort this algorithm.
3. In the order in which they were added to [[input webcodecs configs and chunks]] , prepare the EncodedChunks for coded frame processing by running the following steps for each chunk:
   1. Let chunk be the next encoded chunk being prepared, dequeued (removed) from the front of [[input webcodecs configs and chunks]] .
   2. If chunk is an EncodedAudioChunk but the most recently processed config by this object was a VideoDecoderConfig , or if chunk is an EncodedVideoChunk but the most recently processed config by this object was an AudioDecoderConfig , then run the append error algorithm and abort this algorithm.
   3. Create a coded frame from chunk . Treat the frame as a random access point iff chunk 's type is "key". Set the frame's decode timestamp to be 0.
      Note

      Using a constant decode timestamp disables the coded frame processing algorithm's discontinuity detection. Note that other mechanisms for signalling discontinuous sequences of frames survive this simplification, with the most reliable yet lightweight being calling abort () when updating is false. A more heavyweight option would be to call changeType () . Also note that, though WebCodecs does not define a decode timestamp attribute for encoded chunks, if reliable buffering of chunks into MSE needs real decode timestamps, this spec may be refined to improve that support.
4. Run the coded frame processing algorithm with the frames in order from the previous step.
   Note

   This algorithm, unlike the segment parser loop algorithm, deterministically consumes and clears all of its input state and processes all corresponding coded frames. Even the steps which might abort this algorithm early do so after the append error algorithm clears any remaining input state and rejects the promise.
5. Set the updating attribute to false.
6. Resolve the [[pending append chunks promise]] and unset that internal slot.

3.5.7 Range Removal

Follow these steps when a caller needs to initiate a JavaScript visible range removal operation that blocks other SourceBuffer updates:

1. Let start equal the starting presentation timestamp for the removal range, in seconds measured from presentation start time .
2. Let end equal the end presentation timestamp for the removal range, in seconds measured from presentation start time .
3. Set the updating attribute to true.
4. Queue a task to fire an event named updatestart at this SourceBuffer object.
5. Return control to the caller and run the rest of the steps asynchronously.
6. Run the coded frame removal algorithm with start and end as the start and end of the removal range.
7. Set the updating attribute to false.
8. Queue a task to fire an event named update at this SourceBuffer object.
9. Queue a task to fire an event named updateend at this SourceBuffer object.

3.5.7 3.5.8 Initialization Segment Received

The following steps are run when the segment parser loop successfully parses a complete initialization segment or the chunks append algorithm handles a SourceBufferConfig :

Each SourceBuffer object has a [[first initialization segment received flag]] internal slot that tracks whether the first initialization segment has been appended and received by this algorithm. This flag is set to false when the SourceBuffer is created and updated by the algorithm below.

Each SourceBuffer object has a [[pending initialization segment for changeType flag]] internal slot that tracks whether an initialization segment is needed since the most recent changeType () . This flag is set to false when the SourceBuffer is created, set to true by changeType () and reset to false by the algorithm below.

1. Update the duration attribute if it currently equals NaN:

   If the initialization segment contains a duration:

   Run the duration change algorithm with new duration set to the duration in the initialization segment.

   Otherwise:

   Run the duration change algorithm with new duration set to positive Infinity.

2. If the initialization segment has no audio, video, or text tracks, then run the append error algorithm and abort these steps.
3. If the [[first initialization segment received flag]] is true, then run the following steps:
   1. Verify the following properties. If any of the checks fail then run the append error algorithm and abort these steps.
      • The number of audio, video, and text tracks match what was in the first initialization segment .
      • If more than one track for a single type are present (e.g., 2 audio tracks), then the Track IDs match the ones in the first initialization segment .
      • The codecs for each track are supported by the user agent.
        Note

        User agents MAY consider codecs, that would otherwise be supported, as "not supported" here if the codecs were not specified in type parameter passed to (a) the most recently successful changeType () on this SourceBuffer object, or (b) if no successful changeType () has yet occurred on this object, the addSourceBuffer () that created this SourceBuffer object. For example, if the most recently successful changeType () was called with 'video/webm' or 'video/webm; codecs="vp8"' , and a video track containing vp9 appears in the initialization segment, then the user agent MAY use this step to trigger a decode error even if the other two properties' checks, above, pass. Implementations are encouraged to trigger error in such cases only when the codec is indeed not supported or the other two properties' checks fail. Web authors are encouraged to use changeType () , addSourceBuffer () and isTypeSupported () with precise codec parameters to more proactively detect user agent support. changeType () is required if the SourceBuffer object's bytestream format is changing.

   2. Add the appropriate track descriptions from this initialization segment to each of the track buffers .
   3. Set the need random access point flag on all track buffers to true.
4. Let active track flag equal false.

5. If the [[first initialization segment received flag]] is false, then run the following steps:

   1. If the initialization segment contains tracks with codecs the user agent does not support, then run the append error algorithm and abort these steps.
      Note

      User agents MAY consider codecs, that would otherwise be supported, as "not supported" here if the codecs were not specified in type parameter passed to (a) the most recently successful changeType () on this SourceBuffer object, or (b) if no successful changeType () has yet occurred on this object, the addSourceBuffer () that created this SourceBuffer object. For example, MediaSource.isTypeSupported('video/webm;codecs="vp8,vorbis"') may return true, but if addSourceBuffer () was called with 'video/webm;codecs="vp8"' and a Vorbis track appears in the initialization segment , then the user agent MAY use this step to trigger a decode error. Implementations are encouraged to trigger error in such cases only when the codec is indeed not supported. Web authors are encouraged to use changeType () , addSourceBuffer () and isTypeSupported () with precise codec parameters to more proactively detect user agent support. changeType () is required if the SourceBuffer object's bytestream format is changing.

   2. For each audio track in the initialization segment , run following steps:

      1. Let audio byte stream track ID be the Track ID for the current track being processed.
      2. Let audio language be a BCP 47 language tag for the language specified in the initialization segment for this track or an empty string if no language info is present.
      3. If audio language equals the 'und' BCP 47 value, then assign an empty string to audio language .
      4. Let audio label be a label specified in the initialization segment for this track or an empty string if no label info is present.
      5. Let audio kinds be a sequence of kind strings specified in the initialization segment for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in audio kinds , run the following steps:
         1. Let current audio kind equal the value from audio kinds for this iteration of the loop.
         2. Let new audio track be a new AudioTrack object.
         3. Generate a unique ID and assign it to the id property on new audio track .
         4. Assign audio language to the language property on new audio track .
         5. Assign audio label to the label property on new audio track .
         6. Assign current audio kind to the kind property on new audio track .

         7. If this SourceBuffer object's audioTracks . length equals 0, then run the following steps:

            1. Set the enabled property on new audio track to true.
            2. Set active track flag to true.
         8. Add new audio track to the audioTracks attribute on this SourceBuffer object.
            Note

            This should trigger AudioTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to new audio track , at the AudioTrackList object referenced by the audioTracks attribute on this SourceBuffer object.

         9. If the parent media source was constructed in a DedicatedWorkerGlobalScope :

            Post an internal create track mirror message to [[port to main]] whose implicit handler in Window runs the following steps:

            1. Let mirrored audio track be a new AudioTrack object.
            2. Assign the same property values to mirrored audio track as were determined for new audio track .
            3. Add mirrored audio track to the audioTracks attribute on the HTMLMediaElement.

            Otherwise:

            Add new audio track to the audioTracks attribute on the HTMLMediaElement.

            Note

            This should trigger AudioTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to mirrored audio track or new audio track , at the AudioTrackList object referenced by the audioTracks attribute on the HTMLMediaElement.

      7. Create a new track buffer to store coded frames for this track.
      8. Add the track description for this track to the track buffer .

   3. For each video track in the initialization segment , run following steps:

      1. Let video byte stream track ID be the Track ID for the current track being processed.
      2. Let video language be a BCP 47 language tag for the language specified in the initialization segment for this track or an empty string if no language info is present.
      3. If video language equals the 'und' BCP 47 value, then assign an empty string to video language .
      4. Let video label be a label specified in the initialization segment for this track or an empty string if no label info is present.
      5. Let video kinds be a sequence of kind strings specified in the initialization segment for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in video kinds , run the following steps:
         1. Let current video kind equal the value from video kinds for this iteration of the loop.
         2. Let new video track be a new VideoTrack object.
         3. Generate a unique ID and assign it to the id property on new video track .
         4. Assign video language to the language property on new video track .
         5. Assign video label to the label property on new video track .
         6. Assign current video kind to the kind property on new video track .

         7. If this SourceBuffer object's videoTracks . length equals 0, then run the following steps:

            1. Set the selected property on new video track to true.
            2. Set active track flag to true.
         8. Add new video track to the videoTracks attribute on this SourceBuffer object.
            Note

            This should trigger VideoTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to new video track , at the VideoTrackList object referenced by the videoTracks attribute on this SourceBuffer object.

         9. If the parent media source was constructed in a DedicatedWorkerGlobalScope :

            Post an internal create track mirror message to [[port to main]] whose implicit handler in Window runs the following steps:

            1. Let mirrored video track be a new VideoTrack object.
            2. Assign the same property values to mirrored video track as were determined for new video track .
            3. Add mirrored video track to the videoTracks attribute on the HTMLMediaElement.

            Otherwise:

            Add new video track to the videoTracks attribute on the HTMLMediaElement.

            Note

            This should trigger VideoTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to mirrored video track or new video track , at the VideoTrackList object referenced by the videoTracks attribute on the HTMLMediaElement.

      7. Create a new track buffer to store coded frames for this track.
      8. Add the track description for this track to the track buffer .

   4. For each text track in the initialization segment , run following steps:

      1. Let text byte stream track ID be the Track ID for the current track being processed.
      2. Let text language be a BCP 47 language tag for the language specified in the initialization segment for this track or an empty string if no language info is present.
      3. If text language equals the 'und' BCP 47 value, then assign an empty string to text language .
      4. Let text label be a label specified in the initialization segment for this track or an empty string if no label info is present.
      5. Let text kinds be a sequence of kind strings specified in the initialization segment for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in text kinds , run the following steps:
         1. Let current text kind equal the value from text kinds for this iteration of the loop.
         2. Let new text track be a new TextTrack object.
         3. Generate a unique ID and assign it to the id property on new text track .
         4. Assign text language to the language property on new text track .
         5. Assign text label to the label property on new text track .
         6. Assign current text kind to the kind property on new text track .
         7. Populate the remaining properties on new text track with the appropriate information from the initialization segment .
         8. If the mode property on new text track equals "showing" or "hidden" , then set active track flag to true.
         9. Add new text track to the textTracks attribute on this SourceBuffer object.
            Note

            This should trigger TextTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to new text track , at the TextTrackList object referenced by the textTracks attribute on this SourceBuffer object.

         10. If the parent media source was constructed in a DedicatedWorkerGlobalScope :

             Post an internal create track mirror message to [[port to main]] whose implicit handler in Window runs the following steps:

             1. Let mirrored text track be a new TextTrack object.
             2. Assign the same property values to mirrored text track as were determined for new text track .
             3. Add mirrored text track to the textTracks attribute on the HTMLMediaElement.

             Otherwise:

             Add new text track to the textTracks attribute on the HTMLMediaElement.

             Note

             This should trigger TextTrackList [ HTML ] logic to queue a task to fire an event named addtrack using TrackEvent with the track attribute initialized to mirrored text track or new text track , at the TextTrackList object referenced by the textTracks attribute on the HTMLMediaElement.

      7. Create a new track buffer to store coded frames for this track.
      8. Add the track description for this track to the track buffer .
   5. If active track flag equals true, then run the following steps:
      1. Add this SourceBuffer to activeSourceBuffers .
      2. Queue a task to fire an event named addsourcebuffer at activeSourceBuffers
   6. Set [[first initialization segment received flag]] to true.
6. Set [[pending initialization segment for changeType flag]] to false.
7. If the active track flag equals true, then run the following steps:

   If the parent media source was constructed in a DedicatedWorkerGlobalScope :

   Post an internal active track added message to [[port to main]] whose implicit handler in Window runs the following step:

   1. If the HTMLMediaElement . readyState attribute is greater than HAVE_CURRENT_DATA , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA .

   Otherwise:

   If the HTMLMediaElement . readyState attribute is greater than HAVE_CURRENT_DATA , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA .

   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

8. If each object in sourceBuffers of the parent media source has [[first initialization segment received flag]] equal to true, then run the following steps:

   If the parent media source was constructed in a DedicatedWorkerGlobalScope :

   Post an internal sourcebuffers ready message to [[port to main]] whose implicit handler in Window runs the following step:

   1. If the HTMLMediaElement . readyState attribute is HAVE_NOTHING , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA .

   Otherwise:

   If the HTMLMediaElement . readyState attribute is HAVE_NOTHING , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA .

   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement. If transition from HAVE_NOTHING to HAVE_METADATA occurs, it should trigger HTMLMediaElement logic to queue a task to fire an event named loadedmetadata at the media element.

3.5.8 3.5.9 Coded Frame Processing

When complete coded frames have been parsed by the segment parser loop or emitted by the chunks append algorithm, then the following steps are run:

1. For each coded frame in the media segment run the following steps:

   1. Loop Top:

      If [[generate timestamps flag]] equals true:

      1. Let presentation timestamp equal 0.
      2. Let decode timestamp equal 0.

      Otherwise:

      1. Let presentation timestamp be a double precision floating point representation of the coded frame's presentation timestamp in seconds.
         Note

         Special processing may be needed to determine the presentation and decode timestamps for timed text frames since this information may not be explicitly present in the underlying format or may be dependent on the order of the frames. Some metadata text tracks, like MPEG2-TS PSI data, may only have implied timestamps. Format specific rules for these situations SHOULD be in the byte stream format specifications or in separate extension specifications.

      2. Let decode timestamp be a double precision floating point representation of the coded frame's decode timestamp in seconds.
         Note

         Implementations don't have to internally store timestamps in a double precision floating point representation. This representation is used here because it is the representation for timestamps in the HTML spec. The intention here is to make the behavior clear without adding unnecessary complexity to the algorithm to deal with the fact that adding a timestampOffset may cause a timestamp rollover in the underlying timestamp representation used by the byte stream format. Implementations can use any internal timestamp representation they wish, but the addition of timestampOffset SHOULD behave in a similar manner to what would happen if a double precision floating point representation was used.

   2. Let frame duration be a double precision floating point representation of the coded frame's duration in seconds.
   3. If mode equals " sequence " and [[group start timestamp]] is set, then run the following steps:
      1. Set timestampOffset equal to [[group start timestamp]] minus presentation timestamp .
      2. Set [[group end timestamp]] equal to [[group start timestamp]] .
      3. Set the need random access point flag on all track buffers to true.
      4. Unset [[group start timestamp]] .

   4. If timestampOffset is not 0, then run the following steps:

      1. Add timestampOffset to the presentation timestamp .
      2. Add timestampOffset to the decode timestamp .
   5. Let track buffer equal the track buffer that the coded frame will be added to.

   6. If last decode timestamp for track buffer is set and decode timestamp is less than last decode timestamp :

      OR

      If last decode timestamp for track buffer is set and the difference between decode timestamp and last decode timestamp is greater than 2 times last frame duration :

      1. If mode equals " segments ":

         Set [[group end timestamp]] to presentation timestamp .

         If mode equals " sequence ":

         Set [[group start timestamp]] equal to the [[group end timestamp]] .

      2. Unset the last decode timestamp on all track buffers .
      3. Unset the last frame duration on all track buffers .
      4. Unset the highest end timestamp on all track buffers .
      5. Set the need random access point flag on all track buffers to true.
      6. Jump to the Loop Top step above to restart processing of the current coded frame .

      Otherwise:

      Continue.

   7. Let frame end timestamp equal the sum of presentation timestamp and frame duration .
   8. If presentation timestamp is less than appendWindowStart , then set the need random access point flag to true, drop the coded frame, and jump to the top of the loop to start processing the next coded frame.
      Note

      Some implementations MAY choose to collect some of these coded frames with presentation timestamp less than appendWindowStart and use them to generate a splice at the first coded frame that has a presentation timestamp greater than or equal to appendWindowStart even if that frame is not a random access point . Supporting this requires multiple decoders or faster than real-time decoding so for now this behavior will not be a normative requirement.

   9. If frame end timestamp is greater than appendWindowEnd , then set the need random access point flag to true, drop the coded frame, and jump to the top of the loop to start processing the next coded frame.
      Note

      Some implementations MAY choose to collect coded frames with presentation timestamp less than appendWindowEnd and frame end timestamp greater than appendWindowEnd and use them to generate a splice across the portion of the collected coded frames within the append window at time of collection, and the beginning portion of later processed frames which only partially overlap the end of the collected coded frames. Supporting this requires multiple decoders or faster than real-time decoding so for now this behavior will not be a normative requirement. In conjunction with collecting coded frames that span appendWindowStart , implementations MAY thus support gapless audio splicing.

   10. If the need random access point flag on track buffer equals true, then run the following steps:
       1. If the coded frame is not a random access point , then drop the coded frame and jump to the top of the loop to start processing the next coded frame.
       2. Set the need random access point flag on track buffer to false.
   11. Let spliced audio frame be an unset variable for holding audio splice information
   12. Let spliced timed text frame be an unset variable for holding timed text splice information
   13. If last decode timestamp for track buffer is unset and presentation timestamp falls within the presentation interval of a coded frame in track buffer , then run the following steps:
       1. Let overlapped frame be the coded frame in track buffer that matches the condition above.

       2. If track buffer contains audio coded frames :

          Run the audio splice frame algorithm and if a splice frame is returned, assign it to spliced audio frame .

          If track buffer contains video coded frames :

          1. Let remove window timestamp equal the overlapped frame presentation timestamp plus 1 microsecond.
          2. If the presentation timestamp is less than the remove window timestamp , then remove overlapped frame from track buffer .
             Note

             This is to compensate for minor errors in frame timestamp computations that can appear when converting back and forth between double precision floating point numbers and rationals. This tolerance allows a frame to replace an existing one as long as it is within 1 microsecond of the existing frame's start time. Frames that come slightly before an existing frame are handled by the removal step below.

          If track buffer contains timed text coded frames :

          Run the text splice frame algorithm and if a splice frame is returned, assign it to spliced timed text frame .

   14. Remove existing coded frames in track buffer :

       If highest end timestamp for track buffer is not set:

       Remove all coded frames from track buffer that have a presentation timestamp greater than or equal to presentation timestamp and less than frame end timestamp .

       If highest end timestamp for track buffer is set and less than or equal to presentation timestamp :

       Remove all coded frames from track buffer that have a presentation timestamp greater than or equal to highest end timestamp and less than frame end timestamp .

   15. Remove all possible decoding dependencies on the coded frames removed in the previous two steps by removing all coded frames from track buffer between those frames removed in the previous two steps and the next random access point after those removed frames.
       Note

       Removing all coded frames until the next random access point is a conservative estimate of the decoding dependencies since it assumes all frames between the removed frames and the next random access point depended on the frames that were removed.

   16. If spliced audio frame is set:

       Add spliced audio frame to the track buffer .

       If spliced timed text frame is set:

       Add spliced timed text frame to the track buffer .

       Otherwise:

       Add the coded frame with the presentation timestamp , decode timestamp , and frame duration to the track buffer .

   17. Set last decode timestamp for track buffer to decode timestamp .
   18. Set last frame duration for track buffer to frame duration .
   19. If highest end timestamp for track buffer is unset or frame end timestamp is greater than highest end timestamp , then set highest end timestamp for track buffer to frame end timestamp .
       Note

       The greater than check is needed because bidirectional prediction between coded frames can cause presentation timestamp to not be monotonically increasing even though the decode timestamps are monotonically increasing.

   20. If frame end timestamp is greater than [[group end timestamp]] , then set [[group end timestamp]] equal to frame end timestamp .
   21. If [[generate timestamps flag]] equals true, then set timestampOffset equal to frame end timestamp .

2. If the HTMLMediaElement . readyState attribute is HAVE_METADATA and the new coded frames cause HTMLMediaElement . buffered to have a TimeRanges for the current playback position, then set the HTMLMediaElement . readyState attribute to HAVE_CURRENT_DATA .

   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

3. If the HTMLMediaElement . readyState attribute is HAVE_CURRENT_DATA and the new coded frames cause HTMLMediaElement . buffered to have a TimeRanges that includes the current playback position and some time beyond the current playback position, then set the HTMLMediaElement . readyState attribute to HAVE_FUTURE_DATA .

   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

4. If the HTMLMediaElement . readyState attribute is HAVE_FUTURE_DATA and the new coded frames cause HTMLMediaElement . buffered to have a TimeRanges that includes the current playback position and enough data to ensure uninterrupted playback , then set the HTMLMediaElement . readyState attribute to HAVE_ENOUGH_DATA .

   Note

   Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

5. If the media segment contains result of coded frame processing buffers data beyond the current duration , then run the duration change algorithm with new duration set to the maximum of the current duration and the [[group end timestamp]] .

3.5.9 3.5.10 Coded Frame Removal

Follow these steps when coded frames for a specific time range need to be removed from the SourceBuffer:

1. Let start be the starting presentation timestamp for the removal range.
2. Let end be the end presentation timestamp for the removal range.

3. For each track buffer in this SourceBuffer , run the following steps:

   1. Let remove end timestamp be the current value of duration

   2. If this track buffer has a random access point timestamp that is greater than or equal to end , then update remove end timestamp to that random access point timestamp.

      Note

      Random access point timestamps can be different across tracks because the dependencies between coded frames within a track are usually different than the dependencies in another track.

   3. Remove all media data, from this track buffer , that contain starting timestamps greater than or equal to start and less than the remove end timestamp .

      1. For each removed frame, if the frame has a decode timestamp equal to the last decode timestamp for the frame's track, run the following steps:

         If mode equals " segments ":

         Set [[group end timestamp]] to presentation timestamp .

         If mode equals " sequence ":

         Set [[group start timestamp]] equal to the [[group end timestamp]] .

      2. Unset the last decode timestamp on all track buffers .
      3. Unset the last frame duration on all track buffers .
      4. Unset the highest end timestamp on all track buffers .
      5. Set the need random access point flag on all track buffers to true.
   4. Remove all possible decoding dependencies on the coded frames removed in the previous step by removing all coded frames from this track buffer between those frames removed in the previous step and the next random access point after those removed frames.
      Note

      Removing all coded frames until the next random access point is a conservative estimate of the decoding dependencies since it assumes all frames between the removed frames and the next random access point depended on the frames that were removed.

   5. If this object is in activeSourceBuffers , the current playback position is greater than or equal to start and less than the remove end timestamp , and HTMLMediaElement . readyState is greater than HAVE_METADATA , then set the HTMLMediaElement . readyState attribute to HAVE_METADATA and stall playback.

      Note

      Per HTMLMediaElement ready states [ HTML ] logic, HTMLMediaElement . readyState changes may trigger events on the HTMLMediaElement.

      Note

      This transition occurs because media data for the current position has been removed. Playback cannot progress until media for the current playback position is appended or the selected/enabled tracks change .

4. If the [[buffer full flag]] equals true and this object is ready to accept more bytes, then set the [[buffer full flag]] to false.

3.5.10 3.5.11 Coded Frame Eviction

This algorithm is run to free up space in this SourceBuffer when new data is appended. New data is either the bytes being appended via appendBuffer () or the WebCodecs EncodedChunks being appended via appendEncodedChunks () .

1. Let new data equal the data that is about to be appended to this SourceBuffer.
   Issue 289 : Editorial? Coded Frame eviction algorithm needs to note that "buffer full flag" may be updated immediately based on |new data|

   Need to recognize step here that implementations MAY decide to set [[buffer full flag]] true here if it predicts that processing new data in addition to any existing bytes in [[input buffer]] would exceed the capacity of the SourceBuffer . Such a step enables more proactive push-back from implementations before accepting new data which would overflow resources, for example. In practice, at least one implementation already does this.

2. If the [[buffer full flag]] equals false, then abort these steps.
3. Let removal ranges equal a list of presentation time ranges that can be evicted from the presentation to make room for the new data .
   Note

   Implementations MAY use different methods for selecting removal ranges so web applications SHOULD NOT depend on a specific behavior. The web application can use the buffered attribute to observe whether portions of the buffered data have been evicted.

4. For each range in removal ranges , run the coded frame removal algorithm with start and end equal to the removal range start and end timestamp respectively.

3.5.11 3.5.12 Audio Splice Frame

Follow these steps when the coded frame processing algorithm needs to generate a splice frame for two overlapping audio coded frames :

1. Let track buffer be the track buffer that will contain the splice.
2. Let new coded frame be the new coded frame , that is being added to track buffer , which triggered the need for a splice.
3. Let presentation timestamp be the presentation timestamp for new coded frame .
4. Let decode timestamp be the decode timestamp for new coded frame .
5. Let frame duration be the coded frame duration of new coded frame .
6. Let overlapped frame be the coded frame in track buffer with a presentation interval that contains presentation timestamp .
7. Update presentation timestamp and decode timestamp to the nearest audio sample timestamp based on sample rate of the audio in overlapped frame . If a timestamp is equidistant from both audio sample timestamps, then use the higher timestamp (e.g., floor(x * sample_rate + 0.5) / sample_rate ).
   Note

   For example, given the following values:

   • The presentation timestamp of overlapped frame equals 10.
   • The sample rate of overlapped frame equals 8000 Hz
   • presentation timestamp equals 10.01255
   • decode timestamp equals 10.01255

   presentation timestamp and decode timestamp are updated to 10.0125 since 10.01255 is closer to 10 + 100/8000 (10.0125) than 10 + 101/8000 (10.012625)

8. If the user agent does not support crossfading then run the following steps:
   1. Remove overlapped frame from track buffer .
   2. Add a silence frame to track buffer with the following properties:
      • The presentation timestamp set to the overlapped frame presentation timestamp .
      • The decode timestamp set to the overlapped frame decode timestamp .
      • The coded frame duration set to difference between presentation timestamp and the overlapped frame presentation timestamp .
      Note

      Some implementations MAY apply fades to/from silence to coded frames on either side of the inserted silence to make the transition less jarring.

   3. Return to caller without providing a splice frame.
      Note

      This is intended to allow new coded frame to be added to the track buffer as if overlapped frame had not been in the track buffer to begin with.

9. Let frame end timestamp equal the sum of presentation timestamp and frame duration .
10. Let splice end timestamp equal the sum of presentation timestamp and the splice duration of 5 milliseconds.
11. Let fade out coded frames equal overlapped frame as well as any additional frames in track buffer that have a presentation timestamp greater than presentation timestamp and less than splice end timestamp .
12. Remove all the frames included in fade out coded frames from track buffer .
13. Return a splice frame with the following properties:
    • The presentation timestamp set to the overlapped frame presentation timestamp .
    • The decode timestamp set to the overlapped frame decode timestamp .
    • The coded frame duration set to difference between frame end timestamp and the overlapped frame presentation timestamp .
    • The fade out coded frames equals fade out coded frames .
    • The fade in coded frame equals new coded frame .
      Note

      If the new coded frame is less than 5 milliseconds in duration, then coded frames that are appended after the new coded frame will be needed to properly render the splice.

    • The splice timestamp equals presentation timestamp .
    Note

    See the audio splice rendering algorithm for details on how this splice frame is rendered.

3.5.12 3.5.13 Audio Splice Rendering

The following steps are run when a spliced frame, generated by the audio splice frame algorithm, needs to be rendered by the media element:

1. Let fade out coded frames be the coded frames that are faded out during the splice.
2. Let fade in coded frames be the coded frames that are faded in during the splice.
3. Let presentation timestamp be the presentation timestamp of the first coded frame in fade out coded frames .
4. Let end timestamp be the sum of the presentation timestamp and the coded frame duration of the last frame in fade in coded frames .
5. Let splice timestamp be the presentation timestamp where the splice starts. This corresponds with the presentation timestamp of the first frame in fade in coded frames .
6. Let splice end timestamp equal splice timestamp plus five milliseconds.
7. Let fade out samples be the samples generated by decoding fade out coded frames .
8. Trim fade out samples so that it only contains samples between presentation timestamp and splice end timestamp .
9. Let fade in samples be the samples generated by decoding fade in coded frames .
10. If fade out samples and fade in samples do not have a common sample rate and channel layout, then convert fade out samples and fade in samples to a common sample rate and channel layout.
11. Let output samples be a buffer to hold the output samples.
12. Apply a linear gain fade out with a starting gain of 1 and an ending gain of 0 to the samples between splice timestamp and splice end timestamp in fade out samples .
13. Apply a linear gain fade in with a starting gain of 0 and an ending gain of 1 to the samples between splice timestamp and splice end timestamp in fade in samples .
14. Copy samples between presentation timestamp to splice timestamp from fade out samples into output samples .
15. For each sample between splice timestamp and splice end timestamp , compute the sum of a sample from fade out samples and the corresponding sample in fade in samples and store the result in output samples .
16. Copy samples between splice end timestamp to end timestamp from fade in samples into output samples .
17. Render output samples .

Note

Here is a graphical representation of this algorithm.

3.5.13 3.5.14 Text Splice Frame

Follow these steps when the coded frame processing algorithm needs to generate a splice frame for two overlapping timed text coded frames :

1. Let track buffer be the track buffer that will contain the splice.
2. Let new coded frame be the new coded frame , that is being added to track buffer , which triggered the need for a splice.
3. Let presentation timestamp be the presentation timestamp for new coded frame
4. Let decode timestamp be the decode timestamp for new coded frame .
5. Let frame duration be the coded frame duration of new coded frame .
6. Let frame end timestamp equal the sum of presentation timestamp and frame duration .
7. Let first overlapped frame be the coded frame in track buffer with a presentation interval that contains presentation timestamp .
8. Let overlapped presentation timestamp be the presentation timestamp of the first overlapped frame .
9. Let overlapped frames equal first overlapped frame as well as any additional frames in track buffer that have a presentation timestamp greater than presentation timestamp and less than frame end timestamp .
10. Remove all the frames included in overlapped frames from track buffer .
11. Update the coded frame duration of the first overlapped frame to presentation timestamp minus overlapped presentation timestamp .
12. Add first overlapped frame to the track buffer .
13. Return to caller without providing a splice frame.
    Note

    This is intended to allow new coded frame to be added to the track buffer as if it hadn't overlapped any frames in track buffer to begin with.