Carriage of AV1 in MPEG-2 TS

1.1 Modal verbs terminology

In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).

1.2 Definition of mnemonics and syntax function

In the present document the mnemonics, the syntax functions, and the syntax descriptors are to be interpreted as described in [ MPEG-2 TS ]. The uimsbf and bslbf mnemonics are defined in Section 2.2.6 of [ MPEG-2 TS ]. The nextbits() function is interpreted as in [ MPEG-2 TS ].

3.1 General constraints

For AV1 video streams, the following constraints apply:

• An AV1 video stream conforming to a profile defined in Annex A of [ AV1 ] shall be an element of an MPEG-2 program and the stream_type for this elementary stream shall be equal to 0x06 (MPEG-2 PES packets containing private data).
• An AV1 video stream shall have the low overhead byte stream format as defined in [ AV1 ].
• The sequence_header_obu as specified in [ AV1 ], that are necessary for decoding an AV1 video stream shall be present within the elementary stream carrying that AV1 video stream.
• An OBU may contain the obu_size field. For applications that need easy conversion to MP4, using the obu_size field is recommended.
• OBU trailing bits should be limited to byte alignment and should not be used for padding.
• Tile List OBUs shall not be used
• Temporal Delimiters may be removed
• Redundant Frame Headers and Padding OBUs may be used.

In addition, a start code insertion and emulation prevention process shall be performed on the AV1 Bitstream prior to its PES encapsulation. This process is described in section 3.2. §  3.2 Start-code based format .

3.2 Start-code based format

Prior to carriage into PES, the AV1 open_bitstream_unit() open_bitstream_unit is encapsulated into ts_open_bitstream_unit() . ts_open_bitstream_unit . This is required to provide direct access to OBU through a start-code mechanism inserted prior to each OBU. The following syntax describes how to retrieve the open_bitstream_unit() open_bitstream_unit from the ts_open_bitstream_unit() (tsOBU). ts_open_bitstream_unit .

obu_start_code - This value shall be set to 0x000001. 0x000001 .

open_bitstream_unit[i] - i-th byte of the AV1 open bitstream unit (As defined in section 5.3 of [ AV1 ]).

It is the responsability of the TS muxer to prevent start code emulation by escaping all the forbidden three-byte sequences using the emulation_prevention_three_byte (always equal to 0x03). 0x03 ). The forbidden sequences are defined below.

Within the ts_open_bitstream_unit() ts_open_bitstream_unit payload, the following three-byte sequences shall not occur at any byte-aligned position :

• 0x000000
• 0x000001
• 0x000002

Within the ts_open_bitstream_unit() ts_open_bitstream_unit payload, any four-byte sequence that starts with 0x000003 other than the following sequences shall not occur at any byte-aligned position :

• 0x00000300
• 0x00000301
• 0x00000302
• 0x00000303

3.3 The AV1 Access Unit

An AV1 Access Unit consists of all OBUs, OBUs , including headers, between the end of the last OBU associated with the previous frame, and the end of the last OBU associated with the current frame. With this definition, an Access Unit sometimes maps with a Decodable Frame Group (DFG) as defined in Annex E of [ AV1 ] and some other times to a Temporal Unit (TU) as defined in [ AV1 ], or both, as illustrated in the figure below. An illustration is provided in the figure below for a group of pictures with frames predicted as follows :

3.4 Use of PES packets

AV1 video encapsulated as defined in clause 4.2 §  3.2 Start-code based format is carried in PES packets as PES_packet_data_bytes, PES_packet_data_bytes , using the stream_id 0xBD (private_stream_id_1).

A PES shall encapsulate one, and only one, AV1 access unit as defined in clause 4.3. . All the PES shall have data_alignment_indicator set to 1. 1 . Usage of data_stream_alignment_descriptor is not specified and the only allowed alignment_type is 1 (Access unit level).

The highest level that may occur in an AV1 video stream, as well as a profile and tier that the entire stream conforms to, shall be signalled using the AV1 video descriptor. AV1_video_descriptor .

If an AV1 video descriptor is associated with an AV1 video stream, then this descriptor shall be conveyed in the descriptor loop for the respective elementary stream entry in the program map table. This specification does not specify the presentation of AV1 streams in the context of a program stream.

3.5 Assignment of DTS and PTS

For AV1 video stream multiplexed into [ MPEG-2 TS ], the decoder_model_info may not be present. If the decoder_model_info is present, then the STD model shall match with the decoder model defined in Annex E of [ AV1 ].

For synchronization and STD management, PTSs and, when appropriate, DTSs are encoded in the header of the PES packet that carries the AV1 video stream data setting the PTS_DTS_flags to '01' 01 or '11'. 11 . For PTS and DTS encoding, the constraints and semantics apply as defined in the PES Header and associated constraints on timestamp intervals.

There are cases in AV1 bitstreams where information about a frame is sent multiple times. For example, first to be decoded, and subsequently to be displayed. In the case of a frame being decoded but not displayed, it is desired to assign a valid DTS but without need for a PTS. However, the MPEG2-TS [ MPEG-2 TS ] specification prevents a DTS from being transmitted without a PTS. Hence, a PTS is always assigned for AV1 access units and its value is not relevant for frames being decoded but not displayed.

To achieve consistency between the STD model and the buffer model defined in Annex E of [ AV1 ], the following PTS and DTS assignment rules shall be applied :

Note : The ScheduleRemovalTiming[] ScheduledRemovalTiming and PresentationTime[] PresentationTime are defined in the Annex E of [ AV1 ].

3.6 Buffer considerations

3.6.2 T-STD Extensions for AV1

When there is an AV1 video stream in an [ MPEG-2 TS ] program, the T-STD model as described in the section "Transport stream system target decoder" is extended as specified below.

3.6.2.1 TB _n , MB _n , EB _n buffer management

The following additional notations are used to describe the T-STD extensions and are illustrated in the figure above.

Notation	Definition
t(i)	indicates the time in seconds at which the i-th byte of the transport stream enters the system target decoder
TB n	is the transport buffer for elementary stream n
TBS	is the size of the transport buffer TBn, measured in bytes
MB n	is the multiplexing buffer for elementary stream n
MBS n	is the size of the multiplexing buffer MBn, measured in bytes
EB n	is the elementary stream buffer for the AV1 video stream
EBS n	is the size of the multiplexing buffer MBn, measured in bytes
j	is an index to the AV1 access unit of the AV1 video stream
A n (j)	is the j-th access unit of the AV1 video bitstream
td n (j)	is the decoding time of An(j), measured in seconds, in the system target decoder
Rx n	is the transfer rate from the transport buffer TBn to the multiplex buffer MBn as specified below.
Rbx n	is the transfer rate from the multiplex buffer MBn to the elementary stream buffer EBn as specified below

The following apply:

• There is exactly one transport buffer TB _n for the received AV1 video stream where the size TBS is fixed to 512 bytes.
• There is exactly one multiplexing buffer MB _n for the AV1 video stream, where the size MBS _n of the multiplexing buffer MB is constrained as follows: MBS _n = BS _mux + BS _oh + 0.1 x BufferSize where BS _oh , packet overhead buffering, is defined as: BS _oh = (1/750) seconds × max{ 1100 × BitRate, 2 000 000 bit/s} and BS _mux , additional mutliplex buffering, is defined as: BS _mux = 0.004 seconds ×max{ 1100 × BitRate, 2 000 000 bit/s} BufferSize and BitRate are defined in Annex E of the [ AV1 ]
• There is exactly one elementary stream buffer EB _n for all the elementary streams in the set of received elementary streams associated by hierarchy descriptors, with a total size EBS _n : EBS _n = BufferSize
• Transfer from TB _n to MB _n is applied as follows: When there is no data in TB _n then Rx _n is equal to zero. Otherwise: Rx _n = 1.1 x BitRate
• The leak method shall be used to transfer data from MB _n to EB _n as follows: Rbx _n = 1.1 × BitRate
• The removal of start-code and emulation prevention as defined in section 4.2 is instantaneously performed between MB _n and EB _n .

If there is PES packet payload data in MB _n , and buffer EB _n is not full, the PES packet payload is transferred from MB _n to EB _n at a rate equal to Rbx _n . If EB _n is full, data are not removed from MB _n . When a byte of data is transferred from MB _n to EB _n , all PES packet header bytes that are in MB _n and precede that byte are instantaneously removed and discarded. When there is no PES packet payload data present in MB _n , no data is removed from MB _n . All data that enters MB _n leaves it. All PES packet payload data bytes enter EB _n instantaneously upon leaving MB _n .

3.6.2.2 STD delay

The STD delay of any AV1 video through the system target decoders buffers TB _n , MB _n , and EB _n shall be constrained by td _n (j) – t(i) ≤ 10 seconds for all j, and all bytes i in access unit A _n (j).

3.6.2.3 Buffer management conditions

Transport streams shall be constructed so that the following conditions for buffer management are satisfied:

• Each TB _n shall not overflow and shall be empty at least once every second.
• Each MB _n , EB _n and Buffer Pool shall not overflow.
• EB _n shall not underflow, except when the Operating parameters info syntax has low_delay_mode_flag set to '1'. Underflow of EB _n occurs for AV1 access unit A _n (j) when one or more bytes of A _n (j) are not present in EB _n at the decoding time td _n (j).