WebRTC Extensions

Append the following paragraph to the administratively prohibited algorithm:

If should RTC connections be blocked for global? with the relevant global object of the RTCPeerConnection object in question returns "Blocked" , then all candidates MUST be administratively prohibited .

The RTCRtpTransceiver interface is defined in [ WEBRTC ]. This document extends that interface by adding an additional method and attribute in order to control negotiation of RTP header extensions.

WebIDLpartial dictionary RTCRtpHeaderExtensionCapability {
  required RTCRtpTransceiverDirection direction;
};
partial interface RTCRtpTransceiver {
  sequence<RTCRtpHeaderExtensionCapability> getHeaderExtensionsToNegotiate();
  undefined setHeaderExtensionsToNegotiate(
      sequence<RTCRtpHeaderExtensionCapability> extensions);
  sequence<RTCRtpHeaderExtensionCapability> getNegotiatedHeaderExtensions();
};

Let [[HeaderExtensionsToNegotiate]] and [[NegotiatedHeaderExtensions]] be internal slots of the RTCRtpTransceiver , initialized as follows:

1. Set [[HeaderExtensionsToNegotiate]] to the platform-specific list of implemented RTP header extensions. The direction attribute for all extensions that are mandatory to use MUST be initialized to an appropriate value other than " stopped ". The direction attribute for extensions that will not be offered by default in an initial offer MUST be initialized to " stopped ".

   Note

   The list of header extensions that MUST / SHOULD be supported is listed in [ RTCWEB-RTP ], section 5.2. The "mid" extension is mandatory to use when BUNDLE is in use, per [ BUNDLE ] section 9.1.

2. Set [[NegotiatedHeaderExtensions]] to an empty list.

Make the following modifications to the set a session description algorithm:

• In the step for processing a description of type " answer " that runs for both local and remote descriptions, insert the following steps:

  1. For each media description and corresponding transceiver , let negotiatedExtensions be the value of transceiver . [[HeaderExtensionsToNegotiate]] .

  2. For each extension in negotiatedExtensions , run the following steps:

     1. Let answeredDirection be the direction attribute of the "a=extmap" line in the answer's media description that correspond to this extension if one exists, otherwise set answeredDirection to " stopped ".

     2. If description is a remote description, reverse answeredDirection to represent the peer's point of view.

     3. Set extension . direction to answeredDirection .

  3. Set transceiver . [[NegotiatedHeaderExtensions]] to negotiatedExtensions .

In the algorithms for generating initial offers in [ RTCWEB-JSEP ] section 5.2.1, replace "for each supported RTP header extension, an "a=extmap" line, as specified in [ RFC5285 ], section 5" " with "For each RTP header extension "e" listed in [[HeaderExtensionsToNegotiate]] where direction is not " stopped ", an "a=extmap" line, as specified in [ RFC5285 ], section 5, with direction taken from "e"'s direction attribute."

In the algorithm for generating subsequent offers in [ RTCWEB-JSEP ] section 5.2.2, replace "The RTP header extensions MUST only include those that are present in the most recent answer" with "For each RTP header extension listed in [[HeaderExtensionsToNegotiate]] , and where direction is not " stopped ", generate an appropriate "a=extmap" line with "direction" set according to the rules of [ RFC5285 ] section 6, considering the direction in [[HeaderExtensionsToNegotiate]] to indicate the answerer's desired usage".

In the algorithm for generating initial answers in [ RTCWEB-JSEP ] section 5.3.1, replace "For each supported RTP header extension that is present in the offer" with "For each supported RTP header extension that is present in the offer and is also present in [[HeaderExtensionsToNegotiate]] with a direction different from " stopped ", set the appropriate direction based on direction that does not exceed the direction in the offer".

getHeaderExtensionsToNegotiate

Execute the following steps:

1. Let transceiver be the RTCRtpTransceiver that this method was invoked on.

2. Return transceiver . [[HeaderExtensionsToNegotiate]] .

setHeaderExtensionsToNegotiate

Execute the following steps:

1. Let transceiver be the RTCRtpTransceiver that this method was invoked on.

2. Let extensions be the first argument of this method.

3. If the size of extensions does not match the size of transceiver . [[HeaderExtensionsToNegotiate]] throw an InvalidModificationError .

4. For each index i of extensions , run the following steps:

   1. Let extension be the i -th element of extensions .

   2. If extension . uri is not equal to the uri of the i -th element of transceiver . [[HeaderExtensionsToNegotiate]] , throw an InvalidModificationError .

   3. If extension . direction is not " sendrecv " and uri indicates a mandatory-to-use attribute that is required to be both sent and received, throw an InvalidModificationError .

   4. If extension . direction is " stopped " and uri indicates a mandatory-to-implement extension, throw an InvalidModificationError .

   5. If necessary, restrict extension . direction as to not exceed the user agent's capabilities for this extension.

5. Set transceiver . [[HeaderExtensionsToNegotiate]] to extensions .

getNegotiatedHeaderExtensions

Execute the following steps:

1. Let transceiver be the RTCRtpTransceiver that this method was invoked on.

2. Return transceiver . [[NegotiatedHeaderExtensions]] .

Let RTCRtpReceiver objects have a [[JitterBufferTarget]] internal slot initially initialized to null .

WebIDLpartial interface RTCRtpReceiver {
  attribute DOMHighResTimeStamp? jitterBufferTarget;
};

jitterBufferTarget of type DOMHighResTimeStamp , nullable

This attribute allows the application to specify a target duration of time in milliseconds of media for the RTCRtpReceiver 's jitter buffer to hold. This influences the amount of buffering done by the user agent , which in turn affects retransmissions and packet loss recovery. Altering the target value allows applications to control the tradeoff between playout delay and the risk of running out of audio or video frames due to network jitter.

The user agent MUST have a minimum allowed target and a maximum allowed target reflecting what the user agent is able or willing to provide based on network conditions and memory constraints, which can change at any time.

Note

This is a target value. The resulting change in delay can be gradually observed over time. The receiver's average jitter buffer delay can be measured as the delta jitterBufferDelay divided by the delta jitterBufferEmittedCount .

An average delay is expected even if DTX is used. For example, if DTX is used and packets start flowing after silence, larger targets can influence the user agent to buffer these packets rather than playing them out.

On getting, this attribute MUST return the value of the [[JitterBufferTarget]] internal slot.

On setting, the user agent MUST run the following steps:

1. Let receiver be the RTCRtpReceiver object on which the setter is invoked.

2. Let target be the argument to the setter.

3. If target is negative or larger than 4000 milliseconds, then throw a RangeError .

4. Set receiver 's [[JitterBufferTarget]] to target .

5. Let track be receiver 's [[ReceiverTrack]] .

6. In parallel, begin executing the following steps:

   1. Update the underlying system about the new target , or that there is no application preference if target is null .

      If track is synchronized with another RTCRtpReceiver 's track for audio/video synchronization , then the user agent SHOULD use the larger of the two receivers' [[JitterBufferTarget]] for both receivers.

      When the underlying system is applying a jitter buffer target, it will continuously make sure that the actual jitter buffer target is clamped within the minimum allowed target and maximum allowed target .

      Note

      If the user agent ends up using a target different from the requested one (e.g. due to network conditions or physical memory constraints), this is not reflected in the [[JitterBufferTarget]] internal slot.

   2. Modifying the jitter buffer target of the underlying system SHOULD affect the internal audio or video buffering gradually in order not to hurt user experience. Audio samples or video frames SHOULD be accelerated or decelerated before playout, similarly to how it is done for audio/video synchronization or in response to congestion control.

      The acceleration or deceleration rate may vary depending on network conditions or the type of audio received (e.g. speech or background noise). It MAY take several seconds to achieve 1 second of buffering but SHOULD not take more than 30 seconds assuming packets are being received. The speed MAY be different for audio and video.

      Note

      For audio, acceleration and deceleration can be measured with insertedSamplesForDeceleration and removedSamplesForAcceleration . For video, this may result in the same frame being rendered multiple times or frames may be dropped.

ptime of type unsigned long

The preferred duration of media represented by a packet in milliseconds.

(Feature at Risk) Issue 1

ptime was moved from [ WEBRTC ] to this specification due to lack of support from implementers. It is therefore marked as a feature at risk.

adaptivePtime of type boolean , defaulting to false .

Indicates whether this encoding MAY dynamically change the frame length. If the value is true , the user agent MAY use any valid frame length for any of its frames, and MAY change this at any time. Valid values are multiples of 10ms. If the maxptime attribute (defined in [ RFC4566 ] Section 6) is specified, that maximum applies. If the value is false , the user agent MUST use a fixed frame length.

If adaptivePtime is set to true , ptime MUST NOT be set; otherwise, InvalidModificationError MUST be thrown .

Note

Using a longer frame length reduces the bandwidth consumption due to overhead, but does so at the cost of increased latency. Changing the frame length dynamically allows the user agent to adapt its bandwidth allocation strategy based on the current network conditions.

codec of type RTCRtpCodec

Optional value selecting which codec is used for this encoding's RTP stream. The RTCRtpCodec dictionary is defined in [ WEBRTC ]. If absent , the user agent can chose to use any negotiated codec.

The RTCSetParameterOptions are extended by a sequence of RTCEncodingOptions , one for each encoding.

encodingOptions of type sequence< RTCEncodingOptions > , defaulting to [] .

A sequence containing encoding options for each RTP encoding.

RTCEncodingOptions is the WebRTC equivalent of VideoEncoderEncodeOptions in [ WebCodecs ].

keyFrame of type boolean , defaulting to false .

When set to true, request that RTCRtpSender's encoder generates a keyframe for the encoding. The semantic of this boolean is similar to the RTCP FIR message described in [RFC5104], section 3.5.1.

In the steps to call the setParameters () method, let parameters be the method's first argument and let setParameterOptions be the method's second argument.

Append the following steps after the steps to validate the parameters :

• If the transceiver kind of the associated kind is "audio" , set setParameterOptions.encodingOptions to the empty list.

• If setParameterOptions.encodingOptions is not empty and setParameterOptions.encodingOptions .length is different from parameters .encodings.length , return a promise rejected with a newly created InvalidModificationError .

In the steps to configure the media stack to use parameters , append the following step:

• For each setParameterOptions.encodingOptions indexed by i , if setParameterOptions.encodingOptions [i].keyFrame is set to true , request that the encoder associated with parameters .encodings[i] generates a key frame.

onicecandidatepairadd of type EventHandler

The event type of this event handler is icecandidatepairadd , and is fired as part of the add a candidate pair algorithm.

onicecandidatepairremove of type EventHandler

The event type of this event handler is icecandidatepairremove , and is fired as part of the remove a candidate pair algorithm.

onicecandidatepairnominate of type EventHandler

The event type of this event handler is icecandidatepairnominate , and is fired as part of the nominate a candidate pair algorithm.

selectCandidatePair

The selectCandidatePair method attempts to select a different candidate pair to send data over. If successful, data will be sent on the provided candidate pair. It is meant to be called after the application defers the nomination of a candidate pair by cancelling the icecandidatepairnominate event.

When this method is invoked, the user agent MUST run the following steps:

1. Let connection be the RTCPeerConnection object associated with this .

2. If connection . [[IsClosed]] is true , throw an InvalidStateError .

3. If this . [[ProposalPending]] is true , throw an InvalidStateError .

4. If this . [[IceTransportState]] is either of " new ", " failed " or " closed ", throw an InvalidStateError .

5. Let candidatePair be the method's first argument.

6. If candidatePair does not match any item in this . [[CandidatePairs]] , throw a NotFoundError .

7. Let p be a new promise.

8. In parallel, instruct the ICE agent to use candidatePair to send data.

   1. When the ICE agent has completed selecting candidatePair , queue a task to run the following steps:

      1. Run the change the selected candidate pair and state steps to update this . [[SelectedCandidatePair]] and this . [[IceTransportState]] as necessary and fire any associated events.

      2. Resolve p .

9. Return p .

Note

After changing the selected candidate pair, the controlling ICE agent may attempt to nominate the candidate pair as well to conclude ICE processing. The application may cancel the nomination to allow further changes to the selected candidate pair.

removeCandidatePair

The removeCandidatePair method removes the provided candidate pair. The ICE agent will stop sending and responding to ICE connectivity checks on the removed candidate pair, and it can no longer be used to send data for this transport. This method is meant to be called when the application wants to allow the ICE agent to free candidates that it no longer needs.

When this method is invoked, the user agent MUST run the following steps:

1. Let connection be the RTCPeerConnection object associated with this .

2. If connection . [[IsClosed]] is true , throw an InvalidStateError .

3. If this . [[ProposalPending]] is true , throw an InvalidStateError .

4. If this . [[IceTransportState]] is either of " new ", " failed " or " closed ", throw an InvalidStateError .

5. Let candidatePair be the method's first argument.

6. If candidatePair does not match any item in this . [[CandidatePairs]] , throw a NotFoundError .

7. Remove the item in this . [[CandidatePairs]] that matches candidatePair .

8. Let p be a new promise.

9. In parallel, instruct the ICE agent to remove the candidate pair indicated by candidatePair .

   1. When the ICE agent has completed the removal, queue a task to run the following steps:

      1. Fire an event named icecandidatepairremove at transport , using RTCIceCandidatePairEvent , with the cancelable attribute initialized to false , and the local and remote attributes initialized to the local and remote candidates, respectively, of candidatePair .

      2. Resolve p .

10. Return p .

The icecandidatepairadd and icecandidatepairremove events use the RTCIceCandidatePairEvent interface.

WebIDL[Exposed=Window]
  interface RTCIceCandidatePairEvent : Event {
    constructor(DOMString type, RTCIceCandidatePairEventInit eventInitDict);
    readonly attribute RTCIceCandidate local;
    readonly attribute RTCIceCandidate remote;
};

WebIDLdictionary RTCIceCandidatePairEventInit : EventInit {
  required RTCIceCandidate local;
  required RTCIceCandidate remote;
};

captureTimestamp of type DOMHighResTimeStamp .

The captureTimestamp is the timestamp that, the most recent frame (from an RTP packet originating from this source) delivered to the RTCRtpReceiver 's MediaStreamTrack , was originally captured. Its reference clock is the capture system 's NTP clock (same clock used to generate NTP timestamps for RTCP sender reports on that system).

On populating this member, the user agent MUST run the following steps:

1. If the relevant RTP packet contains the RTP Header Extension for Absolute Capture Time , return the value of the absolute capture timestamp field and abort these steps.

2. Otherwise, if the relevant RTP packet does not contain the RTP Header Extension for Absolute Capture Time but a previous RTP packet did, return the result of calculating the absolute capture timestamp according to timestamp interpolation and abort these steps.

3. Otherwise, return undefined .

Note

If multiple receiving tracks are sourced from the same capture system , two captureTimestamp s can be used to accurately measure audio-video synchronization since both timestamps are based on the same system's clock.

senderCaptureTimeOffset of type DOMHighResTimeStamp .

The senderCaptureTimeOffset is the sender system 's estimate of the offset between its own NTP clock and the capture system 's NTP clock, for the same frame that the captureTimestamp was originated from.

On populating this member, the user agent MUST run the following steps:

1. If the relevant RTP packet contains the RTP Header Extension for Absolute Capture Time and the estimated capture clock offset field is present, return the value of the estimated capture clock offset field and abort these steps.

2. Otherwise, if the relevant RTP packet does not contain the RTP Header Extension for Absolute Capture Time 's estimated capture clock offset field, but a previous RTP packet did, return the most recent value that was present and abort these steps.

3. Otherwise, return undefined .

Note

The time of capture can estimatedly be expressed in the sender system 's clock as follows: senderCaptureTimestamp = captureTimestamp + senderCaptureTimeOffset .

Note

The offset between the sender system 's clock and the receiver system 's clock can be estimated as follows: senderReceiverTimeOffset = RTCRemoteOutboundRtpStreamStats . timestamp }} - ( RTCRemoteOutboundRtpStreamStats . remoteTimestamp + RTCRemoteInboundRtpStreamStats . roundTripTime / 2).

Note

The time of capture can estimatedly be expressed in the receiver system 's clock as follows: receiverCaptureTimestamp = senderCaptureTimestamp + senderReceiverTimeOffset .

Note

The one-way delay between the capture system 's time of capture and the receiver system 's time of playout can be estimated as follows: RTCRtpContributingSource . timestamp - receiverCaptureTimestamp .

This section extends RTCDataChannel by making it transferable .

This allows sending and receiving messages outside the context the connection was created, for instance in workers or third-party iframes.

WebIDL  [Exposed=(Window,Worker), Transferable]
  partial interface RTCDataChannel {
};

RTP header extension encryption policy affects whether RTP header extension encryption is negotiated if the remote endpoint does not support [ RFC9335 ]. If the remote endpoint supports [ RFC9335 ], all media streams are sent utilizing [ RFC9335 ].

WebIDLenum RTCRtpHeaderEncryptionPolicy {
  "negotiate",
  "require"
};

rtpHeaderEncryptionNegotiated defines whether the transceiver is sending enrypted RTP header extensions as defined in [ RFC9335 ].

WebIDLpartial interface RTCRtpTransceiver {
  readonly attribute boolean rtpHeaderEncryptionNegotiated;
};

rtpHeaderEncryptionPolicy defines the policy for negotiation of RTP header encryption using [ RFC9335 ].

WebIDLpartial dictionary RTCConfiguration {
  RTCRtpHeaderEncryptionPolicy rtpHeaderEncryptionPolicy = "negotiate";
};

The RTCRtpReceiver interface is defined in [ WEBRTC ]. This document extends this interface by adding a static method and internal slot [[HardwareDisabled]] initialized to false .

WebIDLpartial interface RTCRtpReceiver {
  static undefined disableHardwareDecoding();
};

When the RTCRtpReceiver 's disableHardwareDecoding method is called, the user agent MUST run the following steps:

1. When the RTCPeerConnection.constructor() has been invoked abort these steps.

2. Set the RTCRtpReceiver's [[HardwareDisabled]] slot to true .

The RTCRtpSender interface is defined in [ WEBRTC ]. This document extends this interface by adding a static method and internal slot [[HardwareDisabled]] initialized to false .

WebIDLpartial interface RTCRtpSender {
  static undefined disableHardwareEncoding();
};

When the RTCRtpSender 's disableHardwareEncoding method is called, the user agent MUST run the following steps:

1. When the RTCPeerConnection.constructor() has been invoked abort these steps.

2. Set the RTCRtpSender's [[HardwareDisabled]] slot to true .

In the set a session description algorithm, add a step right after the step that sets transceiver.[[Receiver]].[[ReceiveCodecs]], saying "If the RTCRtpReceiver's [[HardwareDisabled]] slot is true , remove any codec from transceiver.[[Receiver]].[[ReceiveCodecs]] for which the underlying decoder is hardware-accelerated".

In the set a session description algorithm, add a step right after the step that sets transceiver.[[Sender]].[[SendCodecs]], saying "If the RTCRtpSender's [[HardwareDisabled]] slot is true , remove any codec from transceiver.[[Sender]].[[SendCodecs]] for which the underlying encoder is hardware-accelerated".

The RTCRtpSenderEncodedSource interface allows applications to inject encoded frames into an RTCRtpSender . This is intended to support use cases where an application already has access to the encoded frames it wants to send. An example is forwarding encoded frames coming from one or more peer connections to another peer connection. In this case, the application can use an encoded transform on the RTCRtpReceivers of the input peer connections to get the encoded frames and inject them on the RTCRtpSender of the output peer connection without having to decode and reencode the frames.

WebIDL[Exposed=DedicatedWorker]
interface RTCRtpSenderEncodedSource {
  constructor(DOMString kind);
  readonly attribute RTCRtpSenderEncodedSourceHandle handle;
  undefined enqueue((RTCEncodedVideoFrame or RTCEncodedAudioFrame) encodedFrame);  undefined stop();
};

An interface that acts as an opaque cross-realm reference to an RTCRtpSenderEncodedSource . It is transferable to the Window context and can be assigned to an RTCRtpSender using replaceTrack . An RTCRtpSenderEncodedSource always stays associated to its RTCRtpSenderEncodedSourceHandle regardless of the RTCRtpSenderEncodedSourceHandle 's realm, similar to the way a MediaStreamTrack source maintains references to the tracks it backs. The main difference is that an RTCRtpSenderEncodedSource has exactly one associated handle.

The advantage of separating the RTCRtpSenderEncodedSource from the RTCRtpSenderEncodedSourceHandle is that the realm that created the RTCRtpSenderEncodedSource can keep the RTCRtpSenderEncodedSource and send frames to it, while the realm that owns the RTCRtpSender can take care of the tasks related to managing the connection.

WebIDL[Exposed=(Window,DedicatedWorker), Transferable]
interface

RTCRtpSenderEncodedSourceHandle

{};

To create an RTCRtpSenderEncodedSourceHandle object, given an RTCRtpSenderEncodedSource source :

1. Let handle be a new RTCRtpSenderEncodedSourceHandle object.
2. Let handle have a [[Sender]] internal slot, initialized to null .
3. Let handle have a [[Kind]] internal slot, initialized to source . [[Kind]] .
4. Let handle have an [[IsDetached]] internal slot, initialized to false .
5. Let handle have an [[Id]] internal slot, initialized to a unique string.
6. Return handle .

RTCRtpSenderEncodedSourceHandle objects are transferable . Their transfer steps , given value , and dataHolder , are:

1. If value . [[IsDetached]] is true , throw a DataCloneError DOMException .
2. Set dataHolder . [[Kind]] to value . [[Kind]]
3. Set dataHolder . [[Id]] to value . [[Id]]
4. Set value . [[IsDetached]] to true .

The RTCRtpSenderEncodedSourceHandle transfer-receiving steps , given dataHolder and handle , are:

1. Set handle . [[Kind]] to dataHolder . [[Kind]]
2. Set handle . [[Id]] to dataHolder . [[Id]]
3. Set handle . [[Sender]] to null

We define the handle associated with an RTCRtpSenderEncodedSource source to be an RTCRtpSenderEncodedSourceHandle object handle for which the following holds:

1. handle . [[Id]] equals source . [[Handle]] . [[Id]]
2. handle . [[IsDetached]] is false

Given that RTCRtpSenderEncodedSourceHandle is transferable , but not serializable , there is at most one handle associated with an RTCRtpSenderEncodedSource and it is possible that they exist in different realms. The user agent can use whatever mechanism works best to keep track of the potentially cross-realm association between an encoded source and its associated handle, similarly to the way a MediaStreamTrack and its corresponding source are connected.

WebIDL  partial interface RTCRtpSender {    undefined replaceTrack(RTCRtpSenderEncodedSourceHandle withSourceHandle);
    readonly attribute RTCRtpSenderEncodedSourceHandle encodedSourceHandle;
};

• Make the following modification to the Create an RTCRtpSender algorithm:
  • After step 2, insert the following step:
    • Let sender have a [[SenderEncodedSourceHandle]] internal slot initialized to null .
• Make the following modification to RTCPeerConnection . addTrack () :
  • After step 8.1, insert the following step:
    • Set sender . [[SenderEncodedSourceHandle]] to null
• Make the following modifications to RTCRtpSender . removeTrack() :
  1. Replace step 9 with the following step:
     • If sender . [[SenderTrack]] is null and sender . [[SenderEncodedSourceHandle]] is null , abort these steps.
  2. After step 10, insert the following steps:
     1. Set sender . [[SenderEncodedSourceHandle]] . [[Sender]] to null
     2. Set sender . [[SenderEncodedSourceHandle]] to null .
• Make the following modifications to RTCRtpSender . setParameters () :
  • Replace step 8 with the following step:
    • In parallel, configure the media stack to use parameters to transmit sender . [[SenderTrack]] or sender . [[SenderEncodedSourceHandle]] .
• Make the following modifications to RTCRtpSender . replaceTrack() :
  • After step 6.4.4.2, insert the following steps:
    1. Set sender . [[SenderEncodedSourceHandle]] . [[Sender]] to null .
    2. Set sender . [[SenderEncodedSourceHandle]] to null .