WebXR Plane Detection Module

1. Introduction

2. Initialization

2.1. Feature descriptor

In order for the applications to signal their interest in using plane detection during a session, the session must be requested with appropriate feature descriptor . The string plane-detection is introduced by this module as a new valid feature descriptor for plane detection feature.

A device is capable of supporting the plane-detection feature if the device’s tracking system exposes a native plane detection capability. The inline XR device MUST NOT be treated as capable of supporting the plane-detection feature.

When a session is created with plane-detection feature enabled, the update planes algorithm MUST be added to the list of frame updates of that session.

const session = await navigator.xr.requestSession("immersive-ar", {
  requiredFeatures: ["plane-detection"]
});

3. Planes

3.1. XRPlaneOrientation

enum XRPlaneOrientation {
    "horizontal",
    "vertical"
};

• A plane orientation of "horizontal" indicates that the plane is primarily oriented horizontally (according to the conventions of the underlying platform).

• A plane orientation of "vertical" indicates that the plane is primarily oriented vertically (according to the conventions of the underlying platform).

3.2. XRPlane

[Exposed=Window]
interface XRPlane {
    [SameObject] readonly attribute XRSpace planeSpace;
    readonly attribute FrozenArray<DOMPointReadOnly> polygon;
    readonly attribute XRPlaneOrientation? orientation;
    readonly attribute DOMHighResTimeStamp lastChangedTime;
    readonly attribute DOMString? semanticLabel;
};

An XRPlane represents a single, flat surface detected by the underlying XR system.

The planeSpace is an XRSpace that establishes the coordinate system of the plane. The native origin of the planeSpace tracks plane’s center. The underlying XR system defines the exact meaning of the plane center. The Y axis of the coordinate system defined by planeSpace MUST represent the plane’s normal vector.

Each XRPlane has an associated native entity .

Each XRPlane has an associated frame .

The polygon is an array of vertices that describe the shape of the plane. They are returned in the form of a loop of points at the edges of the polygon, expressed in the coordinate system defined by planeSpace . The Y coordinate of each vertex MUST be 0.0 .

The semanticLabel attribute is a string that describes the semantic label of the polygon. This string MAY be either null or empty if there is no semantic information. The XRSystem SHOULD populate this with the semantic label it has knowledge of.

A semantic label is an ASCII lowercase DOMString that describes the name in the real world of the XRPlane as known by the XRSystem . The list of semantic labels is defined in the semantic label registry .

The orientation describes orientation of the plane, as classified by the underlying XR system. In case the orientation cannot be classified into "horizontal" or "vertical" by the underlying XR system, this attribute will be set to null .

The lastChangedTime is the last time some of the plane attributes have been changed.

Note: The pose of a plane is not considered a plane attribute and therefore updates to plane pose will not cause the lastChangedTime to change. This is because plane pose is a property that is derived from two different entities - planeSpace and the XRSpace relative to which the pose is to be computed via getPose() function.

4. Obtaining detected planes

4.1. XRPlaneSet

[Exposed=Window]
interface XRPlaneSet {
  readonly setlike<XRPlane>;
};

An XRPlaneSet is a collection of XRPlane s. It is the primary mechanism of obtaining the collection of planes detected in an XRFrame .

partial interface XRFrame {
  readonly attribute XRPlaneSet detectedPlanes;
};

XRFrame is extended to contain detectedPlanes attribute which contains all planes that are still tracked in the frame. The set is initially empty and will be populated by the update planes algorithm. If this attribute is accessed when the frame is not active , the user agent MUST throw InvalidStateError .

partial interface XRSession {
  Promise<undefined> initiateRoomCapture();
};

XRSession is extended to contain associated set of tracked planes , which is initially empty. The elements of the set will be of XRPlane type.

XRSession is extended to contain a boolean room capture completed , which is initially false.

If XR device supports manual capture, it has an async room capture method which returns a boolean.

XRSession is also extended to contain the initiateRoomCapture method which, if supported, will ask the XR device to capture the current room layout. It is up to the XR device if this will replace or augment the set of tracked planes .

// `planes` will track all detected planes that the application is aware of,
// and at what timestamp they were updated. Initially, this is an empty map.
const planes = Map();
function onXRFrame(timestamp, frame) {
  const detectedPlanes = frame.detectedPlanes;
  // First, let's check if any of the planes we knew about is no longer tracked:
  for (const [plane, timestamp] of planes) {
    if(!detectedPlanes.has(plane)) {
      // Handle removed plane - `plane` was present in previous frame,
      // but is no longer tracked.
      // We know the plane no longer exists, remove it from the map:
      planes.delete(plane);
    }
  }
  // Then, let's handle all the planes that are still tracked.
  // This consists both of tracked planes that we have previously seen (may have
  // been updated), and new planes.
  detectedPlanes.forEach(plane => {
    if (planes.has(plane)) {
      // Handle previously-seen plane:
      if(plane.lastChangedTime > planes.get(plane)) {
        // Handle previously seen plane that was updated.
        // It means that one of the plane's properties is different than
        // it used to be - most likely, the polygon has changed.
        ... // Render / prepare the plane for rendering, etc.
        // Update the time when we have updated the plane:
        planes.set(plane, plane.lastChangedTime);
      } else {
        // Handle previously seen plane that was not updated in current frame.
        // Note that plane's pose relative to some other space MAY have changed.
      }
    } else {
      // Handle new plane.
      // Set the time when we have updated the plane:
      planes.set(plane, plane.lastChangedTime);
    }
    // Irrespective of whether the plane was previously seen or not,
    // & updated or not, its pose MAY have changed:
    const planePose = frame.getPose(plane.planeSpace, xrReferenceSpace);
  });
  frame.session.requestAnimationFrame(onXRFrame);
}

5. Native device concepts

5.1. Native plane detection

6. Privacy & Security Considerations

7. Acknowledgements

The following individuals have contributed to the design of the WebXR Plane Detection specification: