This
document
introduces
an
API
for
cropping
a
video
track
derived
from
display-capture
of
the
current
tab.
Status
of
This
Document
This
is
a
preview
Do
not
attempt
to
implement
this
version
of
the
specification.
Do
not
reference
this
version
as
authoritative
in
any
way.
Instead,
see
https://w3c.github.io/mediacapture-region/
for
the
Editor's
draft.
This
section
describes
the
status
of
this
document
at
the
time
of
its
publication.
A
list
of
current
W3C
publications
and
the
latest
revision
of
this
technical
report
can
be
found
in
the
W3C
technical
reports
index
at
https://www.w3.org/TR/.
This
First
Public
Working
Draft
represents
the
direction
the
Web
Real-Time
Communications
Working
Group
intends
to
explore
to
solve
the
use
case
of
partial
capture
of
browsing
contexts.
The
Working
Group
is
particularly
interested
in
feedback
on
how
well
this
direction
matches
the
said
use
case
from
potential
adopters
of
the
API.
Publication
as
an
Editor's
Draft
does
not
imply
endorsement
by
W3C
and
its
Members.
This
is
a
draft
document
and
may
be
updated,
replaced
or
obsoleted
by
other
documents
at
any
time.
It
is
inappropriate
to
cite
this
document
as
other
than
work
in
progress.
As
well
as
sections
marked
as
non-normative,
all
authoring
guidelines,
diagrams,
examples,
and
notes
in
this
specification
are
non-normative.
Everything
else
in
this
specification
is
normative.
The
key
words
MUST
and
MUST
NOT
in
this
document
are
to
be
interpreted
as
described
in
BCP
14
[
RFC2119
]
[
RFC8174
]
when,
and
only
when,
they
appear
in
all
capitals,
as
shown
here.
Complex
applications
often
comprise
multiple
documents
in
distinct
iframes
,
all
displayed
within
the
same
browsing
context
.
Consider
such
an
application.
Assume
one
of
these
documents,
CAPTURING-DOC
uses
getDisplayMedia
()
or
getViewportMedia
to
capture
the
entire
current
browsing
context
.
If
this
document
then
wishes
to
crop
the
video
track
to
the
coordinates
of
some
sub-section
CAPTURE-TARGET
of
a
collaborating
document
CAPTURED-DOC
,
how
can
CAPTURING-DOC
do
so
performantly
and
reliably?
Recall
especially
that
changes
in
layout
due
to
scrolling,
zooming
or
window
resizing
present
additional
challenges.
3.2
Practical
Use-Case
Consider
a
combo-application
consisting
of
two
major
parts
hosted
in
different
iframes
within
the
same
tab
-
a
video-conferencing
application
and
a
productivity-suite
application.
Assume
the
video-conferencing
uses
existing/upcoming
APIs
such
as
getDisplayMedia
()
and/or
getViewportMedia
and
captures
the
entire
tab.
Now
it
needs
to
crop
away
everything
other
than
a
particular
section
of
the
productivity-suite.
It
needs
to
crop
away
its
own
video-conferencing
content,
any
speaker
notes
and
other
private
and/or
irrelevant
content
in
the
productivity-suite,
before
transmitting
the
resulting
cropped
video
remotely.
Moreover,
consider
that
it
is
likely
that
the
two
collaborating
applications
are
cross-origin
from
each
other.
They
can
post
messages,
but
all
communication
is
asynchronous,
and
it's
easier
and
more
performant
if
information
is
transmitted
sparingly
between
them.
That
precludes
solutions
involving
posting
of
entire
frames,
as
well
as
solutions
which
are
too
slow
to
react
to
changes
in
layout
(e.g.
scrolling,
zooming
and
window-size
changes).
It
is
worthwhile
to
note
that
most
applications
would
likey
prefer
to
use
getViewportMedia
in
such
scenarios.
However,
as
of
this
writing,
getViewportMedia
is
still
unspecified
and
unimplemented.
It
will
have
non-trivial
requirements
whose
adoption
will
take
some
time
and
effort.
As
such,
many
applications
will
likely
use
a
combination
of
getDisplayMedia
()
and
Region
Capture
for
some
time
to
come.
The
combination
of
getDisplayMedia
()
and
Region
Capture
is
also
useful
for
applications
that
allow
the
users
to
choose
whichever
display-surface
they
wish,
but
offer
distinct
functionality
depending
on
whether
users
choose
to
self-capture
or,
conversely,
choose
to
capture
a
window
or
monitor.
Such
applications
would
only
succeed
in
using
Region
Capture
if
the
user
chose
to
self-capture
;
otherwise,
the
attempt
to
apply
cropping
would
be
a
no-op.
Cropping
mechanism
:
A
mechanism
for
instructing
the
user
agent
to
start
cropping
a
video
track
to
the
contours
of
a
previously
tagged
Element
,
or
to
stop
such
cropping
and
revert
a
track
to
its
uncropped
state.
We
define
two
crop-states
for
video
tracks
-
cropped
and
uncropped
.
Tracks
start
out
uncropped
,
and
may
turn
to
cropped
when
cropTo
is
successfully
called
on
them.
It
allows
cropping
by
one
document
to
coordinates
specified
in
another
document.
Tagging
an
Element
as
a
potential
crop-target
allows
the
user
agent
to
avoid
unnecessary
work
on
all
other
elements,
like
the
calculation
of
bounding
boxes
and
sending
such
coordinates
cross-process.
6.2
CropTarget
Definition
CropTarget
is
an
intentionally
empty,
opaque
identifier
that
exposes
nothing.
identifier.
Its
sole
purpose
is
to
be
handed
to
cropTo
as
input.
]
interface
// Intentionally empty; just an opaque identifier.
There
is
no
consensus
yet
on
the
name
for
CropTarget
.
This
is
under
discussion
in
issue
#18
.
To
create
a
CropTarget
with
element
as
input,
run
the
following
steps:
Let
cropTarget
be
a
new
object
of
type
CropTarget
.
Let
weakRef
be
a
weak
reference
to
element
.
Create
cropTarget
.[[Element]]
initialized
to
weakRef
.
Note
cropTarget
keeps
a
weak
reference
to
the
element
it
represents.
In
other
words,
cropTarget
will
not
prevent
garbage
collection
of
its
element.
CropTarget
objects
are
serializable.
The
serialization
steps
,
given
value
,
serialized
,
and
a
boolean
forStorage
,
are:
If
forStorage
There
is
true
,
throw
with
a
new
DOMException
object
whose
name
attribute
has
the
value
"
no
consensus
yet
on
whether
DataCloneError
fromElement
".
Set
serialized
.[[CropTargetElement]]
to
value
.[[Element]].
The
deserialization
steps
,
given
serialized
and
value
are:
Set
value
.[[Element]]
to
serialized
.[[CropTargetElement]].
should
be
exposed
beyond
secure
contexts.
When
produceCropTarget
fromElement
is
called
on
with
a
given
element
,
the
user
agent
creates
a
CropTarget
with
element
as
input.
The
user
agent
MUST
return
a
Promise
p
.
The
user
agent
MUST
resolve
p
only
after
it
has
finished
all
the
necessary
internal
propagation
of
state
associated
with
the
new
CropTarget
,
at
which
point
the
user
agent
MUST
be
ready
to
receive
the
new
CropTarget
as
a
valid
parameter
to
cropTo
.
Calls
to
this
method
instruct
the
user
agent
to
start/stop
cropping
a
self-capture
video
track
to
the
bounding
client
rectangle
of
cropTarget
.[[Element]].
Since
the
track
is
restricted
to
the
visible
viewport
of
the
display-surface
,
the
captured
area
will
be
the
intersection
of
the
visible
viewport
and
the
element
bounding
client
rectangle.
Whenever
cropTo
is
invoked,
the
user
agent
MUST
execute
the
following
algorithm:
If
cropTarget
is
set
to
undefined
,
the
user
agent
MUST
stop
cropping.
This
video
track
reverts
to
the
uncropped
state.
If
cropTarget
is
a
valid
CropTarget
,
the
user
agent
MUST
start
cropping
this
video
track
to
the
contours
of
the
element
referenced
by
this
CropTarget
.
This
means
that
for
each
new
frame
produced
on
the
track,
the
user
agent
calculates
the
bounding
box
of
the
pixels
belonging
to
the
element,
and
crops
the
frame
to
the
coordinates
of
this
bounding
box.
Call
the
track's
state
before
this
method
invocation
PRE-STATE
,
and
after
this
method
invocation
POST-STATE
.
The
user
agent
MUST
resolve
p
when
it
is
guaranteed
that
no
more
frames
cropped
(or
uncropped)
according
to
PRE-STATE
have
been
delivered
to
the
application,
and
that
any
additional
frames
delivered
to
the
application
will
therefore
be
cropped
(or
uncropped)
according
to
either
POST-STATE
or
a
later
state.
Note
The
timing
of
the
cropTo
promise
resolution
and
the
timing
of
the
actual
cropping
of
video
frames
is
observable
to
JavaScript
through
MediaStreamTrack
transforms
.
It
is
expected
that
the
first
newly
cropped
video
frame
will
be
enqueued
on
the
MediaStreamTrack
ReadableStream
just
after
the
cropTo
promise
is
resolved.
Consider
a
frame
produced
on
a
cropped
video
track.
The
user
agent
calculates
the
intersection
of
(i)
the
top-level
browsing
context
's
viewport
and
(ii)
the
bounding
box
of
all
pixels
belonging
to
the
crop-session
target
.
This
intersection
is
defined
as
the
crop-session
target's
coordinates
for
that
frame.
7.2.2
Crop-Session
Edge
Cases
Consider
a
video
track
VT
cropped
to
a
given
crop-session
target
TARGET
.
We
define
the
behavior
of
the
crop-session
of
the
VT
in
the
face
of
changes
undergone
by
TARGET
.
const mainContentArea = navigator.getElementById('mainContentArea');
navigator.mediaDevices.produceCropTarget(mainContentArea);
const cropTarget = await CropTarget.fromElement(mainContentArea);
sendCropTarget(cropTarget);
functionsendCropTarget(cropTarget) {
// Can send the crop-target to another document in this tab// using postMessage() or using any other means.// Possibly there is no other document, and this is just consumed locally.
}