VideoFrame

The VideoFrame interface of the Web Codecs API represents a frame of a video.

Description

A VideoFrame object can be created or accessed in a number of ways. The MediaStreamTrackProcessor breaks a media track into individual VideoFrame objects.

A VideoFrame is a CanvasImageSource and has a constructor that accepts a CanvasImageSource. This means that a frame can be created from an image or video element.

A second constructor enables the creation of a VideoFrame from its binary pixel representation in a BufferSource.

Created frames may then turned into a media track, for example with the MediaStreamTrackGenerator interface that creates a media track from a stream of frames.

Constructor

VideoFrame.VideoFrame()

Creates a new VideoFrame object.

Properties

VideoFrame.formatRead only

Returns the pixel format of the VideoFrame.

VideoFrame.codedWidthRead only

Returns the width of the VideoFrame in pixels, potentially including non-visible padding, and prior to considering potential ratio adjustments.

VideoFrame.codedHeightRead only

Returns the height of the VideoFrame in pixels, potentially including non-visible padding, and prior to considering potential ratio adjustments.

VideoFrame.codedRectRead only

Returns a DOMRectReadOnly with the width and height matching codedWidth and codedHeight.

VideoFrame.visibleRectRead only

Returns a DOMRectReadOnly describing the visible rectangle of pixels for this VideoFrame.

VideoFrame.displayWidthRead only

Returns the width of the VideoFrame when displayed after applying aspect ratio adjustments.

VideoFrame.displayHeightRead only

Returns the height of the VideoFrame when displayed after applying aspect ratio adjustments.

VideoFrame.durationRead only

Returns an integer indicating the duration of the audio in microseconds.

VideoFrame.timestampRead only

Returns an integer indicating the timestamp of the audio in microseconds.

VideoFrame.colorSpaceRead only

Returns a VideoColorSpace object.

Methods

VideoFrame.allocationSize()

Returns the number of bytes required to hold the VideoFrame as filtered by options passed into the method.

VideoFrame.clone()

Creates a new VideoFrame object with reference to the same media resource as the original.

VideoFrame.close()

Clears all states and releases the reference to the media resource.

Examples

In the following example frames are returned from a MediaStreamTrackProcessor, then encoded. See the full example and read more about it in the article Video processing with WebCodecs.

let frame_counter = 0;

const track = stream.getVideoTracks()[0];
const media_processor = new MediaStreamTrackProcessor(track);

const reader = media_processor.readable.getReader();
while (true) {
    const result = await reader.read();
    if (result.done)
      break;

    let frame = result.value;
    if (encoder.encodeQueueSize > 2) {
      // Too many frames in flight, encoder is overwhelmed
      // let's drop this frame.
      frame.close();
    } else {
      frame_counter++;
      const insert_keyframe = (frame_counter % 150) == 0;
      encoder.encode(frame, { keyFrame: insert_keyframe });
      frame.close();
    }
}

Specifications

Specification
WebCodecs
# videoframe-interface

Browser compatibility

Desktop Mobile
Chrome Edge Firefox Internet Explorer Opera Safari WebView Android Chrome Android Firefox for Android Opera Android Safari on IOS Samsung Internet
VideoFrame
94
94
No
No
80
No
94
94
No
No
No
No
VideoFrame
94
94
No
No
80
No
94
94
No
No
No
No
allocationSize
94
94
No
No
80
No
94
94
No
No
No
No
clone
94
94
No
No
80
No
94
94
No
No
No
No
close
94
94
No
No
80
No
94
94
No
No
No
No
codedHeight
94
94
No
No
80
No
94
94
No
No
No
No
codedRect
94
94
No
No
80
No
94
94
No
No
No
No
codedWidth
94
94
No
No
80
No
94
94
No
No
No
No
colorSpace
94
94
No
No
80
No
94
94
No
No
No
No
displayHeight
94
94
No
No
80
No
94
94
No
No
No
No
displayWidth
94
94
No
No
80
No
94
94
No
No
No
No
duration
94
94
No
No
80
No
94
94
No
No
No
No
format
94
94
No
No
80
No
94
94
No
No
No
No
timestamp
94
94
No
No
80
No
94
94
No
No
No
No
visibleRect
94
94
No
No
80
No
94
94
No
No
No
No

See also

© 2005–2021 MDN contributors.
Licensed under the Creative Commons Attribution-ShareAlike License v2.5 or later.
https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame