Add tutorial for HEVC MediaRecorder/VideoEncoder support

README.md

@@ -187,7 +187,7 @@ Chrome 125 solves all issues with Intel HDR10 MPO, the feature has been re-enabl
 
 Edge 125 solves the issue of no zero-copy output when using `VDAVideoDecoder` decodes HEVC Main10 10bit contents on the Windows platform, and the issue of PQ/HDR10/HLG bad tone-mapping result could also be solved. The HDR rendering results of later versions of Edge are expected to be exactly the same as Chrome, performed by Skia, and the rendering results of various GPU manufacturers will remain consistent no matter system HDR mode on or off (Intel HDR10 MPO may be enabled if system HDR mode is turned on and GPU generation >= 11, which may result in slight inconsistencies with Skia rendering results).
 
-## How to verify certain profile or resolution is supported？
+## HEVC Decoding Support Verification
 
 ### Clear Content
 
@@ -199,7 +199,7 @@ const mediaConfig = {
    * You can use `file` or `media-source` and the result are same here. And if type is `webrtc`,
    * `contentType` should be replaced with `video/h265` (NOTE: `webrtc` feature is only for
    * testing purpose, official Chrome may won't enable this by default, so you should find 
-   * chromium v128 binary in this repo and test by yourself).
+   * chromium binary in this repo and test by yourself).
    */
   type: 'file',
   video: {
@@ -212,24 +212,24 @@ const mediaConfig = {
      * Range extensions: `hvc1.4.10.L93.B0`
      */
     contentType : 'video/mp4;codecs="hev1.1.6.L120.90"',
-    /* Width */
+    /** Width */
     width: 1920,
-    /* Height */
+    /** Height */
     height: 1080,
-    /* Any number */
+    /** Any number */
     bitrate: 10000, 
-    /* Any number */
+    /** Any number */
     framerate: 30
   }
 }
 
 navigator.mediaCapabilities.decodingInfo(mediaConfig)
   .then(result => {
-     /* Indicate whether or not the video with given profile, width, and height can played well on the browser */
+     /** Indicate whether or not the video with given profile, width, and height can played well on the browser */
     if (result.supported) {
-      console.log('Video can play!');
+      console.log('Video can decode!');
     } else {
-      console.log('Video can\'t play!');
+      console.log('Video can\'t decode!');
     }
   });
 ```
@@ -289,25 +289,31 @@ const videoConfig = {
    * Range extensions: `hvc1.4.10.L93.B0`
    */
   codec: 'hev1.1.6.L120.90',
-  /* HEVC is always hw accelerated */
+  /** HEVC is always hw accelerated */
   hardwareAcceleration: 'prefer-hardware',
-  /* Width */
+  /** Width */
   codedWidth: 1280,
-  /* Height */
+  /** Height */
   codedHeight: 720,
 }
 
 try {
   const result = await VideoDecoder.isConfigSupported(videoConfig);
-  /* Indicate whether or not the video with given profile, width, and height can be decoded by WebCodecs API */
+  /**
+   * Indicate whether or not the video with given profile, width, and height can be decoded
+   * by WebCodecs API.
+   */
   if (result.supported) {
-    console.log('Video can play!');
+    console.log('Video can decode!');
   } else {
-    console.log('Video can\'t play!');
+    console.log('Video can\'t decode!');
   }
 } catch (e) {
-  /* There is a bug that in previous version of Chromium, the api may throw Error if config is not supported */
-  console.log('Video can\'t play!');
+  /**
+   * There is a bug that in previous version of Chromium, the api may throw Error if config
+   * is not supported.
+   */
+  console.log('Video can\'t decode!');
 }
 ```
 
@@ -369,6 +375,120 @@ try {
 }
 ```
 
+## HEVC Encoding Support Verification
+
+### MediaRecorder
+
+The feature has been enabled by default in Chromium >= 133, supports three platforms including `Windows`, `macOS`, and `Android`, supports two formats, namely `hvc1` and `hev1`, as well as two muxer formats, `mkv` and `mp4`.
+
+```javascript
+/**
+ * Note 1: For the two formats, `mp4` and `mkv` (whose underlying implementation is
+ * the same as that of `webm`), it is always recommended to choose `mp4` without hesitation
+ * to ensure that there will be no issues with the duration and that normal seeking can
+ * be achieved.
+ * 
+ * Note 2: Chromium supports both `hev1` and `hvc1` tags. When the codecs is set to
+ * `hev1.1.6.L93.B0`, the advantage is that it supports the dynamic resolution change of
+ * `MediaStream`. However, the recorded `mp4` file cannot be played natively on Apple
+ * devices' players like Safari and QuickTime. When it is set to `hvc1.1.6.L93.B0`,
+ * the advantages and disadvantages are just the opposite. If the recorded video is
+ * only to be played on Chromium, it is recommended to choose `hev1`. If the recording
+ * resolution will never change, it is recommended to choose `hvc1`.
+ * 
+ * Note 3: Currently, `mp4` only supports two audio encoding formats for recording,
+ * namely `opus` and `mp4a.40.2` (AAC). Among them, AAC relies on the platform's
+ * hardware encoder but has a better compression ratio.
+ * 
+ * Note 4: Even if `isTypeSupported()` returns `true`, it doesn't necessarily mean
+ * that recording can be carried out. For example, if the recording resolution of
+ * the `VideoStream` during actual recording is greater than the maximum resolution
+ * supported by the hardware encoder (for example, some Intel iGPU only support 1080p
+ * while the video source is 4K), then the `onerror` callback will be triggered. In
+ * this case, it is necessary to handle the fallback situation of switching to `avc3`
+ * or `avc1` to re-create the recorder.
+ */
+const supported = MediaRecorder.isTypeSupported('video/mp4;codecs=hev1.1.6.L93.B0,opus');
+/** Indicate whether or not the HEVC main profile be encoded by MediaRecorder API */
+if (supported) {
+  console.log('Video can encode!');
+} else {
+  console.log('Video can\'t encode!');
+}
+```
+
+### VideoEncoder
+
+The feature has been enabled by default in Chromium >= 130, supports three platforms including `Windows`, `macOS`, and `Android`.
+
+```javascript
+const videoConfig = {
+  /** Chromium currently only supports the HEVC Main profile. */
+  codec: 'hev1.1.6.L120.90',
+  /**
+   * HEVC only supports hardware encoding (Note: macOS is an exception as it also supports 
+   * software encoding)
+   */
+  hardwareAcceleration: 'prefer-hardware',
+  /**
+   * The width, height, and framerate of the video.
+   * 
+   * - `macOS`: On the `arm64` Macs, it supports up to `8192x4352 & 120fps`, and on
+   *            the `x64` Macs, it supports up to `4096x2304 & 120fps`.
+   * - `Windows`: Latest Nvidia GPUs support up to `7680x4360 & 68fps` (RTX 4080).
+   *              Latest AMD, Intel, Qualcomm GPUs support up to `3840 x 2160 & 96fps`.
+   *              Some Intel iGPU only support up to `1920x1080 & 30fps`.
+   * - `Android`: Some Qualcomm chips support up to `8192 x 4352 & 60fps`, and most
+   *              chips support up to 4K.
+   * 
+   * Note: The actual encoding frame rate is based on the speed at which `VideoFrame`
+   * is sent. In general, when the requirements are not high, `1920x1080 & 30fps` can
+   * ensure that the old Intel iGPU on Windows can perform hardware encoding safely.
+   */
+  width: 1920,
+  height: 1080,
+  framerate: 30,
+  /** 
+   * Except that macOS HEVC software encoder does not support the `realtime` mode,
+   * hardware encoder basically supports both. Please set it according to actual needs.
+   */
+  latencyMode: 'quality',
+  /** 
+   * Set the output format to be `hevc` (with a 4-byte NALU size start code, and
+   * SPS/PPS/VPS exposed through `codecDescription`) or `annexb` (with a 4-byte 
+   * `00 00 00 01` start code + SPS/PPS/VPS inserted in front of the I frame).
+   */
+  hevc: {
+    format: 'annexb'
+  },
+  /**
+   * SVC mode, It is meaningful only when the latencyMode is `realtime`. When set
+   * to `L1T2`, it is only supported on macOS 14+ Arm64 models or some Intel GPU
+   * models on Windows.
+   */
+  scalabilityMode: 'L1T1',
+}
+
+try {
+  const result = await VideoEncoder.isConfigSupported(videoConfig);
+  /**
+   * Indicate whether or not the video with given profile, width, height, and
+   * framerate can be encoded by WebCodecs API.
+   */
+  if (result.supported) {
+    console.log('Video can encode!');
+  } else {
+    console.log('Video can\'t encode!');
+  }
+} catch (e) {
+  /**
+   * There is a bug that in previous version of Chromium, the api may throw
+   * Error if config is not supported.
+   */
+  console.log('Video can\'t encode!');
+}
+``` 
+
 ## What's the hardware decoding tech diff? (Compared with Edge / Safari / Firefox)
 
 #### Windows

README.zh_CN.md

@@ -185,7 +185,7 @@ Chrome 125 在解决了 Intel HDR10 MPO 的各种问题后，重启启用了该
 
 Edge 125 解决了 Windows 平台 `VDAVideoDecoder` 解码 HEVC Main10 10bit 视频后，没有零拷贝输出的问题，PQ/HDR10/HLG Tone-mapping 异常的问题方可亦得到解决。后期版本的 HDR 渲染结果预期将与 Chrome 完全一致，由于统一由 Skia 渲染，最终各显卡厂商的渲染结果在 HDR 模式开启/关闭前后，都可以保持一致（在 HDR 模式开启后，11 代以后的 Intel GPU, Intel HDR10 MPO 可能会被启用，此时与 Skia 渲染的结果有少许不一致）。
 
-## 如何验证特定 Profile, 分辨率的视频是否可以播放？
+## HEVC 解码验证
 
 ### 非加密内容
 
@@ -197,7 +197,7 @@ const mediaConfig = {
    * 这里写 `file` 或 `media-source` 都可以, 结果一致。当要检测 `webrtc`
    * 可用性时，`contentType` 应该被替换为 `video/h265` (注意: `webrtc` 功能 
    * 仅用于测试目的, Chrome 官方可能未来不会默认启用该功能, 你可以传入命令行开启，
-   * 或在本仓库使用自定义 Chromium v128 二进制测试)
+   * 或在本仓库使用自定义 Chromium 二进制测试)
    */
   type: 'file',
   video: {
@@ -210,24 +210,24 @@ const mediaConfig = {
      * Range extensions: `hvc1.4.10.L93.B0`
      */
     contentType : 'video/mp4;codecs="hev1.1.6.L120.90"',
-    /* 视频的宽度 */
+    /** 视频的宽度 */
     width: 1920,
-    /* 视频的高度 */
+    /** 视频的高度 */
     height: 1080,
-    /* 随便写 */
+    /** 随便写 */
     bitrate: 10000, 
-    /* 随便写 */
+    /** 随便写 */
     framerate: 30
   }
 }
 
 navigator.mediaCapabilities.decodingInfo(mediaConfig)
   .then(result => {
-    /* 指定的 Profile + 宽高的视频是否可解码 */
+    /** 指定的 Profile + 宽高的视频是否可解码 */
     if (result.supported) {
-      console.log('Video can play!');
+      console.log('Video can decode!');
     } else {
-      console.log('Video can\'t play!');
+      console.log('Video can\'t decode!');
     }
   });
 ```
@@ -287,25 +287,25 @@ const videoConfig = {
    * Range extensions: `hvc1.4.10.L93.B0`
    */
   codec: 'hev1.1.6.L120.90',
-  /* HEVC 只支持硬解 */
+  /** HEVC 只支持硬解 */
   hardwareAcceleration: 'prefer-hardware',
-  /* 视频的宽度 */
+  /** 视频的宽度 */
   codedWidth: 1280,
-  /* 视频的高度 */
+  /** 视频的高度 */
   codedHeight: 720,
 }
 
 try {
   const result = await VideoDecoder.isConfigSupported(videoConfig);
-  /* 指定的 Profile + 宽高的视频是否可解码 */
+  /** 指定的 Profile + 宽高的视频是否可解码 */
   if (result.supported) {
-    console.log('Video can play!');
+    console.log('Video can decode!');
   } else {
-    console.log('Video can\'t play!');
+    console.log('Video can\'t decode!');
   }
 } catch (e) {
-  /* 老版本 Chromium 可能对不支持的 Profile, throw Error */
-  console.log('Video can\'t play!');
+  /** 老版本 Chromium 可能对不支持的 Profile, throw Error */
+  console.log('Video can\'t decode!');
 }
 ```
 
@@ -366,6 +366,91 @@ try {
 }
 ```
 
+## HEVC 编码验证
+
+### MediaRecorder
+
+Chromium 133 及以上版本默认启用，支持 `Windows`, `macOS`, `Android` 三个平台，支持 `hvc1` 和 `hev1` 两种格式，支持 `mkv` 和 `mp4` 两种封装。
+
+```javascript
+/**
+ * 注1：`mp4` 和 `mkv` (底层实现和 `webm` 一样) 两种封装，建议无脑选 `mp4` 以保证 duration 无问题，可正常 Seek。
+ * 
+ * 注2：Chromium `hev1` 和 `hvc1` 两种 Tag 都支持，codecs 为 `hev1.1.6.L93.B0` 时，优点是支持 `MediaStream`
+ * 动态分辨率变换，但是录出来的 mp4 在 Apple 设备上的 Safari, QuickTime 等播放器无法原生播放，codecs 为 
+ * `hvc1.1.6.L93.B0` 时，优点和缺点刚好反过来，如果录制视频仅在 Chromium 上播，建议选 `hev1`，如果录制分辨率永远不变，
+ * 建议选 `hvc1`.
+ * 
+ * 注3：mp4 录制音频目前仅支持 `opus` 和 `mp4a.40.2` (AAC), 其中 AAC 依赖平台硬件编码器，但具有更好的压缩比
+ * 
+ * 注4：即使 `isTypeSupported()` 返回 true 也不代表一定能录制，例如实际录制时如果 `VideoStream` 的录制分辨率
+ * 大于硬件编码器支持的最大分辨率（e.g. 部分 Intel 核显只支持 1080p，但视频源是 4k 的情况），此时 `onerror` 
+ * 回调会触发，此时需要做好切换到 `avc3` 或 `avc1` 重建 Recorder 的兜底处理。
+ */
+const supported = MediaRecorder.isTypeSupported('video/mp4;codecs=hev1.1.6.L93.B0,opus');
+/** 是否可使用 MediaRecorder 硬件编码 HEVC. */
+if (supported) {
+  console.log('Video can encode!');
+} else {
+  console.log('Video can\'t encode!');
+}
+```
+
+### VideoEncoder
+
+Chromium 130 及以上版本默认启用，支持 `Windows`, `macOS`, `Android` 三个平台。
+
+```javascript
+const videoConfig = {
+  /** Chromium 目前只支持 HEVC Main Profile */
+  codec: 'hev1.1.6.L120.90',
+  /** HEVC 只支持硬编 (注: macOS 是个例外, 它也支持软编) */
+  hardwareAcceleration: 'prefer-hardware',
+  /**
+   * 视频的宽度，高度，以及帧率
+   * 
+   * - `macOS`: `arm64` 架构最高支持 `8192x4352 & 120fps`, `x64` 架构最高支持 `4096x2304 & 120fps`
+   * - `Windows`: 最新款 Nvidia 显卡最高支持 `7680x4360 & 68fps` (RTX 4080), 最新款 AMD, Intel,
+   *              Qualcomm 显卡最高支持 `3840 x 2160 & 96fps`, 部分 Intel 核显最高仅支持到
+   *              `1920x1080 & 30fps`
+   * - `Android`: 部分 Qualcomm 芯片最高支持 `8192 x 4352 & 60fps`，大部分芯片最高支持到 4K
+   * 
+   * 注：实际编码帧率以送 `VideoFrame` 的速度为准，通常情况要求不高的情况，`1920x1080 & 30fps` 可以保证
+   * Windows 老核显可安全硬编
+   */
+  width: 1920,
+  height: 1080,
+  framerate: 30,
+  /** 除 macOS HEVC 软编码器不支持 `realtime` 模式外，硬编码器基本两种都支持，请根据实际需要设置 */
+  latencyMode: 'quality',
+  /** 
+   * 设置输出的格式是 `hevc` (4字节 NALU size 起始头，SPS/PPS/VPS 通过 `codecDescription` 暴露)
+   * 还是 `annexb` (4字节 `00 00 00 01` 起始头 + SPS/PPS/VPS 插入在 I 帧前面)
+   */
+  hevc: {
+    format: 'annexb'
+  },
+  /**
+   * SVC 模式，仅当设置 `latencyMode` 为 `realtime` 时生效，当设置为 `L1T2` 时，仅 macOS14+ Arm64
+   * 机型或 Windows 部分 Intel 显卡机型支持
+   */
+  scalabilityMode: 'L1T1',
+}
+
+try {
+  const result = await VideoEncoder.isConfigSupported(videoConfig);
+  /** 指定的 Profile + 宽高 + 帧率等配置的视频是否可编码 */
+  if (result.supported) {
+    console.log('Video can encode!');
+  } else {
+    console.log('Video can\'t encode!');
+  }
+} catch (e) {
+  /** 老版本 Chromium 可能对不支持的 Profile, throw Error */
+  console.log('Video can\'t encode!');
+}
+```
+
 ## 硬解技术实现区别？(与 Edge / Safari / Firefox 的对比)
 
 #### Windows
@@ -435,7 +520,7 @@ Electron >= v33.0.0 已集成好 macOS, Windows 平台的 HEVC 硬编码功能
 
 ## 更新历史
 
-`2024-12-16` 新增 `avc3`, `hev1` MediaRecorder mime type 支持，解决 `mp4` 不支持录制动态分辨率视频的问题（Chrome >= `133.0.6901.0`
+`2024-12-16` 新增 `avc3`, `hev1` MediaRecorder mime type 支持，解决 `mp4` 不支持录制动态分辨率视频的问题 (Chrome >= `133.0.6901.0`)
 
 `2024-12-05` 默认为 WebRTC 启用 VideoToolbox L1T2 HEVC 编码支持 (Chrome >= `133.0.6878.0`)