|
| 1 | +--- |
| 2 | +image: /generated/articles-docs-webcodecs-create-audio-decoder.png |
| 3 | +sidebar_label: createAudioDecoder() |
| 4 | +title: createAudioDecoder() |
| 5 | +slug: /webcodecs/create-audio-decoder |
| 6 | +crumb: '@remotion/webcodecs' |
| 7 | +--- |
| 8 | + |
| 9 | +import {UnstableDisclaimer} from './UnstableDisclaimer'; |
| 10 | + |
| 11 | +:::warning |
| 12 | + |
| 13 | +<UnstableDisclaimer /> |
| 14 | +::: |
| 15 | + |
| 16 | +# createAudioDecoder()<AvailableFrom v="4.0.307" /> |
| 17 | + |
| 18 | +This function is a wrapper around the [`AudioDecoder`](https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder) Web API. |
| 19 | + |
| 20 | +```tsx twoslash title="Create an audio decoder" |
| 21 | +import type {MediaParserAudioTrack} from '@remotion/media-parser'; |
| 22 | + |
| 23 | +const track = {} as unknown as MediaParserAudioTrack; |
| 24 | + |
| 25 | +// ---cut--- |
| 26 | + |
| 27 | +import {createAudioDecoder} from '@remotion/webcodecs'; |
| 28 | + |
| 29 | +const decoder = createAudioDecoder({ |
| 30 | + track, |
| 31 | + onFrame: console.log, |
| 32 | + onError: console.error, |
| 33 | +}); |
| 34 | +``` |
| 35 | + |
| 36 | +## Differences to `AudioDecoder` |
| 37 | + |
| 38 | +- Samples with a `codec` of `pcm-s16` are accepted and passed through, even if the `AudioDecoder` object does not exist or support it. |
| 39 | +- Two new methods are added: [`.waitForQueueToBeLessThan()`](#waitforqueuetobelessthanqueue-size-number) and [`.waitForFinish()`](#waitforfinish). |
| 40 | +- The [`dequeue`](https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/dequeue_event) event is not supported as it is not reliable across browsers. |
| 41 | +- In addition to [`EncodedAudioChunk`](https://developer.mozilla.org/en-US/docs/Web/API/EncodedAudioChunk), [`EncodedAudioChunkInit`](https://www.w3.org/TR/webcodecs/#dictdef-encodedaudiochunkinit) objects are also accepted for [`.decode()`](#decode). |
| 42 | +- A [`webcodecsController()`](/docs/webcodecs/webcodecs-controller) instance can be passed in to the function, allowing for decoding to be paused, resumed and aborted. |
| 43 | +- [`.decode()`](#decode) is async, and returns a promise, allowing for a halt if the decoder is paused. |
| 44 | +- Only samples with a size of 16 bytes or more are actually being input, [to avoid a bug in Chrome](https://github.com/remotion-dev/remotion/blob/c7f18a85bcb1bc58d3811efcd9e68c96ff38ccae/packages/webcodecs/src/create-audio-decoder.ts#L121-L129). |
| 45 | +- A [`logLevel`](#loglevel) can be passed in to the function, allowing the queue to be debugged. |
| 46 | +- The [`onFrame`](#onframe) callback is being awaited. When rejected, the error lands in the [`onError`](#onerror) callback. When resolved, only then the queue size counter will be decreased. |
| 47 | + |
| 48 | +## API |
| 49 | + |
| 50 | +Takes an object with the following properties: |
| 51 | + |
| 52 | +### `track` |
| 53 | + |
| 54 | +An [`AudioDecoderConfig`](https://www.w3.org/TR/webcodecs/#dictdef-audiodecoderconfig) object. |
| 55 | +You may pass a [`MediaParserAudioTrack`](/docs/media-parser/types#mediaparseraudiotrack) object from [`parseMedia()`](/docs/media-parser/parse-media), which also is an `AudioDecoderConfig` object. |
| 56 | + |
| 57 | +### `onFrame` |
| 58 | + |
| 59 | +A callback that is called when a frame is decoded. |
| 60 | + |
| 61 | +Takes a single argument, which is an [`AudioData`](https://developer.mozilla.org/en-US/docs/Web/API/AudioData) object. |
| 62 | + |
| 63 | +Only once the callback resolves, the queue size counter will be decreased. |
| 64 | + |
| 65 | +### `onError` |
| 66 | + |
| 67 | +A callback that is called when an error occurs or the decode is aborted through the controller. |
| 68 | + |
| 69 | +Takes a single argument, which is an [`Error`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error) object. |
| 70 | + |
| 71 | +### `controller?` |
| 72 | + |
| 73 | +A [`webcodecsController()`](/docs/webcodecs/webcodecs-controller) instance. |
| 74 | + |
| 75 | +If provided, you can call [`.pause()`](/docs/webcodecs/webcodecs-controller#pause), [`.resume()`](/docs/webcodecs/webcodecs-controller#resume) and [`.abort()`](/docs/webcodecs/webcodecs-controller#abort) on the controller to pause, resume and abort the decoding. |
| 76 | + |
| 77 | + |
| 78 | +### `logLevel?` |
| 79 | + |
| 80 | +_string_ <TsType type="LogLevel" source="@remotion/media-parser"/> |
| 81 | + |
| 82 | +One of `"error"`, `"warn"`, `"info"`, `"debug"`, `"trace"`. |
| 83 | +Default value: `"info"`, which logs only important information. |
| 84 | + |
| 85 | +## Return type |
| 86 | + |
| 87 | +Returns an object with the following properties: |
| 88 | + |
| 89 | + |
| 90 | +### `decode(sample: EncodedAudioChunkInit | EncodedAudioChunk)` |
| 91 | + |
| 92 | +Decodes a sample. Same as [`AudioDecoder.decode()`](https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/decode). |
| 93 | +You can pass in a [`MediaParserAudioSample`](/docs/media-parser/types#mediaparseraudiosample) object from [`parseMedia()`](/docs/media-parser/parse-media), which also satisfies the [`EncodedAudioChunkInit`](https://www.w3.org/TR/webcodecs/#dictdef-encodedaudiochunkinit) interface. |
| 94 | + |
| 95 | + |
| 96 | +### `waitForQueueToBeLessThan(queueSize: number)` |
| 97 | + |
| 98 | +A promise that resolves when the queue size is less than the given number. |
| 99 | +The queue is only decremented when the[ `onFrame`](#onframe) callback resolves. |
| 100 | + |
| 101 | +### `waitForFinish()` |
| 102 | + |
| 103 | +A promise that resolves when the decoder is finished. |
| 104 | + |
| 105 | +### `flush()` |
| 106 | + |
| 107 | +Flushes the decoder. Same as [`AudioDecoder.flush()`](https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/flush). |
| 108 | + |
| 109 | +### `close()` |
| 110 | + |
| 111 | +Closes the decoder. Same as [`AudioDecoder.close()`](https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/close). |
| 112 | + |
| 113 | +## Example usage with `@remotion/media-parser` |
| 114 | + |
| 115 | +In this example, the whole audio track is decoded and the decoder is closed when the track is done. |
| 116 | +By using `async` / `await`, the [`parseMedia()`](/docs/media-parser/parse-media) call is artificially slowed down to not flood the `AudioDecoder` and cause much memory to be allocated. |
| 117 | + |
| 118 | +```tsx twoslash title="Decode an audio track" |
| 119 | +import {parseMedia} from '@remotion/media-parser'; |
| 120 | +import {createAudioDecoder} from '@remotion/webcodecs'; |
| 121 | + |
| 122 | +await parseMedia({ |
| 123 | + src: 'https://parser.media/video.mp4', |
| 124 | + onAudioTrack: ({track, container}) => { |
| 125 | + const decoder = createAudioDecoder({ |
| 126 | + track, |
| 127 | + onFrame: console.log, |
| 128 | + onError: console.error, |
| 129 | + }); |
| 130 | + |
| 131 | + return async (sample) => { |
| 132 | + // Called on every sample |
| 133 | + await decoder.waitForQueueToBeLessThan(10); |
| 134 | + await decoder.decode(sample); |
| 135 | + |
| 136 | + return async () => { |
| 137 | + // Called when the track is done |
| 138 | + await decoder.flush(); |
| 139 | + await decoder.waitForFinish(); |
| 140 | + decoder.close() |
| 141 | + }; |
| 142 | + }; |
| 143 | + }, |
| 144 | +}); |
| 145 | +``` |
| 146 | + |
| 147 | + |
| 148 | +## See also |
| 149 | + |
| 150 | +- [Source code for this function](https://github.com/remotion-dev/remotion/blob/main/packages/webcodecs/src/create-audio-decoder.ts) |
| 151 | +- [`@remotion/webcodecs`](/docs/webcodecs) |
0 commit comments