Skip to content

✨ [Feature Request] Expose audio level / audio buffer while recording video on iOS #3732

@itzdanishsalmani

Description

@itzdanishsalmani

What feature or enhancement are you suggesting?

Audio frames are not passed to frame processors — only video frames are. There is no way to get real-time audio levels while recording on iOS without a dedicated audio callback from VisionCamera itself.

What Platforms whould this feature/enhancement affect?

Android, iOS

Alternatives/Workarounds

The only workarounds attempted, all failed on iOS:

  1. react-native-sound-level — fails silently on iOS, callbacks never fire

  2. expo-av Audio.Recording with isMeteringEnabled — throws "recorder not prepared" error because VisionCamera exclusively owns
    AVAudioSession and iOS does not allow two simultaneous audio recorders

  3. InterruptionModeIOS.DuckOthers + LINEARPCM format — same error, AVAudioSession ownership cannot be shared regardless of configuration

  4. Frame Processor Plugin reading audio buffer — plugin registers but frame.buffer only ever contains video frames, never audio frames.
    CMFormatDescriptionGetMediaType always returns kCMMediaType_Video, so audio data is never accessible via this path

There is currently NO way to get real-time audio levels while recording video on iOS using VisionCamera v4. The only correct solution would be
VisionCamera exposing audio data via a dedicated callback or property, similar to how frame processors work for video.

IOS Audio Issue (1).docx

Additional information

Metadata

Metadata

Assignees

No one assigned

    Labels

    ✨ featureProposes a new feature or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions