Add two-way audio to the native WebRTC player: audio focus handling and push-to-talk

[TimoPtr]

Summary

Phase 3 of the native WebRTC camera player (ADR, stacked on #7140): sending microphone audio to cameras that support talk-back.

• AudioController in :webrtc-core — a small interface with an AudioManager-backed implementation that, while held, keeps the device in MODE_IN_COMMUNICATION (enabling the platform echo cancellation path used for calls) and takes transient voice-communication audio focus. Holds are reference counted so concurrent sessions balance correctly, and the previous audio mode is restored on the last release.
• Structural release in WebRtcSession — the session acquires the audio mode when the microphone track actually starts sending and releases it on every exit path (mic off, stop(), signaling/connection failure, release()). This makes the class of leaks fixed in Microphone remains locked after using two-way audio in Companion App (WebRTC) #6153 (communication mode / audio focus left behind) structurally impossible for consumers: they only toggle setMicEnabled and can't forget the cleanup. The hold is kept across automatic reconnections, so a Wi-Fi roam doesn't glitch the audio route.
• Hilt wiring — one process-wide AudioController provided in WebRtcModule (it must be shared for the reference counting to balance) and passed to both WebRtcSession consumers: the developer debug screen and the external-bus dashboard player.
• Push-to-talk in the WebRTC debug screen — a hold-to-talk button that keeps the mic live only while pressed; the first press requests the RECORD_AUDIO runtime permission instead. The mic state (Off/Live/Unavailable) is shown next to the player state.

The webrtc/mic external-bus message for dashboard push-to-talk is intentionally deferred until the frontend has a talk UI to drive it; the dashboard player already benefits from the session-level guarantees.

Checklist

• New or updated tests have been added to cover the changes following the testing guidelines.
• The code follows the project's code style and best_practices.
• The changes have been thoroughly tested, and edge cases have been considered.
• Changes are backward compatible whenever feasible. Any breaking changes are documented in the changelog for users and/or in the code for developers depending on the relevance.

Any other notes

• Stacked PR: based on Add external bus protocol for the native WebRTC camera player #7140 (external bus integration), which is based on Add Compose WebRTC renderer and developer debug screen #7139 (Compose renderer), which is based on Add native WebRTC session foundation (core engine + HA signaling) #7137 (session library + ADR). Only the last commit is new here.
• Draft until validated end-to-end on a device with a talk-back capable camera (the "thoroughly tested" checklist item).
• Tests: 12 new unit tests — 7 WebRtcSessionTest cases covering the acquire/release balance across mic toggling, reconnection, stop, failure and disposal, and 5 Robolectric AndroidAudioControllerTest cases covering the AudioManager mode set/restore, reference counting and idempotent release.
• RECORD_AUDIO and MODIFY_AUDIO_SETTINGS were already declared in the app manifest, so no new manifest permissions.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TxHn9ySi8aRFvvmfcPRgVu

---

Generated by Claude Code

[markfrancisonly]

This is a substantial stack—session engine #7137, renderer #7139, external-bus protocol #7140, and audio handling #7141—so I want to raise a concern while the direction is still open.

For roughly four years, I have tried to make two-way audio work reliably with Amcrest and Reolink cameras and doorbells. In my deployments, the client has rarely been the limiting layer. Camera firmware fails first: low-power hardware, fragile audio implementations, and codec defects the manufacturer may never fix. Streaming fails next: inexpensive cameras often serve only one or two consumers reliably, so go2rtc must proxy or transcode, and the user must find a codec, resolution, and audio combination that every device in the path accepts.

Most users never get past those layers. The WebView can already provide the foreground client experience: a notification opens the app and a two-way-audio camera card answers. A native runtime adds answer-before-unlock, CallStyle presentation, and pre-negotiated media. Those are valuable UX improvements, but they do not repair the device and transport layers where two-way audio most often fails.

Before merging, please validate end to end with go2rtc and representative Amcrest, Reolink, and similar devices. If the goal is two-way audio that works broadly, coordination with @AlexxIT and camera manufacturers is important than adding another client engine.

A native client introduces risk to WebRTC consumers that already work. I run the companion app on always-on wall panels that publish camera and microphone through webcam-publisher, and I have built browser-side calling with ha-videocall.

Once the native stack acquires the microphone—#7141 adds capture, MODE_IN_COMMUNICATION, and audio focus—the app has two WebRTC runtimes sharing the same media devices. Android documents that a losing audio capture may continue running while receiving silence, but it does not define deterministic arbitration between two capture runtimes inside one process. Frontend getUserMedia() consumers—custom cards, browser calls, and WHIP publishers—also have no way to know that a native acquisition is coming.

Without an explicit release-and-confirm handoff, existing capture may remain apparently connected while becoming silent or otherwise unstable.

As proposed, these PRs risks expanding the Android client without addressing the layers where two-way audio most often fails, while destabilizing WebRTC consumers that already work. I recommend that the ADR require both end-to-end validation across representative cameras and go2rtc configurations, and deterministic cross-stack acquisition, failure, release, and recovery behavior before dashboard talk-back or default enablement.

[TimoPtr]

This is a substantial stack—session engine #7137, renderer #7139, external-bus protocol #7140, and audio handling #7141—so I want to raise a concern while the direction is still open.

For roughly four years, I have tried to make two-way audio work reliably with Amcrest and Reolink cameras and doorbells. In my deployments, the client has rarely been the limiting layer. Camera firmware fails first: low-power hardware, fragile audio implementations, and codec defects the manufacturer may never fix. Streaming fails next: inexpensive cameras often serve only one or two consumers reliably, so go2rtc must proxy or transcode, and the user must find a codec, resolution, and audio combination that every device in the path accepts.

Most users never get past those layers. The WebView can already provide the foreground client experience: a notification opens the app and a two-way-audio camera card answers. A native runtime adds answer-before-unlock, CallStyle presentation, and pre-negotiated media. Those are valuable UX improvements, but they do not repair the device and transport layers where two-way audio most often fails.

Before merging, please validate end to end with go2rtc and representative Amcrest, Reolink, and similar devices. If the goal is two-way audio that works broadly, coordination with @AlexxIT and camera manufacturers is important than adding another client engine.

A native client introduces risk to WebRTC consumers that already work. I run the companion app on always-on wall panels that publish camera and microphone through webcam-publisher, and I have built browser-side calling with ha-videocall.

Once the native stack acquires the microphone—#7141 adds capture, MODE_IN_COMMUNICATION, and audio focus—the app has two WebRTC runtimes sharing the same media devices. Android documents that a losing audio capture may continue running while receiving silence, but it does not define deterministic arbitration between two capture runtimes inside one process. Frontend getUserMedia() consumers—custom cards, browser calls, and WHIP publishers—also have no way to know that a native acquisition is coming.

Without an explicit release-and-confirm handoff, existing capture may remain apparently connected while becoming silent or otherwise unstable.

As proposed, these PRs risks expanding the Android client without addressing the layers where two-way audio most often fails, while destabilizing WebRTC consumers that already work. I recommend that the ADR require both end-to-end validation across representative cameras and go2rtc configurations, and deterministic cross-stack acquisition, failure, release, and recovery behavior before dashboard talk-back or default enablement.

These stacked PR are not meant to be merge, it is a PoC before doing anything. Once we agreed at the OHF level to work on it's going to be a cross-repository effort to deliver the feature core, frontend, android and iOS. If you are interested into following the work that we are going to do please go in OpenHomeFoundation/roadmap#201