fix: Hold the 2-way audio backchannel open for the shortest possible time (#2697)

The card claimed a camera's ONVIF audio backchannel in two places that
had
nothing to do with a call: the capability probe at camera init
(`&microphone`),
and a pre-armed `sendonly` audio transceiver on every live WebRTC offer.
Merely
looking at a dashboard occupied the camera's speaker line. Outbound
audio now
travels on its own audio-only WebRTC connection, opened when a call is
answered
and closed when it ends.

- The backchannel is claimed only for the duration of a call. Idle
viewing
  claims nothing.
- Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the
outbound
  audio still traverses WebRTC).
- No renegotiation and no video blink at call start or end.
- A call that cannot carry audio now reports it and ends, instead of
showing a
  live microphone that goes nowhere.
- `live.microphone.always_connected` is now purely about the browser
microphone
  permission prompt.
- Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and
DTLS, plus
  ~300ms for `go2rtc` to open an RTSP backchannel.

Verified against a live Frigate + `go2rtc` instance, and by unit tests
at 100%
coverage.

 - Closes #2691
 - Closes #2039
 - Closes #2178

Ref #2299 -- the probe no longer opens a backchannel, but it still runs
per
camera on every load and reconnect, and still dials the camera on the
direct-`go2rtc` path. Caching remains to be done.

Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel,
`go2rtc` keeps
that media set up on the camera's RTSP session for the life of the
producer.

Diagnoses #2678
This commit is contained in:
Dermot Duffy
2026-08-21 20:13:58 -07:00
committed by GitHub
parent 2d7c86c93c
commit 11cc543406
65 changed files with 2251 additions and 1045 deletions
+14 -4
View File
@@ -17,13 +17,19 @@ challenging.
- Only Frigate cameras are supported.
- Only the `go2rtc` and `go2rtc-experimental` live providers are supported.
- Only the `webrtc` mode supports 2-way audio.
- The browser must be able to reach `go2rtc` over WebRTC. Outbound audio always
travels on its own WebRTC connection, regardless of what mode is carrying the
video.
If your setup supports 2-way audio but detection is intermittent on load:
- Increase `cameras[].go2rtc.metadata_fetch_timeout_seconds`.
- Or force the capability with `cameras[].capabilities.force: ['2-way-audio']`.
If detection never succeeds for a camera, the `go2rtc` stream itself may not
offer 2-way audio -- see
[`go2rtc` live provider configuration](../configuration/cameras/live-provider.md?id=go2rtc).
## Example configuration
```yaml
@@ -65,10 +71,14 @@ enabled by default and appears in the `live` view whenever the selected camera
in the overlay to speak. Both behaviors are configurable via
[`live.microphone.auto_unmute`](../configuration/live.md?id=microphone) and
[`live.auto_unmute`](../configuration/live.md).
- The camera will always load _without_ the microphone connected, unless the
- The camera loads _without_ the microphone connected, unless the
[`always_connected`](../configuration/live.md?id=microphone) microphone option
is set to `true`. On the first call there may be a brief `webrtc` connection
reset to include 2-way audio.
is set to `true`. Starting a call opens a separate connection that carries your
voice to the camera; ending the call closes it. The camera's audio input is
therefore occupied only while a call is in progress, leaving it free for other
applications the rest of the time. Expect under half a second between starting
a call and being audible on a local network, and a little more remotely. The
video keeps playing throughout.
- The browser asks for microphone permission when a call needs it. How often it
asks depends on the browser: Chrome remembers the choice for the site, Safari
asks once per page load, and Firefox asks for every call unless _Remember this