fix: Hold the 2-way audio backchannel open for the shortest possible time (#2697)

The card claimed a camera's ONVIF audio backchannel in two places that
had
nothing to do with a call: the capability probe at camera init
(`&microphone`),
and a pre-armed `sendonly` audio transceiver on every live WebRTC offer.
Merely
looking at a dashboard occupied the camera's speaker line. Outbound
audio now
travels on its own audio-only WebRTC connection, opened when a call is
answered
and closed when it ends.

- The backchannel is claimed only for the duration of a call. Idle
viewing
  claims nothing.
- Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the
outbound
  audio still traverses WebRTC).
- No renegotiation and no video blink at call start or end.
- A call that cannot carry audio now reports it and ends, instead of
showing a
  live microphone that goes nowhere.
- `live.microphone.always_connected` is now purely about the browser
microphone
  permission prompt.
- Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and
DTLS, plus
  ~300ms for `go2rtc` to open an RTSP backchannel.

Verified against a live Frigate + `go2rtc` instance, and by unit tests
at 100%
coverage.

 - Closes #2691
 - Closes #2039
 - Closes #2178

Ref #2299 -- the probe no longer opens a backchannel, but it still runs
per
camera on every load and reconnect, and still dials the camera on the
direct-`go2rtc` path. Caching remains to be done.

Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel,
`go2rtc` keeps
that media set up on the camera's RTSP session for the life of the
producer.

Diagnoses #2678
This commit is contained in:
Dermot Duffy
2026-08-21 20:13:58 -07:00
committed by GitHub
parent 2d7c86c93c
commit 11cc543406
65 changed files with 2251 additions and 1045 deletions
+6 -1
View File
@@ -42,7 +42,12 @@ cameras:
> microphone button is intermittently missing on load, try increasing
> `metadata_fetch_timeout_seconds` or use
> [`capabilities.force`](./README.md?id=capabilities) to skip metadata
> detection entirely.
> detection entirely. If it is _never_ present, check the `go2rtc` stream
> itself: Frigate [recommends](https://docs.frigate.video/configuration/live/)
> adding `#backchannel=0` to the stream it restreams, which stops that stream
> offering 2-way audio at all. Point `stream` at a talk-capable stream instead.
> Note that _any_ `#` option on an RTSP source has this effect unless it also
> includes `backchannel=1`.
## `go2rtc (experimental)`
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 28 KiB

+7
View File
@@ -40,6 +40,12 @@ note over User, Mic
can ring even where microphone access needs a tap to be granted
end note
note over Card
The card opens an audio connection to the camera and holds it only while the
call is in progress. If the camera has no 2-way audio, or another application
is already using it, the call fails with an error
end note
Card --> User : You hear the caller
note over Card
Inbound audio unmutes if configured (<color:#43a047>**""live.auto_unmute""**</color>)
@@ -61,6 +67,7 @@ note over User, Card
<color:#43a047>**""live.controls.call.lock""**</color> is disabled -- navigating away
end note
note over Card : The audio connection to the camera closes, releasing it for other applications
Card --> User : Inbound audio mutes (<color:#43a047>**""live.auto_mute""**</color>)
Card -> Mic : Microphone mutes and disconnects
note over Mic
+14 -4
View File
@@ -17,13 +17,19 @@ challenging.
- Only Frigate cameras are supported.
- Only the `go2rtc` and `go2rtc-experimental` live providers are supported.
- Only the `webrtc` mode supports 2-way audio.
- The browser must be able to reach `go2rtc` over WebRTC. Outbound audio always
travels on its own WebRTC connection, regardless of what mode is carrying the
video.
If your setup supports 2-way audio but detection is intermittent on load:
- Increase `cameras[].go2rtc.metadata_fetch_timeout_seconds`.
- Or force the capability with `cameras[].capabilities.force: ['2-way-audio']`.
If detection never succeeds for a camera, the `go2rtc` stream itself may not
offer 2-way audio -- see
[`go2rtc` live provider configuration](../configuration/cameras/live-provider.md?id=go2rtc).
## Example configuration
```yaml
@@ -65,10 +71,14 @@ enabled by default and appears in the `live` view whenever the selected camera
in the overlay to speak. Both behaviors are configurable via
[`live.microphone.auto_unmute`](../configuration/live.md?id=microphone) and
[`live.auto_unmute`](../configuration/live.md).
- The camera will always load _without_ the microphone connected, unless the
- The camera loads _without_ the microphone connected, unless the
[`always_connected`](../configuration/live.md?id=microphone) microphone option
is set to `true`. On the first call there may be a brief `webrtc` connection
reset to include 2-way audio.
is set to `true`. Starting a call opens a separate connection that carries your
voice to the camera; ending the call closes it. The camera's audio input is
therefore occupied only while a call is in progress, leaving it free for other
applications the rest of the time. Expect under half a second between starting
a call and being audible on a local network, and a little more remotely. The
video keeps playing throughout.
- The browser asks for microphone permission when a call needs it. How often it
asks depends on the browser: Chrome remembers the choice for the site, Safari
asks once per page load, and Firefox asks for every call unless _Remember this