fix: Hold the 2-way audio backchannel open for the shortest possible time (#2697)
The card claimed a camera's ONVIF audio backchannel in two places that had nothing to do with a call: the capability probe at camera init (`µphone`), and a pre-armed `sendonly` audio transceiver on every live WebRTC offer. Merely looking at a dashboard occupied the camera's speaker line. Outbound audio now travels on its own audio-only WebRTC connection, opened when a call is answered and closed when it ends. - The backchannel is claimed only for the duration of a call. Idle viewing claims nothing. - Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the outbound audio still traverses WebRTC). - No renegotiation and no video blink at call start or end. - A call that cannot carry audio now reports it and ends, instead of showing a live microphone that goes nowhere. - `live.microphone.always_connected` is now purely about the browser microphone permission prompt. - Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and DTLS, plus ~300ms for `go2rtc` to open an RTSP backchannel. Verified against a live Frigate + `go2rtc` instance, and by unit tests at 100% coverage. - Closes #2691 - Closes #2039 - Closes #2178 Ref #2299 -- the probe no longer opens a backchannel, but it still runs per camera on every load and reconnect, and still dials the camera on the direct-`go2rtc` path. Caching remains to be done. Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel, `go2rtc` keeps that media set up on the camera's RTSP session for the life of the producer. Diagnoses #2678
This commit is contained in:
@@ -42,7 +42,12 @@ cameras:
|
||||
> microphone button is intermittently missing on load, try increasing
|
||||
> `metadata_fetch_timeout_seconds` or use
|
||||
> [`capabilities.force`](./README.md?id=capabilities) to skip metadata
|
||||
> detection entirely.
|
||||
> detection entirely. If it is _never_ present, check the `go2rtc` stream
|
||||
> itself: Frigate [recommends](https://docs.frigate.video/configuration/live/)
|
||||
> adding `#backchannel=0` to the stream it restreams, which stops that stream
|
||||
> offering 2-way audio at all. Point `stream` at a talk-capable stream instead.
|
||||
> Note that _any_ `#` option on an RTSP source has this effect unless it also
|
||||
> includes `backchannel=1`.
|
||||
|
||||
## `go2rtc (experimental)`
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 28 KiB |
@@ -40,6 +40,12 @@ note over User, Mic
|
||||
can ring even where microphone access needs a tap to be granted
|
||||
end note
|
||||
|
||||
note over Card
|
||||
The card opens an audio connection to the camera and holds it only while the
|
||||
call is in progress. If the camera has no 2-way audio, or another application
|
||||
is already using it, the call fails with an error
|
||||
end note
|
||||
|
||||
Card --> User : You hear the caller
|
||||
note over Card
|
||||
Inbound audio unmutes if configured (<color:#43a047>**""live.auto_unmute""**</color>)
|
||||
@@ -61,6 +67,7 @@ note over User, Card
|
||||
<color:#43a047>**""live.controls.call.lock""**</color> is disabled -- navigating away
|
||||
end note
|
||||
|
||||
note over Card : The audio connection to the camera closes, releasing it for other applications
|
||||
Card --> User : Inbound audio mutes (<color:#43a047>**""live.auto_mute""**</color>)
|
||||
Card -> Mic : Microphone mutes and disconnects
|
||||
note over Mic
|
||||
|
||||
@@ -17,13 +17,19 @@ challenging.
|
||||
|
||||
- Only Frigate cameras are supported.
|
||||
- Only the `go2rtc` and `go2rtc-experimental` live providers are supported.
|
||||
- Only the `webrtc` mode supports 2-way audio.
|
||||
- The browser must be able to reach `go2rtc` over WebRTC. Outbound audio always
|
||||
travels on its own WebRTC connection, regardless of what mode is carrying the
|
||||
video.
|
||||
|
||||
If your setup supports 2-way audio but detection is intermittent on load:
|
||||
|
||||
- Increase `cameras[].go2rtc.metadata_fetch_timeout_seconds`.
|
||||
- Or force the capability with `cameras[].capabilities.force: ['2-way-audio']`.
|
||||
|
||||
If detection never succeeds for a camera, the `go2rtc` stream itself may not
|
||||
offer 2-way audio -- see
|
||||
[`go2rtc` live provider configuration](../configuration/cameras/live-provider.md?id=go2rtc).
|
||||
|
||||
## Example configuration
|
||||
|
||||
```yaml
|
||||
@@ -65,10 +71,14 @@ enabled by default and appears in the `live` view whenever the selected camera
|
||||
in the overlay to speak. Both behaviors are configurable via
|
||||
[`live.microphone.auto_unmute`](../configuration/live.md?id=microphone) and
|
||||
[`live.auto_unmute`](../configuration/live.md).
|
||||
- The camera will always load _without_ the microphone connected, unless the
|
||||
- The camera loads _without_ the microphone connected, unless the
|
||||
[`always_connected`](../configuration/live.md?id=microphone) microphone option
|
||||
is set to `true`. On the first call there may be a brief `webrtc` connection
|
||||
reset to include 2-way audio.
|
||||
is set to `true`. Starting a call opens a separate connection that carries your
|
||||
voice to the camera; ending the call closes it. The camera's audio input is
|
||||
therefore occupied only while a call is in progress, leaving it free for other
|
||||
applications the rest of the time. Expect under half a second between starting
|
||||
a call and being audible on a local network, and a little more remotely. The
|
||||
video keeps playing throughout.
|
||||
- The browser asks for microphone permission when a call needs it. How often it
|
||||
asks depends on the browser: Chrome remembers the choice for the site, Safari
|
||||
asks once per page load, and Firefox asks for every call unless _Remember this
|
||||
|
||||
Reference in New Issue
Block a user