fix: Hold the 2-way audio backchannel open for the shortest possible time (#2697)

The card claimed a camera's ONVIF audio backchannel in two places that
had
nothing to do with a call: the capability probe at camera init
(`&microphone`),
and a pre-armed `sendonly` audio transceiver on every live WebRTC offer.
Merely
looking at a dashboard occupied the camera's speaker line. Outbound
audio now
travels on its own audio-only WebRTC connection, opened when a call is
answered
and closed when it ends.

- The backchannel is claimed only for the duration of a call. Idle
viewing
  claims nothing.
- Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the
outbound
  audio still traverses WebRTC).
- No renegotiation and no video blink at call start or end.
- A call that cannot carry audio now reports it and ends, instead of
showing a
  live microphone that goes nowhere.
- `live.microphone.always_connected` is now purely about the browser
microphone
  permission prompt.
- Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and
DTLS, plus
  ~300ms for `go2rtc` to open an RTSP backchannel.

Verified against a live Frigate + `go2rtc` instance, and by unit tests
at 100%
coverage.

 - Closes #2691
 - Closes #2039
 - Closes #2178

Ref #2299 -- the probe no longer opens a backchannel, but it still runs
per
camera on every load and reconnect, and still dials the camera on the
direct-`go2rtc` path. Caching remains to be done.

Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel,
`go2rtc` keeps
that media set up on the camera's RTSP session for the life of the
producer.

Diagnoses #2678
This commit is contained in:
Dermot Duffy
2026-08-21 20:13:58 -07:00
committed by GitHub
parent 2d7c86c93c
commit 11cc543406
65 changed files with 2251 additions and 1045 deletions
-71
View File
@@ -2,7 +2,6 @@ import { describe, expect, it, vi } from 'vitest';
import {
addAudioTracksMuteStateListener,
has2WayAudio,
hasAudio,
mayHaveAudio,
type AudioProperties,
@@ -176,76 +175,6 @@ describe('hasAudio', () => {
});
});
describe('has2WayAudio', () => {
const createMockTransceiver = (
trackKind: string | null,
direction: RTCRtpTransceiverDirection,
): RTCRtpTransceiver => {
return {
sender: {
track: trackKind ? { kind: trackKind } : null,
},
direction,
} as unknown as RTCRtpTransceiver;
};
const createMockPeerConnection = (
transceivers: RTCRtpTransceiver[],
): RTCPeerConnection => {
return {
getTransceivers: () => transceivers,
} as unknown as RTCPeerConnection;
};
it('should return false for null peer connection', () => {
expect(has2WayAudio(null)).toBe(false);
});
it('should return false when no transceivers', () => {
const pc = createMockPeerConnection([]);
expect(has2WayAudio(pc)).toBe(false);
});
it('should return true when audio transceiver is sendonly', () => {
const pc = createMockPeerConnection([createMockTransceiver('audio', 'sendonly')]);
expect(has2WayAudio(pc)).toBe(true);
});
it('should return true when audio transceiver is sendrecv', () => {
const pc = createMockPeerConnection([createMockTransceiver('audio', 'sendrecv')]);
expect(has2WayAudio(pc)).toBe(true);
});
it('should return false when audio transceiver is recvonly', () => {
const pc = createMockPeerConnection([createMockTransceiver('audio', 'recvonly')]);
expect(has2WayAudio(pc)).toBe(false);
});
it('should return false when audio transceiver is inactive', () => {
const pc = createMockPeerConnection([createMockTransceiver('audio', 'inactive')]);
expect(has2WayAudio(pc)).toBe(false);
});
it('should return false when only video transceiver with sendonly', () => {
const pc = createMockPeerConnection([createMockTransceiver('video', 'sendonly')]);
expect(has2WayAudio(pc)).toBe(false);
});
it('should return false when transceiver has no track', () => {
const pc = createMockPeerConnection([createMockTransceiver(null, 'sendonly')]);
expect(has2WayAudio(pc)).toBe(false);
});
it('should return true when mixed transceivers include sendonly audio', () => {
const pc = createMockPeerConnection([
createMockTransceiver('video', 'recvonly'),
createMockTransceiver('audio', 'recvonly'),
createMockTransceiver('audio', 'sendonly'),
]);
expect(has2WayAudio(pc)).toBe(true);
});
});
describe('addAudioTracksMuteStateListener', () => {
interface MockTrack {
kind: string;