fix: Hold the 2-way audio backchannel open for the shortest possible time (#2697)

The card claimed a camera's ONVIF audio backchannel in two places that
had
nothing to do with a call: the capability probe at camera init
(`&microphone`),
and a pre-armed `sendonly` audio transceiver on every live WebRTC offer.
Merely
looking at a dashboard occupied the camera's speaker line. Outbound
audio now
travels on its own audio-only WebRTC connection, opened when a call is
answered
and closed when it ends.

- The backchannel is claimed only for the duration of a call. Idle
viewing
  claims nothing.
- Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the
outbound
  audio still traverses WebRTC).
- No renegotiation and no video blink at call start or end.
- A call that cannot carry audio now reports it and ends, instead of
showing a
  live microphone that goes nowhere.
- `live.microphone.always_connected` is now purely about the browser
microphone
  permission prompt.
- Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and
DTLS, plus
  ~300ms for `go2rtc` to open an RTSP backchannel.

Verified against a live Frigate + `go2rtc` instance, and by unit tests
at 100%
coverage.

 - Closes #2691
 - Closes #2039
 - Closes #2178

Ref #2299 -- the probe no longer opens a backchannel, but it still runs
per
camera on every load and reconnect, and still dials the camera on the
direct-`go2rtc` path. Caching remains to be done.

Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel,
`go2rtc` keeps
that media set up on the camera's RTSP session for the life of the
producer.

Diagnoses #2678
This commit is contained in:
Dermot Duffy
2026-08-21 20:13:58 -07:00
committed by GitHub
parent 2d7c86c93c
commit 11cc543406
65 changed files with 2251 additions and 1045 deletions
+76
View File
@@ -0,0 +1,76 @@
import type { EnabledProxyConfig } from '../config/schema/common/proxy';
import { homeAssistantSignAndFetch } from '../ha/fetch';
import type { HomeAssistant } from '../ha/types';
import { createProxiedEndpointIfNecessary } from '../ha/web-proxy';
import type { Endpoint } from '../types';
import { errorToConsole } from '../utils/basic';
import { go2RTCStreamInfoSchema, type Go2RTCStreamInfo } from './types';
const getGo2RTCStreamMetadata = async (
hass: HomeAssistant,
endpoint: Endpoint,
timeoutSeconds: number,
): Promise<Go2RTCStreamInfo | null> => {
try {
return await homeAssistantSignAndFetch(hass, endpoint, go2RTCStreamInfoSchema, {
timeoutSeconds,
});
} catch (e) {
errorToConsole(e);
return null;
}
};
const streamSupports2WayAudio = (streamInfo: Go2RTCStreamInfo | null): boolean => {
if (!streamInfo?.producers) {
return false;
}
return streamInfo.producers.some(
(producer) =>
producer.medias?.some(
(media) =>
media.includes('audio') &&
(media.includes('sendonly') || media.includes('sendrecv')),
) ?? false,
);
};
/**
* Fetch go2rtc metadata and determine if the stream supports 2-way audio.
* Handles proxy transformation if proxy config requires it.
* Returns false if the endpoint is not available or fetch fails.
*
* Note: Caller is responsible for checking if live_provider is 'go2rtc' before calling.
*
* @param hass Home Assistant instance.
* @param go2rtcMetadataEndpoint The go2rtc metadata endpoint.
* @param proxyConfig The resolved proxy configuration for live streams.
* @returns True if supports 2-way audio, false otherwise.
*/
export const supports2WayAudio = async (
hass: HomeAssistant,
metadataFetchTimeoutSeconds: number,
go2rtcMetadataEndpoint?: Endpoint | null,
proxyConfig?: EnabledProxyConfig,
): Promise<boolean> => {
if (!go2rtcMetadataEndpoint) {
return false;
}
const endpoint = await createProxiedEndpointIfNecessary(
hass,
go2rtcMetadataEndpoint,
proxyConfig,
{ openLimit: 1 },
);
if (!endpoint) {
return false;
}
const streamInfo = await getGo2RTCStreamMetadata(
hass,
endpoint,
metadataFetchTimeoutSeconds,
);
return streamSupports2WayAudio(streamInfo);
};