## Summary
- add optional microphone constraints for echo cancellation, noise
suppression, automatic gain control, and channel count
- request configured values as non-mandatory `ideal` constraints
- expose privacy-safe microphone capabilities, requested constraints,
and applied settings in card diagnostics
- document the new configuration and add schema, microphone manager, and
diagnostics tests
## Motivation
The card currently calls `getUserMedia()` with `audio: true`. This
leaves echo cancellation, noise suppression, automatic gain control, and
channel count implicit.
Browser and device behavior differs. Explicit processing defaults can
regress microphone gain or amplify noise on some devices. This change
therefore keeps all processing constraints optional and configurable.
## Configuration
```yaml
live:
microphone:
constraints:
echo_cancellation: true
noise_suppression: true
auto_gain_control: false
channel_count: 1
```
Configured values use `ideal` constraints. A browser can ignore
unsupported values. Card diagnostics show the browser capabilities, the
requested constraints, and the reported applied settings.
## Backward compatibility
- existing configurations still use `audio: true`
- no audio-processing defaults are added
- explicit `false` values are preserved
- diagnostic output excludes device and group identifiers
## Validation
- focused microphone, schema, and diagnostics tests: 46 passed
- full test suite: 7,177 passed
- lint passed
- format check passed
- typecheck passed
- unused-code check passed
- production build passed
The optional constraints were also tested successfully with an iOS Home
Assistant Companion client and a go2rtc-based full-duplex intercom. This
is a client microphone-processing change only. It does not add backend
audio denoise.
---------
Co-authored-by: dermotduffy <dermot.duffy@gmail.com>
The card claimed a camera's ONVIF audio backchannel in two places that
had
nothing to do with a call: the capability probe at camera init
(`µphone`),
and a pre-armed `sendonly` audio transceiver on every live WebRTC offer.
Merely
looking at a dashboard occupied the camera's speaker line. Outbound
audio now
travels on its own audio-only WebRTC connection, opened when a call is
answered
and closed when it ends.
- The backchannel is claimed only for the duration of a call. Idle
viewing
claims nothing.
- Two-way audio now works in `mse`, `mp4` and `mjpeg` modes (note: the
outbound
audio still traverses WebRTC).
- No renegotiation and no video blink at call start or end.
- A call that cannot carry audio now reports it and ends, instead of
showing a
live microphone that goes nowhere.
- `live.microphone.always_connected` is now purely about the browser
microphone
permission prompt.
- Call setup measured at 66ms (LAN) and ~260ms (cellular) for ICE and
DTLS, plus
~300ms for `go2rtc` to open an RTSP backchannel.
Verified against a live Frigate + `go2rtc` instance, and by unit tests
at 100%
coverage.
- Closes#2691
- Closes#2039
- Closes#2178
Ref #2299 -- the probe no longer opens a backchannel, but it still runs
per
camera on every load and reconnect, and still dials the camera on the
direct-`go2rtc` path. Caching remains to be done.
Ref AlexxIT/go2rtc#1860 -- once a call has opened a backchannel,
`go2rtc` keeps
that media set up on the camera's RTSP session for the life of the
producer.
Diagnoses #2678
The microphone is connected when a call needs it and released the moment
that
call ends, so the browser stops reporting it as in use at hangup rather
than
`disconnect_seconds` later.
Existing configurations are migrated automatically by the visual editor.
- Closes: #2681
BREAKING CHANGE: `live.microphone.disconnect_seconds` is removed. The
microphone is released when a call ends, so there is no idle countdown
to
configure. Use `live.microphone.always_connected` to hold it open
instead.
BREAKING CHANGE: The `microphone_connect` and `microphone_disconnect`
actions
are removed. The card owns the microphone lifecycle; `microphone_mute`
and
`microphone_unmute` remain.
BREAKING CHANGE: `call` is removed from `live.microphone.auto_mute`,
whose
default is now `[]`. The microphone is muted when a call ends regardless
of
this option.
BREAKING CHANGE: `microphone_unmute` has no effect outside a call.
Nothing
carries the audio at any other time, so the request is ignored rather
than
opening the microphone.
Draws significant inspiration (and direct styling) from
https://github.com/dermotduffy/advanced-camera-card/pull/2447 . Thank
you @Maudfer !
BREAKING CHANGE:
The microphone condition previously bundled two unrelated signals —
whether a two-way-audio session was connected and whether the microphone
was muted. Connection state is now its own dedicated call condition, and
microphone is reserved purely for mute state. Configs are upgraded
automatically (the card rewrites affected conditions under overrides,
elements, and automations). If you maintain config by hand, convert as
follows:
If you only used connected:
# Before
```yaml
condition: microphone
connected: true
```
# After
```yaml
condition: call
call: true
```
If you used both connected and muted — they must be split into two
conditions, since they no longer live together:
# Before
```yaml
condition: microphone
connected: true
muted: false
```
# After
```yaml
condition: and
conditions:
- condition: call
call: true
- condition: microphone
muted: false
```
- Add support for Frigate reviews / detections.
- Add support for GenAI metadata.
- Significant internal refactor to more flexible "UnifiedQuery" to allow
mixing cameras with simple metadata and review metadata (e.g. a timeline
view of a Frigate camera with reviews, and a Reolink camera with simple
metadata).
- Add support for folder media as camera media.
There are a few more PRs to commit prior to this going live, but
commiting this for now due to the scale of the change.
BREAKING CHANGE: `media_type` and `events_type` are retired under
`live`, `viewer` and `timeline` configuration sections, instead media
type is associated (once) with the camera under `media`.
