diff --git a/README.md b/README.md index aa39417e..82979818 100644 --- a/README.md +++ b/README.md @@ -467,6 +467,7 @@ See the [fully expanded live configuration example](#config-expanded-live) for h | `actions` | | :white_check_mark: | Actions to use for the `live` view. See [actions](#actions) below.| | `controls` | | :white_check_mark: | Configuration for the `live` view controls. See below. | | `layout` | | :white_check_mark: | See [media layout](#media-layout) below.| +| `microphone` | | :white_check_mark: | See [microphone](#microphone) below.| #### Live Controls: Thumbnails @@ -540,6 +541,23 @@ live: | `mode` | `popup-bottom-right` | :white_check_mark: | How to display the live camera title. Acceptable values: `none`, `popup-top-left`, `popup-top-right`, `popup-bottom-left`, `popup-bottom-right` . | | `duration_seconds` | `2` | :white_check_mark: | The number of seconds to display the title popup. `0` implies forever.| + + +#### Live: Microphone + +All configuration is under: + +```yaml +live: + microphone: +``` + +| Option | Default | Overridable | Description | +| - | - | - | - | +| `disconnect_seconds` | `60` | :white_check_mark: | The number of seconds after which to disconnect the microphone from the stream. `0` implies never. | + +See [Using 2-way audio](#using-2-way-audio) for more information about the very particular requirements that must be followed for 2-way audio to work. + ### Media Viewer Options The `media_viewer` is used for viewing all `clip`, `snapshot` or recording media, in a media carousel. @@ -1021,6 +1039,44 @@ See [the WebRTC Card live configuration](#webrtc-live-configuration) above, and documentation](https://github.com/AlexxIT/WebRTC#configuration) for full configuration options that can be used here. + + +## Using 2-Way Audio + +This card supports 2-way audio (e.g. transmitting audio from a microphone to a suitably equipped camera). Requirements for 2-way audio to work: + +Environmental requirements: + * Must have a camera that supports audio out (otherwise what's the point!) + * Camera must be supported by `go2rtc` for 2-way audio (see [supported cameras](https://github.com/AlexxIT/go2rtc#two-way-audio)). + * Must be accessing your Home Assistant instance over `https`. The browser will enforce this. + +Card requirements: + * Only Frigate cameras are supported. + * Only the `go2rtc` live provider is supported. + * Only the `webrtc` mode supports 2-way audio: +```yaml +cameras: + - camera_entity: camera.front_door + live_provider: go2rtc + go2rtc: + modes: + - webrtc +``` + * Must have microphone menu button enabled: +```yaml +menu: + buttons: + microphone: + enabled: true +``` + +Usage: + * The camera will always load without the microphone connected. + * To speak, hold-down the microphone menu button. + * On first press, this will reset the `webrtc` connection to include 2-way audio. + * Thereafter hold the microphone button down to unmute/speak, let go to mute. + * The video will automatically reset to remove the microphone after the number of seconds specified by `disconnect_seconds` in the `microphone` configuration have elapsed since the last mute/unmute press. + ## Frigate Card Conditions @@ -1178,7 +1234,7 @@ Parameters for the `custom:frigate-card-ptz` element: | Parameter | Description | | - | - | | `action` | Must be `custom:frigate-card-action`. | -| `frigate_card_action` | Call a Frigate Card action. Acceptable values are `default`, `clip`, `clips`, `image`, `live`, `recording`, `recordings`, `snapshot`, `snapshots`, `download`, `timeline`, `camera_ui`, `fullscreen`, `camera_select`, `menu_toggle`, `media_player`, `live_substream_select`, `expand`.| +| `frigate_card_action` | Call a Frigate Card action. Acceptable values are `default`, `clip`, `clips`, `image`, `live`, `recording`, `recordings`, `snapshot`, `snapshots`, `download`, `timeline`, `camera_ui`, `fullscreen`, `camera_select`, `menu_toggle`, `media_player`, `live_substream_select`, `expand`, `microphone_mute`, `microphone_unmute`| @@ -1196,6 +1252,7 @@ Parameters for the `custom:frigate-card-ptz` element: |`media_player`| Perform a media player action. Takes a `media_player` parameter with the entity ID of the media_player on which to perform the action, and a `media_player_action` parameter which should be either `play` or `stop` to play or stop the media in question. | |`live_substream_select`| Perform a media player action. Takes a `camera` parameter with the [camera ID](#camera-ids) of the substream camera. | |`expand`| Expand the card into a dialog/popup. | +|`microphone_mute`, `microphone_unmute`| Mute or unmute the microphone. See [Using 2-way audio](#using-2-way-audio). | @@ -1791,6 +1848,8 @@ live: position: x: 50 y: 50 + microphone: + disconnect_seconds: 60 actions: entity: light.office_main_lights tap_action: @@ -3650,6 +3709,7 @@ To send an action to a named Frigate card on the dashboard: | `live` | :white_check_mark: | | | `media_player`| :heavy_multiplication_x: | Please [request](https://github.com/dermotduffy/frigate-hass-card/issues) if you need this. | | `menu_toggle` | :white_check_mark: | | +| `microphone_mute`, `microphone_unmute`| :heavy_multiplication_x: | | | `recording` | :white_check_mark: | | | `recordings` | :white_check_mark: | | | `snapshot` | :white_check_mark: | | @@ -3716,6 +3776,18 @@ live: mode: none ``` +### Microphone / 2-way audio doesn't work + +There are many requirements for 2-way audio to work. See [Using 2-way +audio](#using-2-way-audio) for more information about these. If your microphone +still does not work and you believe you meet all the requirements try +eliminating the card from the picture by going directly to the `go2rtc` UI, +navigating to `links` for your given stream, then to `webrtc.html` with a +microphone. If this does not work correctly with 2-way audio then your issue is +with `go2rtc` not with the card. In this case, you could file an issue in [that +repo](https://github.com/AlexxIT/go2rtc/issues) with debugging information as +appropriate. + ### Static image URL with credentials doesn't load Your browser will not allow a page/script (like this card) to pass credentials to a cross-origin (different host) image URL for security reasons. There is no way around this unless you could also control the webserver that is serving the image to specifically allow `crossorigin` requests (which is typically not the case for an image served from a camera, for example). The stock Home Assistant Picture Glance card has the same limitation, for the same reasons. diff --git a/src/action-handler-directive.ts b/src/action-handler-directive.ts index d62eb71e..d0323dd5 100644 --- a/src/action-handler-directive.ts +++ b/src/action-handler-directive.ts @@ -78,14 +78,10 @@ class ActionHandler extends HTMLElement implements ActionHandler { }); const start = (): void => { - const options = element.actionHandlerOptions; - - if (options?.hasHold) { - this.held = false; - this.timer = window.setTimeout(() => { - this.held = true; - }, this.holdTime); - } + this.held = false; + this.timer = window.setTimeout(() => { + this.held = true; + }, this.holdTime); fireEvent(element, 'action', { action: 'start_tap' }); }; @@ -102,14 +98,13 @@ class ActionHandler extends HTMLElement implements ActionHandler { // This action handler by default relies on synthetic click events for // touch devices, in order to ensure that embedded cards (e.g. WebRTC) // can use stock click handlers. The exception is for hold events. - !(options?.hasHold && this.held) + !this.held ) { return; } - if (options?.hasHold) { - clearTimeout(this.timer); - this.timer = undefined; - } + + clearTimeout(this.timer); + this.timer = undefined; fireEvent(element, 'action', { action: 'end_tap' }); diff --git a/src/camera-manager/frigate/engine-frigate.ts b/src/camera-manager/frigate/engine-frigate.ts index b501cbbf..53212a41 100644 --- a/src/camera-manager/frigate/engine-frigate.ts +++ b/src/camera-manager/frigate/engine-frigate.ts @@ -989,9 +989,7 @@ export class FrigateCameraManagerEngine const countSegments = () => sum( - cameraIDs.map( - (cameraID) => this._recordingSegmentsCache.getSize(cameraID) ?? 0, - ), + cameraIDs.map((cameraID) => this._recordingSegmentsCache.getSize(cameraID) ?? 0), ); const segmentsStart = countSegments(); @@ -1081,6 +1079,7 @@ export class FrigateCameraManagerEngine supportsSnapshots: !isBirdseye, supportsRecordings: !isBirdseye, supportsTimeline: !isBirdseye, + supports2WayAudio: cameraConfig.live_provider === 'go2rtc', }; } diff --git a/src/camera-manager/generic/engine-generic.ts b/src/camera-manager/generic/engine-generic.ts index 9e9207ba..0f3fe130 100644 --- a/src/camera-manager/generic/engine-generic.ts +++ b/src/camera-manager/generic/engine-generic.ts @@ -182,6 +182,7 @@ export class GenericCameraManagerEngine implements CameraManagerEngine { supportsRecordings: false, supportsSnapshots: false, supportsTimeline: false, + supports2WayAudio: false, }; } diff --git a/src/camera-manager/manager.ts b/src/camera-manager/manager.ts index b686082f..f3b1545b 100644 --- a/src/camera-manager/manager.ts +++ b/src/camera-manager/manager.ts @@ -755,6 +755,7 @@ export class CameraManager { supportsRecordings: perCameraCapabilities.some((cap) => cap?.supportsRecordings), supportsSnapshots: perCameraCapabilities.some((cap) => cap?.supportsSnapshots), supportsTimeline: perCameraCapabilities.some((cap) => cap?.supportsTimeline), + supports2WayAudio: perCameraCapabilities.some((cap) => cap?.supports2WayAudio), }; } } diff --git a/src/camera-manager/types.ts b/src/camera-manager/types.ts index 19b508cc..5810567e 100644 --- a/src/camera-manager/types.ts +++ b/src/camera-manager/types.ts @@ -100,6 +100,7 @@ interface BaseCapabilities { supportsRecordings: boolean; supportsSnapshots: boolean; supportsTimeline: boolean; + supports2WayAudio: boolean; } export type CameraManagerCapabilities = BaseCapabilities; diff --git a/src/card.ts b/src/card.ts index 1b7ace09..b78bb75a 100644 --- a/src/card.ts +++ b/src/card.ts @@ -90,6 +90,7 @@ import { ResolvedMediaCache } from './utils/ha/resolved-media.js'; import { supportsFeature } from './utils/ha/update.js'; import { FrigateCardInitializer } from './utils/initializer.js'; import { isValidMediaLoadedInfo } from './utils/media-info.js'; +import { MicrophoneController } from './utils/microphone'; import { getActionsFromQueryString } from './utils/querystring.js'; import { View } from './view/view.js'; @@ -179,6 +180,9 @@ class FrigateCard extends LitElement { @state() protected _expand?: boolean = false; + // null implies the user refused microphone access. + protected _microphoneController?: MicrophoneController; + protected _conditionState?: ConditionState; protected _refMenu: Ref = createRef(); @@ -341,7 +345,13 @@ class FrigateCard extends LitElement { * Get the style of emphasized menu items. * @returns A StyleInfo. */ - protected _getEmphasizedStyle(): StyleInfo { + protected _getEmphasizedStyle(critical?: boolean): StyleInfo { + if (critical) { + return { + animation: 'pulse 3s infinite', + color: 'var(--error-color, white)', + }; + } return { color: 'var(--primary-color, white)', }; @@ -613,6 +623,27 @@ class FrigateCard extends LitElement { }); } + if (this._microphoneController && cameraCapabilities?.supports2WayAudio) { + const muted = this._microphoneController.isMuted(); + buttons.push({ + icon: this._microphoneController.isForbidden() + ? 'mdi:microphone-message-off' + : muted + ? 'mdi:microphone-off' + : 'mdi:microphone', + ...this._getConfig().menu.buttons.microphone, + type: 'custom:frigate-card-menu-icon', + title: localize('config.menu.buttons.microphone'), + style: muted ? {} : this._getEmphasizedStyle(true), + start_tap_action: createFrigateCardCustomAction( + 'microphone_unmute', + ) as FrigateCardCustomAction, + end_tap_action: createFrigateCardCustomAction( + 'microphone_mute', + ) as FrigateCardCustomAction, + }); + } + if (screenfull.isEnabled && !this._isBeingCasted()) { buttons.push({ icon: screenfull.isFullscreen ? 'mdi:fullscreen-exit' : 'mdi:fullscreen', @@ -949,6 +980,21 @@ class FrigateCard extends LitElement { if (changedProps.has('_view')) { this._setPropertiesForExpandedMode(); } + + const oldConfig: FrigateCardConfig | undefined = + changedProps.get('_overriddenConfig') ?? changedProps.get('_config'); + const newConfig = this._getConfig(); + if ( + (!this._microphoneController || + changedProps.has('_overriddenConfig') || + changedProps.has('_config')) && + oldConfig?.live.microphone.disconnect_seconds !== + newConfig.live.microphone.disconnect_seconds + ) { + this._microphoneController = new MicrophoneController( + this._getConfig().live.microphone.disconnect_seconds, + ); + } } protected _setPropertiesForMinMaxHeight(): void { @@ -1488,6 +1534,24 @@ class FrigateCard extends LitElement { case 'diagnostics': this._diagnostics(); break; + case 'microphone_mute': + this._microphoneController?.mute(); + this.requestUpdate(); + break; + case 'microphone_unmute': + const unmuteAndUpdate = () => { + this._microphoneController?.unmute(); + this.requestUpdate(); + } + if ( + !this._microphoneController?.isConnected() && + !this._microphoneController?.isForbidden() + ) { + this._microphoneController?.connect().then(unmuteAndUpdate); + } else if (this._microphoneController?.isConnected()) { + unmuteAndUpdate(); + } + break; default: console.warn(`Frigate card received unknown card action: ${action}`); } @@ -2020,6 +2084,7 @@ class FrigateCard extends LitElement { .nonOverriddenConfig=${this._config} .conditionState=${this._conditionState} .hide=${!!this._message} + .microphoneStream=${this._microphoneController?.getStream()} >`} ${ // Keep message rendering to last to show messages that may have been diff --git a/src/components/live/live-go2rtc.ts b/src/components/live/live-go2rtc.ts index 83e2f50c..a278cc38 100644 --- a/src/components/live/live-go2rtc.ts +++ b/src/components/live/live-go2rtc.ts @@ -12,6 +12,7 @@ import { CameraConfig, ExtendedHomeAssistant, FrigateCardMediaPlayer, + MicrophoneConfig, } from '../../types.js'; import '../image.js'; import { @@ -34,10 +35,40 @@ const GO2RTC_URL_SIGN_EXPIRY_SECONDS = 24 * 60 * 60; @customElement('frigate-card-live-go2rtc-player') class FrigateCardGo2RTCPlayer extends VideoRTC { + protected _microphoneStream?: MediaStream; + + constructor(microphoneStream?: MediaStream) { + super(); + if (microphoneStream) { + this._microphoneStream = this._microphoneStream; + } + } + public play(): void { // Let Frigate card control auto playing. } + protected reconnect(): void { + if (this.wsState !== WebSocket.CLOSED) { + // The websocket has onclose handlers, so don't call onconnect directly, + // wait until the eventhandlers are finished.. + this.ws?.addEventListener('close', () => this.onconnect()); + this.ondisconnect(); + } else { + // Still call ondisconnect() as there may be an RTC connection to + // terminate even if the websocket is closed. + this.ondisconnect(); + this.onconnect(); + } + } + + public async setMicrophoneStream(microphoneStream?: MediaStream): Promise { + if (this._microphoneStream !== microphoneStream) { + this._microphoneStream = microphoneStream; + this.reconnect(); + } + } + public oninit(): void { super.oninit(); @@ -56,6 +87,85 @@ class FrigateCardGo2RTCPlayer extends VideoRTC { this.video.muted = true; } } + + // This is a modified version of onwebrtc() to support 2-way audio. + override onwebrtc() { + const pc = new RTCPeerConnection(this.pcConfig); + + const video2: HTMLVideoElement = document.createElement('video'); + video2.addEventListener('loadeddata', (ev) => this.onpcvideo(ev), { once: true }); + + pc.addEventListener('icecandidate', (ev) => { + const candidate = ev.candidate ? ev.candidate.toJSON().candidate : ''; + this.send({ type: 'webrtc/candidate', value: candidate }); + }); + + pc.addEventListener('track', (ev) => { + // when stream already init + if (video2.srcObject !== null) return; + + // when audio track not exist in Chrome + if (ev.streams.length === 0) return; + + // when audio track not exist in Firefox + if (ev.streams[0].id[0] === '{') return; + + // Filter out tracks that are not video related. + if (ev.track.kind !== 'video') return; + + video2.srcObject = ev.streams[0]; + }); + + pc.addEventListener('connectionstatechange', () => { + if (pc.connectionState === 'failed' || pc.connectionState === 'disconnected') { + pc.close(); // stop next events + + this.pcState = WebSocket.CLOSED; + this.pc = null; + + this.onconnect(); + } + }); + + this.onmessage['webrtc'] = (msg) => { + switch (msg.type) { + case 'webrtc/candidate': + pc.addIceCandidate({ + candidate: msg.value, + sdpMid: '0', + }).catch(() => console.debug); + break; + case 'webrtc/answer': + pc.setRemoteDescription({ + type: 'answer', + sdp: msg.value, + }).catch(() => console.debug); + break; + case 'error': + if (msg.value.indexOf('webrtc/offer') < 0) return; + pc.close(); + } + }; + + // Safari doesn't support "offerToReceiveVideo" + pc.addTransceiver('video', { direction: 'recvonly' }); + pc.addTransceiver('audio', { direction: 'recvonly' }); + + // Must add microphone tracks prior to making the offer. + this._microphoneStream?.getTracks().forEach((track) => { + console.info('Adding microphone track', track); + pc.addTransceiver(track, { direction: 'sendonly' }); + }); + + pc.createOffer().then((offer) => { + pc.setLocalDescription(offer).then(() => { + this.send({ type: 'webrtc/offer', value: offer.sdp }); + }); + }); + + this.pcState = WebSocket.CONNECTING; + this.pc = pc; + } } @customElement('frigate-card-live-go2rtc') @@ -69,6 +179,12 @@ export class FrigateCardGo2RTC extends LitElement implements FrigateCardMediaPla @property({ attribute: false }) public cameraEndpoints?: CameraEndpoints; + @property({ attribute: false }) + public microphoneStream?: MediaStream; + + @property({ attribute: false }) + public microphoneConfig?: MicrophoneConfig; + protected _player?: FrigateCardGo2RTCPlayer; public async play(): Promise { @@ -141,7 +257,7 @@ export class FrigateCardGo2RTC extends LitElement implements FrigateCardMediaPla return; } - this._player = new FrigateCardGo2RTCPlayer(); + this._player = new FrigateCardGo2RTCPlayer(this.microphoneStream); this._player.src = address; this._player.visibilityCheck = false; @@ -156,6 +272,9 @@ export class FrigateCardGo2RTC extends LitElement implements FrigateCardMediaPla if (!this._player || changedProps.has('cameraEndpoints')) { this._createPlayer(); } + if (changedProps.has('microphoneStream')) { + this._player?.setMicrophoneStream(this.microphoneStream); + } } protected render(): TemplateResult | void { diff --git a/src/components/live/live.ts b/src/components/live/live.ts index cf86604c..443602e8 100644 --- a/src/components/live/live.ts +++ b/src/components/live/live.ts @@ -138,6 +138,9 @@ export class FrigateCardLive extends LitElement { @property({ attribute: false }) public cardWideConfig?: CardWideConfig; + @property({ attribute: false }) + public microphoneStream?: MediaStream; + // Whether or not the live view is currently in the background (i.e. preloaded // but not visible) @state() @@ -251,6 +254,7 @@ export class FrigateCardLive extends LitElement { .liveOverrides=${this.liveOverrides} .cardWideConfig=${this.cardWideConfig} .cameraManager=${this.cameraManager} + .microphoneStream=${this.microphoneStream} @frigate-card:message=${(ev: CustomEvent) => { this._renderKey++; this._messageReceivedPostRender = true; @@ -315,6 +319,9 @@ export class FrigateCardLiveCarousel extends LitElement { @property({ attribute: false }) public cameraManager?: CameraManager; + @property({ attribute: false }) + public microphoneStream?: MediaStream; + // Index between camera name and slide number. protected _cameraToSlide: Record = {}; protected _refMediaCarousel: Ref = createRef(); @@ -541,6 +548,7 @@ export class FrigateCardLiveCarousel extends LitElement {
` diff --git a/src/components/views.ts b/src/components/views.ts index 4c5f17dc..da13525e 100644 --- a/src/components/views.ts +++ b/src/components/views.ts @@ -48,6 +48,9 @@ export class FrigateCardViews extends LitElement { @property({ attribute: false }) public hide?: boolean; + @property({ attribute: false }) + public microphoneStream?: MediaStream; + protected willUpdate(changedProps: PropertyValues): void { if (changedProps.has('view') || changedProps.has('config')) { if (this.view?.is('live') || this._shouldLivePreload()) { @@ -204,6 +207,7 @@ export class FrigateCardViews extends LitElement { .liveOverrides=${getOverridesByKey(this.config.overrides, 'live')} .cameraManager=${this.cameraManager} .cardWideConfig=${this.cardWideConfig} + .microphoneStream=${this.microphoneStream} class="${classMap(liveClasses)}" > diff --git a/src/const.ts b/src/const.ts index 9e79c0b7..c0df8745 100644 --- a/src/const.ts +++ b/src/const.ts @@ -191,6 +191,8 @@ export const CONF_LIVE_PRELOAD = `${CONF_LIVE}.preload` as const; export const CONF_LIVE_TRANSITION_EFFECT = `${CONF_LIVE}.transition_effect` as const; export const CONF_LIVE_SHOW_IMAGE_DURING_LOAD = `${CONF_LIVE}.show_image_during_load` as const; +export const CONF_LIVE_MICROPHONE_DISCONNECT_SECONDS = + `${CONF_LIVE}.microphone.disconnect_seconds` as const; const CONF_IMAGE = 'image' as const; export const CONF_IMAGE_LAYOUT_FIT = `${CONF_IMAGE}.layout.fit` as const; diff --git a/src/editor.ts b/src/editor.ts index 2e637c7d..ef6a1279 100644 --- a/src/editor.ts +++ b/src/editor.ts @@ -147,6 +147,7 @@ import { CONF_TIMELINE_STYLE, CONF_LIVE_CONTROLS_TIMELINE_STYLE, CONF_MEDIA_VIEWER_CONTROLS_TIMELINE_STYLE, + CONF_LIVE_MICROPHONE_DISCONNECT_SECONDS, } from './const.js'; import { localize } from './localize/localize.js'; import frigate_card_editor_style from './scss/editor.scss'; @@ -192,6 +193,7 @@ const MENU_LIVE_CONTROLS_THUMBNAILS = 'live.controls.thumbnails'; const MENU_LIVE_CONTROLS_TIMELINE = 'live.controls.timeline'; const MENU_LIVE_CONTROLS_TITLE = 'live.controls.title'; const MENU_LIVE_LAYOUT = 'live.layout'; +const MENU_LIVE_MICROPHONE = 'live.microphone'; const MENU_MEDIA_GALLERY_CONTROLS_THUMBNAILS = 'media_gallery.controls.thumbnails'; const MENU_MEDIA_GALLERY_CONTROLS_FILTER = 'media_gallery.controls.filter'; const MENU_MEDIA_VIEWER_CONTROLS = 'media_viewer.controls'; @@ -994,10 +996,13 @@ export class FrigateCardEditor extends LitElement implements LovelaceCardEditor configPathShowRecordings: string, defaultShowRecordings: boolean, ): TemplateResult { - return html` - ${this._renderOptionSelector(configPathStyle, this._timelineStyleTypes, { - label: localize(`config.common.${CONF_TIMELINE_STYLE}`), - })} + return html` ${this._renderOptionSelector( + configPathStyle, + this._timelineStyleTypes, + { + label: localize(`config.common.${CONF_TIMELINE_STYLE}`), + }, + )} ${this._renderNumberInput(configPathWindowSeconds, { label: localize(`config.common.${CONF_TIMELINE_WINDOW_SECONDS}`), })} @@ -1755,6 +1760,7 @@ export class FrigateCardEditor extends LitElement implements LovelaceCardEditor ${this._renderMenuButton('expand') /* */} ${this._renderMenuButton('timeline')} ${this._renderMenuButton('media_player')} + ${this._renderMenuButton('microphone')}
` : ''} @@ -1844,6 +1850,15 @@ export class FrigateCardEditor extends LitElement implements LovelaceCardEditor CONF_LIVE_LAYOUT_POSITION_X, CONF_LIVE_LAYOUT_POSITION_Y, )} + ${this._putInSubmenu( + MENU_LIVE_MICROPHONE, + true, + 'config.live.microphone.editor_label', + { name: 'mdi:microphone' }, + html` + ${this._renderNumberInput(CONF_LIVE_MICROPHONE_DISCONNECT_SECONDS)} + `, + )} ` : ''} diff --git a/src/external/go2rtc/video-rtc.d.ts b/src/external/go2rtc/video-rtc.d.ts index 3a0af296..edfa8fdf 100644 --- a/src/external/go2rtc/video-rtc.d.ts +++ b/src/external/go2rtc/video-rtc.d.ts @@ -12,11 +12,19 @@ export class VideoRTC extends HTMLElement { video: HTMLVideoElement; ws: WebSocket | null; wsURL: string; - pc: RTCPeerConnection; + pc: RTCPeerConnection | null; connectTS: number; mseCodecs: string; src: string | URL; oninit(): void; + send(value: unknown): void; + onpcvideo(ev: Event): void; + onconnect(): void; + ondisconnect(): void; + onclose(): void; + onopen(): void; + onwebrtc(): void; + onmessage: Record void>; } diff --git a/src/external/go2rtc/video-rtc.js b/src/external/go2rtc/video-rtc.js index 445aa94d..11ec30be 100644 --- a/src/external/go2rtc/video-rtc.js +++ b/src/external/go2rtc/video-rtc.js @@ -27,7 +27,8 @@ export class VideoRTC extends HTMLElement { "hvc1.1.6.L153.B0", // H.265 main 5.1 (Chromecast Ultra) "mp4a.40.2", // AAC LC "mp4a.40.5", // AAC HE - "opus", // OPUS Chrome + "flac", // FLAC (PCM compatible) + "opus", // OPUS Chrome, Firefox ]; /** diff --git a/src/localize/languages/en.json b/src/localize/languages/en.json index c4d70334..1f7322ec 100644 --- a/src/localize/languages/en.json +++ b/src/localize/languages/en.json @@ -218,6 +218,11 @@ "layout": "Live Layout", "lazy_load": "Live cameras are lazily loaded", "lazy_unload": "Live cameras are lazily unloaded", + "microphone": { + "disconnect_seconds": "Seconds after which to disconnect microphone (0=never)", + "editor_label": "Microphone", + "enabled": "Microphone enabled" + }, "preload": "Preload live view in the background", "show_image_during_load": "Show still image while the live stream is loading", "transition_effect": "Live camera transition effect" @@ -267,6 +272,7 @@ "image": "Image", "live": "Live", "media_player": "Send to media player", + "microphone": "Microphone", "priority": "Priority", "recordings": "Recordings", "snapshots": "Snapshots", diff --git a/src/localize/languages/it.json b/src/localize/languages/it.json index 500571fd..50c6c8ba 100644 --- a/src/localize/languages/it.json +++ b/src/localize/languages/it.json @@ -218,6 +218,11 @@ "layout": "Disposizione dal vivo", "lazy_load": "Le telecamere dal vivo sono pigramente cariche", "lazy_unload": "Le telecamere dal vivo sono pigramente non caricate", + "microphone": { + "disconnect_seconds": "", + "editor_label": "", + "enabled": "" + }, "preload": "Precarica Live View in background", "show_image_during_load": "Mostra un'immagine fissa durante il caricamento del live streaming", "transition_effect": "Effetto di transizione della telecamera dal vivo" diff --git a/src/localize/languages/pt-BR.json b/src/localize/languages/pt-BR.json index cbb19e24..d746f2ec 100644 --- a/src/localize/languages/pt-BR.json +++ b/src/localize/languages/pt-BR.json @@ -218,6 +218,11 @@ "layout": "Layout dinâmico", "lazy_load": "As câmeras ao vivo são carregadas lentamente", "lazy_unload": "As câmeras ao vivo são descarregadas preguiçosamente", + "microphone": { + "disconnect_seconds": "", + "editor_label": "", + "enabled": "" + }, "preload": "Pré-carregar a visualização ao vivo em segundo plano", "show_image_during_load": "Mostrar imagem estática enquanto a transmissão ao vivo está carregando", "transition_effect": "Efeito de transição de câmera ao vivo" diff --git a/src/scss/button.scss b/src/scss/button.scss index 3577c19b..c3148296 100644 --- a/src/scss/button.scss +++ b/src/scss/button.scss @@ -19,7 +19,7 @@ ha-icon-button.button { opacity: 1; } 50% { - opacity: 0; + opacity: 0.6; } 100% { opacity: 1; diff --git a/src/types.ts b/src/types.ts index f619f1a5..fa1cc018 100644 --- a/src/types.ts +++ b/src/types.ts @@ -219,6 +219,8 @@ const FRIGATE_CARD_GENERAL_ACTIONS = [ 'image', 'live', 'menu_toggle', + 'microphone_mute', + 'microphone_unmute', 'recording', 'recordings', 'snapshot', @@ -409,6 +411,20 @@ const imageBaseConfigSchema = z.object({ * Live provider options */ +const microphoneConfigDefault = { + disconnect_seconds: 60, +}; + +const microphoneConfigSchema = z + .object({ + disconnect_seconds: z + .number() + .min(0) + .default(microphoneConfigDefault.disconnect_seconds), + }) + .default(microphoneConfigDefault); +export type MicrophoneConfig = z.infer; + const go2rtcConfigSchema = z.object({ modes: z.enum(['webrtc', 'mse', 'mp4', 'mjpeg']).array().optional(), stream: z.string().optional(), @@ -932,6 +948,9 @@ const liveConfigDefault = { duration_seconds: 2, }, }, + microphone: { + ...microphoneConfigDefault, + }, }; const livethumbnailsControlSchema = thumbnailsControlSchema.extend({ @@ -975,6 +994,7 @@ const liveOverridableConfigSchema = z .boolean() .default(liveConfigDefault.show_image_during_load), layout: mediaLayoutConfigSchema.optional(), + microphone: microphoneConfigSchema.default(liveConfigDefault.microphone), }) .merge(actionsSchema); @@ -1037,6 +1057,7 @@ const menuConfigDefault = { fullscreen: visibleButtonDefault, expand: hiddenButtonDefault, media_player: visibleButtonDefault, + microphone: hiddenButtonDefault, recordings: hiddenButtonDefault, }, button_size: 40, @@ -1073,6 +1094,7 @@ const menuConfigSchema = z media_player: visibleButtonSchema.default( menuConfigDefault.buttons.media_player, ), + microphone: hiddenButtonSchema.default(menuConfigDefault.buttons.microphone), recordings: hiddenButtonSchema.default(menuConfigDefault.buttons.recordings), }) .default(menuConfigDefault.buttons), diff --git a/src/utils/microphone.ts b/src/utils/microphone.ts new file mode 100644 index 00000000..fc138d86 --- /dev/null +++ b/src/utils/microphone.ts @@ -0,0 +1,84 @@ +export class MicrophoneController { + protected _stream?: MediaStream | null; + protected _timerID: number | null = null; + + // We keep mute state separate from the stream state so that mute/unmute can + // be expressed before the stream is created -- and when it's create it will + // have the right mute status. + protected _mute = true; + + protected _disconnectSeconds: number; + + constructor(disconnectSeconds?: number) { + this._disconnectSeconds = disconnectSeconds ?? 0; + } + + public async connect(): Promise { + try { + this._stream = await navigator.mediaDevices.getUserMedia({ + audio: true, + video: false, + }); + } catch (e) { + this._stream = null; + } + this._setMute(); + } + + public async disconnect(): Promise { + this._stream?.getTracks().forEach((track) => track.stop()); + this._stream = undefined; + } + + public getStream(): MediaStream | undefined { + return this._stream ?? undefined; + } + + protected _setMute(): void { + this._stream?.getTracks().forEach((track) => { + track.enabled = !this._mute; + }); + this._startTimer(); + } + + public mute(): void { + this._mute = true; + this._setMute(); + } + + public unmute(): void { + this._mute = false; + this._setMute(); + } + + public isConnected(): boolean { + return !!this._stream; + } + + public isForbidden(): boolean { + return this._stream === null; + } + + public isMuted(): boolean { + // For safety, this function always returns the stream mute status directly + // (rather the internal state). + return !this._stream || this._stream.getTracks().every((track) => !track.enabled); + } + + protected _clearTimer(): void { + if (this._timerID) { + window.clearTimeout(this._timerID); + this._timerID = null; + } + } + + protected _startTimer(): void { + if (this._disconnectSeconds) { + this._clearTimer(); + this._timerID = window.setTimeout(() => { + this._clearTimer(); + this.disconnect(); + }, this._disconnectSeconds * 1000); + } + } +} diff --git a/tests/utils/microphone.test.ts b/tests/utils/microphone.test.ts new file mode 100644 index 00000000..0a3c0aca --- /dev/null +++ b/tests/utils/microphone.test.ts @@ -0,0 +1,102 @@ +import { describe, it, expect, afterEach, beforeEach, vi } from 'vitest'; +import { MicrophoneController } from '../../src/utils/microphone'; +import { mock } from 'vitest-mock-extended'; + +const navigatorMock = { + mediaDevices: { + getUserMedia: vi.fn(), + }, +}; + +// @vitest-environment jsdom +describe('MicrophoneController', () => { + beforeEach(() => { + vi.stubGlobal('navigator', navigatorMock); + }); + + afterEach(() => { + vi.unstubAllGlobals; + }); + + const createMockStream = (mute?: boolean): MediaStream => { + const stream = mock(); + const track = mock(); + track.enabled = !mute; + stream.getTracks.mockImplementation(() => [track]); + return stream; + }; + + it('should be muted on creation', () => { + const controller = new MicrophoneController(); + expect(controller).toBeTruthy(); + expect(controller.isMuted()).toBeTruthy(); + }); + + it('should be undefined without creation', () => { + const controller = new MicrophoneController(); + expect(controller.getStream()).toBeUndefined(); + }); + + it('should connect', async () => { + const controller = new MicrophoneController(); + const stream = createMockStream(); + navigatorMock.mediaDevices.getUserMedia.mockReturnValue(stream); + await controller.connect(); + expect(controller.isConnected()).toBeTruthy(); + expect(controller.getStream()).toBe(stream); + expect(controller.isMuted()).toBeTruthy(); + }); + + it('should be forbidden when permission denied', async () => { + const controller = new MicrophoneController(); + navigatorMock.mediaDevices.getUserMedia.mockRejectedValue(new Error()); + await controller.connect(); + expect(controller.isConnected()).toBeFalsy(); + expect(controller.isForbidden()).toBeTruthy(); + }); + + it('should mute', async () => { + const controller = new MicrophoneController(); + navigatorMock.mediaDevices.getUserMedia.mockReturnValue(createMockStream()); + await controller.connect(); + controller.mute(); + expect(controller.isMuted()).toBeTruthy(); + + controller.unmute(); + expect(controller.isMuted()).toBeFalsy(); + }); + + it('should be unmuted on creation if unmute called first', async () => { + const controller = new MicrophoneController(); + controller.unmute(); + navigatorMock.mediaDevices.getUserMedia.mockReturnValue(createMockStream()); + await controller.connect(); + expect(controller.isMuted()).toBeFalsy(); + }); + + it('should disconnect', async () => { + const controller = new MicrophoneController(); + navigatorMock.mediaDevices.getUserMedia.mockReturnValue(createMockStream()); + await controller.connect(); + expect(controller.isConnected()).toBeTruthy(); + + await controller.disconnect(); + expect(controller.isConnected()).toBeFalsy(); + }); + + it('should automatically disconnect', async () => { + const seconds = 10; + vi.useFakeTimers(); + + const controller = new MicrophoneController(seconds); + navigatorMock.mediaDevices.getUserMedia.mockReturnValue(createMockStream()); + + await controller.connect(); + expect(controller.isConnected()).toBeTruthy(); + + vi.advanceTimersByTime(seconds * 1000); + + expect(controller.isConnected()).toBeFalsy(); + vi.useRealTimers(); + }); +});