feat: Add experimental rewrite of go2rtc live provider (MSE/WebRTC/MP4/MJPEG) (#2580)
- Closes #2556 - Closes #2450 **Key intended features:** - go2rtc compatible - 100% test coverage to significantly improve ability to test, maintain and work around browser weirdnesses (e.g. Safari). - Written from the ground up in the style of the rest of the project. **To use:** - Change `live_provider` from `go2rtc` to `go2rtc-experimental`.
This commit is contained in:
Dermot Duffy
@@ -4,15 +4,16 @@
|
||||
|
||||
The `live_provider` parameter determines what provides the live stream for a camera. Each provider offers different capabilities:
|
||||
|
||||
| Live Provider | Latency | Frame Rate | Loading Time | Installation | Supports [Proxying](./README.md?id=proxy) | Description |
|
||||
| ---------------------------------- | ------- | ---------- | ------------ | ------------------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `go2rtc` | Best | High | Better | Builtin | :white_check_mark: | Uses [go2rtc](https://github.com/AlexxIT/go2rtc) to stream live feeds and supports 2-way audio. |
|
||||
| `ha` (Native WebRTC) | Best | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- will offer a very low-latency feed direct to your browser. |
|
||||
| `ha` (HLS) | Poor | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- HLS fallback when a WebRTC connection cannot be established. |
|
||||
| `ha` (when configured with LL-HLS) | Better | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- can be configured to use an [LL-HLS](https://www.home-assistant.io/integrations/stream/#ll-hls) feed for lower latency. |
|
||||
| `image` | Poor | Poor | Best | Builtin | :heavy_multiplication_x: | Use refreshing snapshots of the built-in Home Assistant camera streams. |
|
||||
| `jsmpeg` | Better | Low | Poor | Builtin | :heavy_multiplication_x: | Use a the JSMPEG stream. |
|
||||
| `webrtc-card` | Best | High | Better | Separate installation required | :heavy_multiplication_x: | Embed's [AlexxIT's WebRTC Card](https://github.com/AlexxIT/WebRTC) to stream live feed, requires manual extra setup. See [`webrtc_card`](#webrtc_card). Not to be confused with native Home Assistant WebRTC (use the `ha` provider). |
|
||||
| Live Provider | Latency | Frame Rate | Loading Time | Installation | Supports [Proxying](./README.md?id=proxy) | Description |
|
||||
| ---------------------------------- | ------- | ---------- | ------------ | ------------------------------ | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `go2rtc` | Best | High | Better | Builtin | :white_check_mark: | Uses [go2rtc](https://github.com/AlexxIT/go2rtc) to stream live feeds and supports 2-way audio. |
|
||||
| `go2rtc-experimental` | Best | High | Better | Builtin | :white_check_mark: | **Experimental** re-implementation of the `go2rtc` provider client (MSE/WebRTC/MP4/MJPEG) that is expected to eventually replace the `go2rtc` live provider. Uses the same configuration as `go2rtc`. See [`go2rtc (experimental)`](#go2rtc-experimental). |
|
||||
| `ha` (Native WebRTC) | Best | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- will offer a very low-latency feed direct to your browser. |
|
||||
| `ha` (HLS) | Poor | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- HLS fallback when a WebRTC connection cannot be established. |
|
||||
| `ha` (when configured with LL-HLS) | Better | High | Better | Builtin | :heavy_multiplication_x: | Use the built-in Home Assistant camera streams -- can be configured to use an [LL-HLS](https://www.home-assistant.io/integrations/stream/#ll-hls) feed for lower latency. |
|
||||
| `image` | Poor | Poor | Best | Builtin | :heavy_multiplication_x: | Use refreshing snapshots of the built-in Home Assistant camera streams. |
|
||||
| `jsmpeg` | Better | Low | Poor | Builtin | :heavy_multiplication_x: | Use a the JSMPEG stream. |
|
||||
| `webrtc-card` | Best | High | Better | Separate installation required | :heavy_multiplication_x: | Embed's [AlexxIT's WebRTC Card](https://github.com/AlexxIT/WebRTC) to stream live feed, requires manual extra setup. See [`webrtc_card`](#webrtc_card). Not to be confused with native Home Assistant WebRTC (use the `ha` provider). |
|
||||
|
||||
## `go2rtc`
|
||||
|
||||
@@ -43,6 +44,20 @@ cameras:
|
||||
> [`capabilities.force`](./README.md?id=capabilities) to skip metadata
|
||||
> detection entirely.
|
||||
|
||||
## `go2rtc (experimental)`
|
||||
|
||||
> [!WARNING] `go2rtc-experimental` is a from-scratch re-implementation of the [`go2rtc`](#go2rtc) live provider client (MSE, WebRTC, MP4 and MJPEG). It is under active development and is expected to eventually replace the `go2rtc` live provider. Please try it and [report any issues you encounter](https://github.com/dermotduffy/advanced-camera-card/issues).
|
||||
|
||||
It uses the exact same configuration as [`go2rtc`](#go2rtc): select it with `live_provider: go2rtc-experimental` and configure the same [`go2rtc`](#go2rtc) block.
|
||||
|
||||
```yaml
|
||||
cameras:
|
||||
- camera_entity: camera.office
|
||||
live_provider: go2rtc-experimental
|
||||
go2rtc:
|
||||
# [...]
|
||||
```
|
||||
|
||||
## `ha`
|
||||
|
||||
The `ha` block configures use of the default Home Assistant (`ha`) live provider. It has no configuration options.
|
||||
@@ -164,6 +179,17 @@ cameras:
|
||||
stream: sitting_room
|
||||
url: 'https://my.custom.go2rtc.backend'
|
||||
metadata_fetch_timeout_seconds: 2
|
||||
- camera_entity: camera.office_go2rtc_experimental
|
||||
live_provider: go2rtc-experimental
|
||||
go2rtc:
|
||||
modes:
|
||||
- webrtc
|
||||
- mse
|
||||
- mp4
|
||||
- mjpeg
|
||||
stream: sitting_room
|
||||
url: 'https://my.custom.go2rtc.backend'
|
||||
metadata_fetch_timeout_seconds: 2
|
||||
- camera_entity: camera.office_jsmpeg
|
||||
live_provider: jsmpeg
|
||||
jsmpeg:
|
||||
|
||||
@@ -16,7 +16,7 @@ challenging.
|
||||
### Card requirements
|
||||
|
||||
- Only Frigate cameras are supported.
|
||||
- Only the `go2rtc` live provider is supported.
|
||||
- Only the `go2rtc` and `go2rtc-experimental` live providers are supported.
|
||||
- Only the `webrtc` mode supports 2-way audio.
|
||||
|
||||
If your setup supports 2-way audio but detection is intermittent on load:
|
||||
|
||||
+1
-1
@@ -106,7 +106,7 @@ export default defineConfig([
|
||||
languageOptions: {
|
||||
parser: tsParser,
|
||||
parserOptions: {
|
||||
project: './tsconfig.json',
|
||||
projectService: true,
|
||||
tsconfigRootDir: __dirname,
|
||||
},
|
||||
},
|
||||
|
||||
@@ -18,7 +18,10 @@ import type { HassStateDifference, HomeAssistant } from '../ha/types';
|
||||
import { localize } from '../localize/localize';
|
||||
import type { CapabilitiesRaw, CapabilityKey, Endpoint } from '../types';
|
||||
import { arrayify } from '../utils/basic';
|
||||
import { liveProviderSupports2WayAudio } from '../utils/live-provider';
|
||||
import {
|
||||
isGo2RTCLiveProvider,
|
||||
liveProviderSupports2WayAudio,
|
||||
} from '../utils/live-provider';
|
||||
import { Capabilities } from './capabilities';
|
||||
import type { CameraManagerEngine } from './engine';
|
||||
import { CameraNoIDError } from './error';
|
||||
@@ -345,9 +348,10 @@ export class Camera {
|
||||
...resolveProxyConfig(this._config.proxy),
|
||||
live:
|
||||
this._config.proxy.live === 'auto'
|
||||
? // Live is proxied if the live provider is go2rtc and if a go2rtc
|
||||
// URL is manually set.
|
||||
this._config.live_provider === 'go2rtc' && !!this._config.go2rtc?.url
|
||||
? // Live is proxied if the live provider streams from go2rtc and a
|
||||
// go2rtc URL is manually set.
|
||||
isGo2RTCLiveProvider(this._config.live_provider) &&
|
||||
!!this._config.go2rtc?.url
|
||||
: this._config.proxy.live,
|
||||
media: this._config.proxy.media === 'auto' ? false : this._config.proxy.media,
|
||||
};
|
||||
|
||||
@@ -6,6 +6,9 @@ export class PauseAction extends AdvancedCameraCardAction<GeneralActionConfig> {
|
||||
public async execute(api: CardActionsAPI): Promise<void> {
|
||||
await super.execute(api);
|
||||
|
||||
await api.getMediaLoadedInfoManager().get()?.mediaPlayerController?.pause();
|
||||
await api
|
||||
.getMediaLoadedInfoManager()
|
||||
.get()
|
||||
?.mediaPlayerController?.playback?.pause();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,6 @@ export class PlayAction extends AdvancedCameraCardAction<GeneralActionConfig> {
|
||||
public async execute(api: CardActionsAPI): Promise<void> {
|
||||
await super.execute(api);
|
||||
|
||||
await api.getMediaLoadedInfoManager().get()?.mediaPlayerController?.play();
|
||||
await api.getMediaLoadedInfoManager().get()?.mediaPlayerController?.playback?.play();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { createNotificationFromText } from '../../components-lib/notification/factory';
|
||||
import type { ConditionStateChange } from '../../condition-trigger/conditions/types';
|
||||
import { localize } from '../../localize/localize';
|
||||
import { Generation } from '../../utils/concurrency/generation';
|
||||
import { Timer } from '../../utils/timer';
|
||||
import { getStreamCameraID } from '../../view/substream';
|
||||
import type { View } from '../../view/view';
|
||||
@@ -22,7 +23,7 @@ export class CallManager {
|
||||
// resumed tail leaks audio onto the shared lock from an instance the user
|
||||
// can no longer see or control, and may install state into a fresh
|
||||
// lifecycle from a request that belongs to the previous one.
|
||||
private _initEpoch = 0;
|
||||
private _initGeneration = new Generation();
|
||||
|
||||
constructor(api: CardCallAPI) {
|
||||
this._api = api;
|
||||
@@ -103,7 +104,7 @@ export class CallManager {
|
||||
return false;
|
||||
}
|
||||
|
||||
const initEpoch = this._initEpoch;
|
||||
const initGeneration = this._initGeneration.current();
|
||||
const microphoneConnected = await this._connectMicrophone();
|
||||
// If the init/uninit lifecycle advanced while the microphone connect was
|
||||
// in flight, this request belongs to a previous lifecycle -- the view we
|
||||
@@ -111,7 +112,7 @@ export class CallManager {
|
||||
// clean teardown path for state we'd install here. Bail before touching
|
||||
// `_call`, the ringtone lock, or surfacing a notification onto a torn-down
|
||||
// NotificationManager.
|
||||
if (initEpoch !== this._initEpoch) {
|
||||
if (!this._initGeneration.isCurrent(initGeneration)) {
|
||||
return false;
|
||||
}
|
||||
if (!microphoneConnected) {
|
||||
@@ -241,7 +242,7 @@ export class CallManager {
|
||||
//
|
||||
// Safe to re-initialize afterwards via `initialize()`.
|
||||
public uninitialize(): void {
|
||||
this._initEpoch++;
|
||||
this._initGeneration.invalidate();
|
||||
this._ringtone.stop();
|
||||
this._unansweredTimer.stop();
|
||||
if (this._call) {
|
||||
|
||||
@@ -21,7 +21,10 @@ export type MediaUnavailableIssueReason =
|
||||
| 'entity_unavailable'
|
||||
| 'not_loading'
|
||||
| 'playback_error'
|
||||
| 'stalled';
|
||||
| 'server_error'
|
||||
| 'stalled'
|
||||
| 'two_way_audio_error'
|
||||
| 'unsupported';
|
||||
|
||||
declare module 'issue' {
|
||||
interface IssueTriggerContext {
|
||||
@@ -50,10 +53,22 @@ export const MEDIA_UNAVAILABLE_REASONS: Record<
|
||||
localizationKey: 'issues.media_unavailable.reasons.playback_error',
|
||||
icon: 'mdi:alert-circle',
|
||||
},
|
||||
server_error: {
|
||||
localizationKey: 'issues.media_unavailable.reasons.server_error',
|
||||
icon: 'mdi:server-network-off',
|
||||
},
|
||||
stalled: {
|
||||
localizationKey: 'issues.media_unavailable.reasons.stalled',
|
||||
icon: 'mdi:motion-pause',
|
||||
},
|
||||
two_way_audio_error: {
|
||||
localizationKey: 'issues.media_unavailable.reasons.two_way_audio_error',
|
||||
icon: 'mdi:microphone-off',
|
||||
},
|
||||
unsupported: {
|
||||
localizationKey: 'issues.media_unavailable.reasons.unsupported',
|
||||
icon: 'mdi:video-off-outline',
|
||||
},
|
||||
};
|
||||
|
||||
export class MediaUnavailableIssue implements Issue {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { MediaUnavailableIssueReason } from '../../../../card-controller/issues/issues/media-unavailable';
|
||||
import type { LivenessDetector, LivenessVerdict } from '../stream-liveness-controller';
|
||||
|
||||
const LIVE_ERROR_EVENT = 'advanced-camera-card:live:error';
|
||||
@@ -38,12 +39,19 @@ export class ProviderErrorDetector implements LivenessDetector {
|
||||
return this._verdict;
|
||||
}
|
||||
|
||||
private _handler = (ev: Event): void => {
|
||||
private _handler = (
|
||||
ev: CustomEvent<MediaUnavailableIssueReason | undefined>,
|
||||
): void => {
|
||||
ev.stopPropagation();
|
||||
if (this._verdict.state !== 'not_live') {
|
||||
// Authoritative: an explicit provider error overrides even direct frame
|
||||
// evidence. No placeholder -- the provider renders its own error.
|
||||
this._verdict = { state: 'not_live', authority: 'hard', reason: 'playback_error' };
|
||||
// evidence. No placeholder -- the provider renders its own error. The
|
||||
// provider may name the cause; otherwise it is a generic playback error.
|
||||
this._verdict = {
|
||||
state: 'not_live',
|
||||
authority: 'hard',
|
||||
reason: ev.detail ?? 'playback_error',
|
||||
};
|
||||
this._onChange();
|
||||
}
|
||||
};
|
||||
|
||||
@@ -41,10 +41,19 @@ export type LivenessVerdict =
|
||||
renderPlaceholder?: boolean;
|
||||
};
|
||||
|
||||
// The reconnecting placeholder the wrapper renders in place of a frozen
|
||||
// provider, carrying the cause so it can show a cause-specific message.
|
||||
interface LivenessPlaceholder {
|
||||
// The wrapper-facing projection of the internal LivenessVerdict: it keeps the
|
||||
// detector-internal `authority` from leaking out and collapses `unknown` into
|
||||
// "no failure", so a consumer sees only a confirmed failure (with its cause) or
|
||||
// nothing. Modeled as the failure rather than a `live` flag because a stream
|
||||
// that is merely still connecting is not a failure yet is not playing either, so
|
||||
// a positive `live` boolean would misleadingly read as "media is playing".
|
||||
interface StreamFailure {
|
||||
reason: MediaUnavailableIssueReason;
|
||||
|
||||
// Whether the wrapper should replace the provider with a reconnecting
|
||||
// placeholder (a silent freeze, e.g. an unavailable camera). False when the
|
||||
// provider renders its own error and should stay mounted.
|
||||
renderPlaceholder: boolean;
|
||||
}
|
||||
|
||||
export interface LivenessDetector {
|
||||
@@ -116,17 +125,15 @@ export class StreamLivenessController implements ReactiveController {
|
||||
}
|
||||
|
||||
public isLive(): boolean {
|
||||
return this._getVerdict().state !== 'not_live';
|
||||
return !this.getFailure();
|
||||
}
|
||||
|
||||
// The reconnecting placeholder to render in place of the (frozen) provider,
|
||||
// carrying the cause so the wrapper can show a cause-specific message. Null
|
||||
// when the provider should stay mounted (live, or a provider error that
|
||||
// renders its own error).
|
||||
public getPlaceholder(): LivenessPlaceholder | null {
|
||||
// The stream's confirmed liveness failure, or null when there is none (the
|
||||
// stream is live, or still connecting).
|
||||
public getFailure(): StreamFailure | null {
|
||||
const verdict = this._getVerdict();
|
||||
return verdict.state === 'not_live' && verdict.renderPlaceholder
|
||||
? { reason: verdict.reason }
|
||||
return verdict.state === 'not_live'
|
||||
? { reason: verdict.reason, renderPlaceholder: !!verdict.renderPlaceholder }
|
||||
: null;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
# go2rtc-experimental provider
|
||||
|
||||
|
||||
|
||||
The SVG is also its own editable source: it embeds the draw.io diagram, so open
|
||||
`architecture.drawio.svg` directly in draw.io / diagrams.net to change it.
|
||||
|
||||
## Maintaining the diagram
|
||||
|
||||
`architecture.drawio.svg` is a dual file: the **rendered SVG** (shown above /
|
||||
in GitHub) plus the **editable mxGraph XML** embedded in its `content="..."`
|
||||
attribute. Keep them in sync -- never commit an SVG whose embedded XML changed
|
||||
but whose rendered picture did not, or the image above goes stale.
|
||||
@@ -0,0 +1,148 @@
|
||||
// Wraps the browser's two incompatible MediaSource "flavors" behind one
|
||||
// interface, so mse.ts stays flavor-agnostic (and mockable in tests):
|
||||
// - classic MediaSource (Chrome/Firefox/desktop): attach via an object URL
|
||||
// on `video.src`.
|
||||
// - ManagedMediaSource (Safari/iOS 17+; iOS had no MediaSource at all before
|
||||
// then): attach via `video.srcObject`, and needs `disableRemotePlayback`.
|
||||
import type { UnsubscribeCallback } from '../../../../../types';
|
||||
|
||||
declare global {
|
||||
interface Window {
|
||||
// Safari 17+ managed variant of MediaSource; API-compatible for the
|
||||
// subset used here.
|
||||
ManagedMediaSource?: typeof MediaSource;
|
||||
}
|
||||
}
|
||||
|
||||
// The unified MediaSource surface mse.ts programs against; the flavor-specific
|
||||
// attach/detach lives in the implementations below.
|
||||
export interface MediaSourceInterface {
|
||||
attach(video: HTMLVideoElement): void;
|
||||
detach(video: HTMLVideoElement): void;
|
||||
|
||||
// Fires once the source is ready to accept SourceBuffers; nothing can be
|
||||
// appended before it.
|
||||
subscribeToSourceOpen(callback: () => void): UnsubscribeCallback;
|
||||
|
||||
// Creates the SourceBuffer that media chunks are appended to, configured for
|
||||
// the given codec MIME string.
|
||||
addSourceBuffer(codecs: string): SourceBuffer;
|
||||
|
||||
// Declares the seekable live window to the browser, which cannot infer it for
|
||||
// an open-ended live source.
|
||||
setLiveSeekableRange(startSeconds: number, endSeconds: number): void;
|
||||
|
||||
// Whether the source is still attached and accepting SourceBuffer operations.
|
||||
isOpen(): boolean;
|
||||
|
||||
// Whether a codec MIME string is playable.
|
||||
isTypeSupported(mimeType: string): boolean;
|
||||
}
|
||||
|
||||
// Creates a wrapped MediaSource, or null when the browser supports no variant.
|
||||
export type MediaSourceFactory = () => MediaSourceInterface | null;
|
||||
|
||||
abstract class MediaSourceInstanceBase implements MediaSourceInterface {
|
||||
protected _mediaSource: MediaSource;
|
||||
|
||||
constructor(mediaSource: MediaSource) {
|
||||
this._mediaSource = mediaSource;
|
||||
}
|
||||
|
||||
public abstract attach(video: HTMLVideoElement): void;
|
||||
public abstract detach(video: HTMLVideoElement): void;
|
||||
public abstract isTypeSupported(mimeType: string): boolean;
|
||||
|
||||
public subscribeToSourceOpen(callback: () => void): UnsubscribeCallback {
|
||||
this._mediaSource.addEventListener('sourceopen', callback);
|
||||
return () => this._mediaSource.removeEventListener('sourceopen', callback);
|
||||
}
|
||||
|
||||
public addSourceBuffer(codecs: string): SourceBuffer {
|
||||
return this._mediaSource.addSourceBuffer(codecs);
|
||||
}
|
||||
|
||||
public setLiveSeekableRange(startSeconds: number, endSeconds: number): void {
|
||||
this._mediaSource.setLiveSeekableRange(startSeconds, endSeconds);
|
||||
}
|
||||
|
||||
public isOpen(): boolean {
|
||||
return this._mediaSource.readyState === 'open';
|
||||
}
|
||||
}
|
||||
|
||||
class ManagedMediaSourceInstance extends MediaSourceInstanceBase {
|
||||
private _mediaSourceConstructor: typeof MediaSource;
|
||||
|
||||
constructor(mediaSourceConstructor: typeof MediaSource) {
|
||||
super(new mediaSourceConstructor());
|
||||
this._mediaSourceConstructor = mediaSourceConstructor;
|
||||
}
|
||||
|
||||
public attach(video: HTMLVideoElement): void {
|
||||
// ManagedMediaSource does not deliver data while remote playback (e.g.
|
||||
// AirPlay) is possible.
|
||||
video.disableRemotePlayback = true;
|
||||
video.srcObject = this._mediaSource;
|
||||
}
|
||||
|
||||
public detach(video: HTMLVideoElement): void {
|
||||
video.srcObject = null;
|
||||
}
|
||||
|
||||
public isTypeSupported(mimeType: string): boolean {
|
||||
return this._mediaSourceConstructor.isTypeSupported(mimeType);
|
||||
}
|
||||
}
|
||||
|
||||
class ClassicMediaSourceInstance extends MediaSourceInstanceBase {
|
||||
private _objectURL: string | null = null;
|
||||
|
||||
constructor() {
|
||||
super(new MediaSource());
|
||||
}
|
||||
|
||||
public attach(video: HTMLVideoElement): void {
|
||||
this._objectURL = URL.createObjectURL(this._mediaSource);
|
||||
|
||||
// The object URL is only needed until the browser has opened the media
|
||||
// source.
|
||||
this._mediaSource.addEventListener('sourceopen', () => this._revokeObjectURL(), {
|
||||
once: true,
|
||||
});
|
||||
|
||||
video.src = this._objectURL;
|
||||
video.srcObject = null;
|
||||
}
|
||||
|
||||
public detach(video: HTMLVideoElement): void {
|
||||
video.src = '';
|
||||
this._revokeObjectURL();
|
||||
}
|
||||
|
||||
public isTypeSupported(mimeType: string): boolean {
|
||||
return MediaSource.isTypeSupported(mimeType);
|
||||
}
|
||||
|
||||
private _revokeObjectURL(): void {
|
||||
if (this._objectURL) {
|
||||
URL.revokeObjectURL(this._objectURL);
|
||||
this._objectURL = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const createBrowserMediaSource: MediaSourceFactory = () => {
|
||||
// Safari exposes both flavors, so check Managed first and prefer it there: it
|
||||
// lets the browser throttle buffering to save battery and memory and hands
|
||||
// off cleanly to AirPlay, which classic MediaSource does not. Classic is the
|
||||
// fallback for other browsers.
|
||||
const managedMediaSource = window.ManagedMediaSource;
|
||||
if (managedMediaSource) {
|
||||
return new ManagedMediaSourceInstance(managedMediaSource);
|
||||
}
|
||||
if ('MediaSource' in window) {
|
||||
return new ClassicMediaSourceInstance();
|
||||
}
|
||||
return null;
|
||||
};
|
||||
@@ -0,0 +1,14 @@
|
||||
export type PeerConnectionFactory = (config: RTCConfiguration) => RTCPeerConnection;
|
||||
|
||||
export const GO2RTC_PEER_CONNECTION_CONFIG: RTCConfiguration = {
|
||||
bundlePolicy: 'max-bundle',
|
||||
iceServers: [
|
||||
{
|
||||
// Two public STUN servers so connectivity survives one being unreachable.
|
||||
urls: ['stun:stun.l.google.com:19302', 'stun:stun.cloudflare.com:3478'],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
export const createBrowserPeerConnection: PeerConnectionFactory = (config) =>
|
||||
new RTCPeerConnection(config);
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 924 KiB |
@@ -0,0 +1,144 @@
|
||||
import type { LitElement } from 'lit';
|
||||
|
||||
import { Generation } from '../../../../utils/concurrency/generation';
|
||||
import { LatestValueRunner } from '../../../../utils/concurrency/latest-value-runner';
|
||||
import { ImageMediaPlayerController } from '../../../media-player/image';
|
||||
import { OffscreenImage } from './offscreen-image';
|
||||
import type { ImageSurface } from './session-controller';
|
||||
|
||||
// Liveness: while frames are expected, a gap beyond the window is a stall.
|
||||
// Omitted -> the surface reports no liveness. `stallWindowSeconds` defaults to
|
||||
// the standard frame-stall window.
|
||||
interface ImageSurfaceLivenessOptions {
|
||||
isFrameExpected: () => boolean;
|
||||
stallWindowSeconds?: number;
|
||||
}
|
||||
|
||||
interface ImageSurfaceOptions {
|
||||
livenessOptions?: ImageSurfaceLivenessOptions;
|
||||
|
||||
// Factory for the detached image loader used to decode a frame.
|
||||
createImage?: () => HTMLImageElement;
|
||||
}
|
||||
|
||||
// Presents a go2rtc MJPEG/MP4 stream as a sequence of frames on an <img>: each
|
||||
// frame is decoded off-DOM, then shown via an object URL (the previous one
|
||||
// revoked), and an ImageMediaPlayerController exposes it to the card as a media
|
||||
// player. The renderer owns the <img> element; this owns what is shown on it.
|
||||
export class ImageSurfaceController implements ImageSurface {
|
||||
private _getImageCallback: () => HTMLImageElement | null;
|
||||
private _mediaPlayerController: ImageMediaPlayerController;
|
||||
|
||||
// An off-screen <img> used to decode each frame before it is shown on the
|
||||
// visible element (reused: only one frame decodes at a time).
|
||||
private _decoder: OffscreenImage;
|
||||
|
||||
// Current frame's object URL.
|
||||
private _currentObjectURL: string | null = null;
|
||||
|
||||
// Invalidated by reset() so a frame whose decode was still in flight when the
|
||||
// surface was switched or torn down does not paint a stale frame afterwards.
|
||||
private _generation = new Generation();
|
||||
|
||||
// Frames can arrive faster than the browser decodes one. Present at most one
|
||||
// at a time and keep only the newest frame while busy, so the display
|
||||
// converges on the latest without unbounded decode work.
|
||||
private _frameRunner = new LatestValueRunner<Blob>((frame) =>
|
||||
this._presentFrame(frame),
|
||||
);
|
||||
|
||||
constructor(
|
||||
host: LitElement,
|
||||
getImageCallback: () => HTMLImageElement | null,
|
||||
options?: ImageSurfaceOptions,
|
||||
) {
|
||||
this._getImageCallback = getImageCallback;
|
||||
this._decoder = new OffscreenImage(options?.createImage);
|
||||
this._mediaPlayerController = new ImageMediaPlayerController(
|
||||
host,
|
||||
getImageCallback,
|
||||
{
|
||||
livenessOptions: options?.livenessOptions,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
public getElement(): HTMLImageElement | null {
|
||||
return this._getImageCallback();
|
||||
}
|
||||
|
||||
public getMediaPlayer(): ImageMediaPlayerController {
|
||||
return this._mediaPlayerController;
|
||||
}
|
||||
|
||||
// Show a new frame. A newer frame supersedes any still waiting to show; the
|
||||
// returned promise resolves once *a* frame has been presented.
|
||||
public showFrame(blob: Blob): Promise<void> {
|
||||
return this._frameRunner.submit(blob);
|
||||
}
|
||||
|
||||
private async _presentFrame(blob: Blob): Promise<void> {
|
||||
const image = this._getImageCallback();
|
||||
|
||||
// Drop frames for a detached element (e.g. a surface superseded
|
||||
// mid-teardown) so a retired surface never paints.
|
||||
if (!image || !image.isConnected) {
|
||||
return;
|
||||
}
|
||||
|
||||
const generation = this._generation.current();
|
||||
const url = URL.createObjectURL(blob);
|
||||
|
||||
// Decode the frame on the off-screen loader before showing it. Assigning an
|
||||
// undecoded object URL to the visible <img> makes WebKit repaint the
|
||||
// element empty until the decode completes, flashing the media background
|
||||
// between frames; decoding first lets the swap paint from cache (as it's
|
||||
// the same url) with no empty state.
|
||||
const decoder = this._decoder.get();
|
||||
decoder.src = url;
|
||||
try {
|
||||
await decoder.decode();
|
||||
} catch {
|
||||
// An undecodable frame is not fatal: drop it and keep the current one.
|
||||
URL.revokeObjectURL(url);
|
||||
return;
|
||||
}
|
||||
|
||||
// The surface may have been switched (reset) or detached during the decode;
|
||||
// painting now would show a stale frame on a retired surface.
|
||||
if (!this._generation.isCurrent(generation) || !image.isConnected) {
|
||||
URL.revokeObjectURL(url);
|
||||
return;
|
||||
}
|
||||
|
||||
image.src = url;
|
||||
|
||||
const previousURL = this._currentObjectURL;
|
||||
this._currentObjectURL = url;
|
||||
if (previousURL) {
|
||||
URL.revokeObjectURL(previousURL);
|
||||
}
|
||||
}
|
||||
|
||||
// Drop the current frame and its object URL (e.g. on a surface switch or
|
||||
// disconnect).
|
||||
public reset(): void {
|
||||
// Invalidate any frame still decoding so it cannot paint onto a surface
|
||||
// that has since been switched or disconnected.
|
||||
this._generation.invalidate();
|
||||
|
||||
// Drop any frame still waiting to show, for the same reason.
|
||||
this._frameRunner.clear();
|
||||
|
||||
if (this._currentObjectURL) {
|
||||
URL.revokeObjectURL(this._currentObjectURL);
|
||||
this._currentObjectURL = null;
|
||||
}
|
||||
this._decoder.clear();
|
||||
|
||||
const image = this._getImageCallback();
|
||||
if (image) {
|
||||
image.removeAttribute('src');
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
type ImageElementFactory = () => HTMLImageElement;
|
||||
|
||||
// A held, injectable off-screen <img> for decoding a frame separate from the
|
||||
// visible display element. Created lazily on first `get()` and reused until
|
||||
// `clear()` detaches its source and forgets it.
|
||||
export class OffscreenImage {
|
||||
private _create: ImageElementFactory;
|
||||
private _image: HTMLImageElement | null = null;
|
||||
|
||||
constructor(create?: ImageElementFactory) {
|
||||
this._create = create ?? (() => new Image());
|
||||
}
|
||||
|
||||
// Return the held image, creating it on first use.
|
||||
public get(): HTMLImageElement {
|
||||
return (this._image ??= this._create());
|
||||
}
|
||||
|
||||
// Detach any source and forget the image. Safe to call when none is held.
|
||||
public clear(): void {
|
||||
if (this._image) {
|
||||
this._image.removeAttribute('src');
|
||||
this._image = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
export type VideoElementFactory = () => HTMLVideoElement;
|
||||
|
||||
// A held, injectable off-screen <video> for decoding a stream separate from the
|
||||
// real display element. Created lazily on first `get()` and reused until
|
||||
// `clear()` detaches its media and forgets it.
|
||||
export class OffscreenVideo {
|
||||
private _create: VideoElementFactory;
|
||||
private _video: HTMLVideoElement | null = null;
|
||||
|
||||
constructor(create?: VideoElementFactory) {
|
||||
this._create = create ?? (() => document.createElement('video'));
|
||||
}
|
||||
|
||||
// Return the held video, creating it on first use.
|
||||
public get(): HTMLVideoElement {
|
||||
return (this._video ??= this._create());
|
||||
}
|
||||
|
||||
// Detach any media (whether attached via `src` or `srcObject`) and forget the
|
||||
// video. Safe to call when none is held.
|
||||
public clear(): void {
|
||||
if (this._video) {
|
||||
this._video.removeAttribute('src');
|
||||
this._video.srcObject = null;
|
||||
this._video = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,721 @@
|
||||
import { isEqual } from 'lodash-es';
|
||||
|
||||
import { GO2RTC_MODES, type Go2RTCMode } from '../../../../config/schema/cameras';
|
||||
import type { CardWideConfig } from '../../../../config/schema/types';
|
||||
import type {
|
||||
MediaPlayerController,
|
||||
UntargetedMediaLoadedInfo,
|
||||
} from '../../../../types';
|
||||
import {
|
||||
addAudioTracksMuteStateListener,
|
||||
type AudioTracksMuteStateCleanup,
|
||||
} from '../../../../utils/audio';
|
||||
import {
|
||||
hideMediaControlsTemporarily,
|
||||
MEDIA_LOAD_CONTROLS_HIDE_SECONDS,
|
||||
} from '../../../../utils/controls';
|
||||
import { log } from '../../../../utils/debug';
|
||||
import { createMediaLoadedInfo } from '../../../../utils/media-info';
|
||||
import { RetryTimer } from '../../../../utils/retry-timer';
|
||||
import { convertToWebSocketURL } from '../../../../utils/websocket-url';
|
||||
import type { MediaSourceFactory } from './adapters/media-source';
|
||||
import type { PeerConnectionFactory } from './adapters/peer-connection';
|
||||
import { OffscreenVideo, type VideoElementFactory } from './offscreen-video';
|
||||
import { SignalingChannel, type WebSocketFactory } from './signaling';
|
||||
import {
|
||||
createBinarySource,
|
||||
createWebRTCSource,
|
||||
type BinarySource,
|
||||
type BinarySourceFactory,
|
||||
type WebRTCSourceFactory,
|
||||
} from './sources/factory';
|
||||
import type { MediaStreamFactory, WebRTCStreamSource } from './sources/webrtc';
|
||||
import type {
|
||||
Lane,
|
||||
StreamSource,
|
||||
StreamSourceContext,
|
||||
StreamSourceFailureReason,
|
||||
SurfaceKind,
|
||||
VideoStreamTarget,
|
||||
} from './types';
|
||||
import { getPreferredSource } from './utils/source-priority';
|
||||
|
||||
// Preference order for modes that stream binary media over the WebSocket; the
|
||||
// protocol permits only one such mode per connection, tried in this order with
|
||||
// fallback (real video first with MSE, then the MP4/MJPEG image fallbacks).
|
||||
// WebRTC is not here: it carries no WebSocket binary and runs in parallel with
|
||||
// the chosen binary mode. The configured modes select membership, not order.
|
||||
const BINARY_MODE_PRECEDENCE: readonly Go2RTCMode[] = ['mse', 'mp4', 'mjpeg'];
|
||||
|
||||
// A dropped connection is retried in place a few times, quickly, so a transient
|
||||
// blip (e.g. a brief network drop on an MSE stream) heals with no full player
|
||||
// remount (e.g. without the card IssueManager). After a short run of failures
|
||||
// the stream is treated as down: the failure is reported upward, handing
|
||||
// recovery to the card's managed retry (which shows a reconnecting indicator,
|
||||
// backs off, and can give up) instead of looping here silently. The attempt cap
|
||||
// keeps a persistently-failing stream from hammering the server.
|
||||
const RECONNECT_INTERVAL_SECONDS = 2;
|
||||
const RECONNECT_MAX_ATTEMPTS = 3;
|
||||
|
||||
// The two render surfaces the session commits onto. The component owns the
|
||||
// elements and the media-player controllers; the session drives which one is
|
||||
// live and reports it back via surfaceCommittedCallback.
|
||||
export interface VideoSurface {
|
||||
getElement(): HTMLVideoElement | null;
|
||||
getMediaPlayer(): MediaPlayerController;
|
||||
}
|
||||
|
||||
export interface ImageSurface {
|
||||
getElement(): HTMLImageElement | null;
|
||||
getMediaPlayer(): MediaPlayerController;
|
||||
|
||||
// Frames are handed over as Blobs through this callback rather than the
|
||||
// session setting `img.src` itself. The implementation (ImageSurfaceController)
|
||||
// creates each frame's object URL and revokes the previous one; the session
|
||||
// only ever holds Blobs, so it cannot leak URLs. The returned promise resolves
|
||||
// once the frame has decoded (real dimensions are then available).
|
||||
showFrame(frame: Blob): Promise<void>;
|
||||
reset(): void;
|
||||
}
|
||||
|
||||
export interface SessionSurfaces {
|
||||
video: VideoSurface;
|
||||
image: ImageSurface;
|
||||
}
|
||||
|
||||
interface Go2RTCSessionCallbacks {
|
||||
getControls: () => boolean;
|
||||
getCardWideConfig: () => CardWideConfig | null;
|
||||
mediaLoadedCallback: (info: UntargetedMediaLoadedInfo) => void;
|
||||
|
||||
// The lane that just committed is live on this surface (the component shows
|
||||
// it and hides the other).
|
||||
surfaceCommittedCallback: (surface: SurfaceKind) => void;
|
||||
|
||||
// The session has exhausted its own quick reconnects and cannot recover the
|
||||
// stream; a higher level should take over (e.g. the card's media-load retry).
|
||||
// The reason is the most recent source failure, or null when there is none
|
||||
// (e.g. the socket dropped with no source having reported a cause).
|
||||
errorCallback: (reason: StreamSourceFailureReason | null) => void;
|
||||
}
|
||||
|
||||
// Injectable platform and factory seams for tests. Every field defaults to
|
||||
// the real browser implementation; production passes none of them.
|
||||
interface Go2RTCSessionOptions {
|
||||
createBinarySource?: BinarySourceFactory;
|
||||
createWebRTCSource?: WebRTCSourceFactory;
|
||||
createWebSocket?: WebSocketFactory;
|
||||
createPeerConnection?: PeerConnectionFactory;
|
||||
createMediaStream?: MediaStreamFactory;
|
||||
createVideoElement?: VideoElementFactory;
|
||||
createMediaSource?: MediaSourceFactory;
|
||||
|
||||
userAgent?: string;
|
||||
}
|
||||
|
||||
// One WebSocket connection attempt: its channel and URL, plus a reference to
|
||||
// the render surfaces to draw onto (the surfaces outlive any single attempt).
|
||||
// Handlers capture this rather than reading session fields so a stale
|
||||
// connection's events cannot act on a newer connection.
|
||||
interface ConnectionContext {
|
||||
channel: SignalingChannel;
|
||||
url: string;
|
||||
surfaces: SessionSurfaces;
|
||||
}
|
||||
|
||||
// Coordinates one go2rtc streaming session. A "binary" lane (MSE, MP4/MJPEG)
|
||||
// plays on the real video with sequential fallback; a parallel WebRTC lane
|
||||
// decodes into an off-screen video element and, if its stream scores
|
||||
// higher, takes over the real video. The session owns the signaling channel, the
|
||||
// bounded fast reconnects, and reporting loaded media.
|
||||
export class Go2RTCSessionController {
|
||||
private _callbacks: Go2RTCSessionCallbacks;
|
||||
private _options: Go2RTCSessionOptions | null;
|
||||
|
||||
private _channel: SignalingChannel | null = null;
|
||||
private _url: string | null = null;
|
||||
private _surfaces: SessionSurfaces | null = null;
|
||||
private _modes: readonly Go2RTCMode[] = GO2RTC_MODES;
|
||||
private _microphoneStream: MediaStream | null = null;
|
||||
|
||||
// Binary lane: the active source paired with the surface it renders on (mse
|
||||
// -> video, MP4/MJPEG -> image). Kept as one unit because the factory returns
|
||||
// them together and they are read together when the source commits. Null when
|
||||
// no binary source is running.
|
||||
private _binary: BinarySource | null = null;
|
||||
|
||||
// Queue of fallback modes not yet tried on this connection, in precedence
|
||||
// order. Starting a binary source consumes the head; teardown empties the
|
||||
// queue so no further fallback is attempted.
|
||||
private _binaryModes: Go2RTCMode[] = [];
|
||||
|
||||
// WebRTC lane.
|
||||
private _webRTCSource: WebRTCStreamSource | null = null;
|
||||
|
||||
// Holds the off-screen video element the WebRTC stream decodes into while the
|
||||
// WebRTC lane races a binary lane.
|
||||
private _offscreenVideo: OffscreenVideo;
|
||||
|
||||
// The committed lane: the one whose media is live (on the video or image
|
||||
// surface) and has reported as loaded; null before anything loads or once the
|
||||
// lanes are torn down. Failure handling reads it to distinguish a committed
|
||||
// (live) stream dying from a losing racer being dropped.
|
||||
private _committedLane: Lane | null = null;
|
||||
|
||||
// The surface currently showing committed media; null before anything commits
|
||||
// or once torn down. On a surface switch (e.g. MSE falls back to MP4, or a
|
||||
// WebRTC win over an MJPEG image) the outgoing surface is reset.
|
||||
private _committedSurface: SurfaceKind | null = null;
|
||||
|
||||
// The source whose media is committed; null before anything commits or once
|
||||
// torn down. A source can re-report loaded (e.g. a <video> re-fires
|
||||
// loadeddata on a mid-stream resolution change), so this distinguishes a
|
||||
// fresh commit from a repeated load.
|
||||
private _committedSource: StreamSource | null = null;
|
||||
|
||||
// Unsubscriber for the listener on the committed WebRTC stream's audio
|
||||
// tracks. Should a WebRTC track's mute state change after load, the listener
|
||||
// sends the card a fresh media-loaded info so its audio capabilities reflect
|
||||
// the stream's current audio, not the state at load time. Null when no WebRTC
|
||||
// stream is committed; invoked and cleared on teardown and before re-arming
|
||||
// so a retired stream's listener can never fire.
|
||||
private _audioTracksMuteStateUnsubscribeCallback: AudioTracksMuteStateCleanup = null;
|
||||
|
||||
// The most recent source failure on this connection, handed to the error
|
||||
// callback when the session finally gives up so the card can name the cause.
|
||||
// Null before any failure and after a healthy commit.
|
||||
private _lastFailureReason: StreamSourceFailureReason | null = null;
|
||||
|
||||
private _retryTimer = new RetryTimer(RECONNECT_INTERVAL_SECONDS);
|
||||
|
||||
constructor(callbacks: Go2RTCSessionCallbacks, options?: Go2RTCSessionOptions) {
|
||||
this._callbacks = callbacks;
|
||||
this._options = options ?? null;
|
||||
|
||||
this._offscreenVideo = new OffscreenVideo(this._options?.createVideoElement);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Public API.
|
||||
// ===========================================================================
|
||||
|
||||
// The `surfaces` object identifies the session target: the same object means
|
||||
// "keep the established session" (so callers may invoke this on every render),
|
||||
// a new object means "reset and reconnect". Callers must therefore hold one
|
||||
// stable surfaces object for as long as the target is unchanged, and hand over
|
||||
// a fresh one (or `reset()` first) if they ever remount the underlying
|
||||
// elements.
|
||||
public connect(url: string, surfaces: SessionSurfaces, modes?: Go2RTCMode[]): void {
|
||||
const normalizedModes: readonly Go2RTCMode[] = modes?.length ? modes : GO2RTC_MODES;
|
||||
|
||||
if (
|
||||
this._url === url &&
|
||||
this._surfaces === surfaces &&
|
||||
isEqual(this._modes, normalizedModes)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.reset();
|
||||
|
||||
this._url = url;
|
||||
this._surfaces = surfaces;
|
||||
this._modes = normalizedModes;
|
||||
|
||||
this._connectChannel(url, surfaces);
|
||||
}
|
||||
|
||||
public reset(): void {
|
||||
this._retryTimer.reset();
|
||||
this._lastFailureReason = null;
|
||||
|
||||
this._teardownLanes();
|
||||
this._channel?.close();
|
||||
this._channel = null;
|
||||
|
||||
const video = this._surfaces?.video.getElement();
|
||||
if (video) {
|
||||
video.srcObject = null;
|
||||
video.src = '';
|
||||
}
|
||||
this._surfaces?.image.reset();
|
||||
|
||||
this._url = null;
|
||||
this._surfaces = null;
|
||||
}
|
||||
|
||||
public setMicrophoneStream(stream: MediaStream | null): void {
|
||||
this._microphoneStream = stream;
|
||||
this._webRTCSource?.setMicrophoneStream(stream).catch(() => {});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Session teardown.
|
||||
// ===========================================================================
|
||||
|
||||
private _teardownLanes(): void {
|
||||
this._teardownBinaryLane();
|
||||
this._teardownWebRTCLane();
|
||||
this._committedLane = null;
|
||||
this._committedSurface = null;
|
||||
this._committedSource = null;
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Signaling channel lifecycle.
|
||||
// ===========================================================================
|
||||
|
||||
private _connectChannel(url: string, surfaces: SessionSurfaces): void {
|
||||
// The open/close callbacks reference `channel`, the very constant this
|
||||
// `new` expression is being assigned to. That is safe because the callbacks
|
||||
// fire on asynchronous WebSocket events, which cannot happen before the
|
||||
// assignment completes.
|
||||
const channel: SignalingChannel = new SignalingChannel(
|
||||
convertToWebSocketURL(url),
|
||||
{
|
||||
openCallback: () => this._handleOpen({ channel, url, surfaces }),
|
||||
disconnectCallback: () => this._handleClose({ channel, url, surfaces }),
|
||||
},
|
||||
{ createWebSocket: this._options?.createWebSocket },
|
||||
);
|
||||
this._channel = channel;
|
||||
channel.connect();
|
||||
}
|
||||
|
||||
private _handleOpen(context: ConnectionContext): void {
|
||||
this._binaryModes = BINARY_MODE_PRECEDENCE.filter((mode) =>
|
||||
this._modes.includes(mode),
|
||||
);
|
||||
|
||||
// While a binary lane races, WebRTC decodes into a separate off-screen video
|
||||
// element (see OffscreenVideo) where its stream can be evaluated without
|
||||
// contending for the real <video> that an MSE binary lane plays on. With no
|
||||
// binary lane the real video element is free, so WebRTC attaches to it
|
||||
// directly; that also avoids decoding into an element outside the document,
|
||||
// which has been implicated in Firefox WebRTC failures. See
|
||||
// https://github.com/dermotduffy/advanced-camera-card/issues/2222
|
||||
//
|
||||
// Every configured mode is a binary mode or webrtc, and empty modes default
|
||||
// to all, so at least one lane always starts.
|
||||
//
|
||||
// Start WebRTC before the binary lane: starting a binary source can fail
|
||||
// synchronously (e.g. an unsupported codec drains the whole mode queue),
|
||||
// which would trigger a premature reconnect while no lane is live yet and
|
||||
// leave a stale WebRTC source running on a closed channel. Starting WebRTC
|
||||
// first means that reconnect check always sees a live lane.
|
||||
const hasBinaryModes = this._binaryModes.length > 0;
|
||||
if (this._modes.includes('webrtc')) {
|
||||
this._startWebRTCSource(context, hasBinaryModes);
|
||||
}
|
||||
if (hasBinaryModes) {
|
||||
this._startNextBinarySource(context);
|
||||
}
|
||||
}
|
||||
|
||||
private _handleClose(context: ConnectionContext): void {
|
||||
this._teardownLanes();
|
||||
this._channel = null;
|
||||
this._reconnectOrEscalateError(context);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Binary lane.
|
||||
// ===========================================================================
|
||||
|
||||
private _startNextBinarySource(context: ConnectionContext): void {
|
||||
const mode = this._binaryModes.shift();
|
||||
if (!mode) {
|
||||
this._maybeReconnectIfLanesDead(context);
|
||||
return;
|
||||
}
|
||||
|
||||
// No element to render onto (the surface was detached); abandon the lane.
|
||||
const video = context.surfaces.video.getElement();
|
||||
if (!video) {
|
||||
return;
|
||||
}
|
||||
|
||||
// The callbacks capture the source's own identity so a retired source
|
||||
// (stopped, replaced or reset) cannot act on the session. The variable is
|
||||
// declared before the factory call so callbacks fired during construction
|
||||
// see null and are ignored.
|
||||
let source: StreamSource | null = null;
|
||||
|
||||
const binarySource = (this._options?.createBinarySource ?? createBinarySource)(
|
||||
mode,
|
||||
{
|
||||
video: { kind: 'video', video },
|
||||
image: {
|
||||
kind: 'image',
|
||||
showFrame: (frame) => context.surfaces.image.showFrame(frame),
|
||||
},
|
||||
},
|
||||
context.channel,
|
||||
{
|
||||
loadedCallback: () => {
|
||||
if (source) {
|
||||
this._handleBinaryLoaded(context, source);
|
||||
}
|
||||
},
|
||||
failedCallback: (reason: StreamSourceFailureReason) => {
|
||||
if (source) {
|
||||
this._lastFailureReason = reason;
|
||||
this._logSourceFailure('binary', reason, mode);
|
||||
this._handleBinaryFailed(context, source);
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
createMediaSource: this._options?.createMediaSource,
|
||||
userAgent: this._options?.userAgent,
|
||||
},
|
||||
);
|
||||
if (!binarySource) {
|
||||
this._startNextBinarySource(context);
|
||||
return;
|
||||
}
|
||||
|
||||
source = binarySource.source;
|
||||
this._binary = binarySource;
|
||||
source.start();
|
||||
}
|
||||
|
||||
private _handleBinaryLoaded(context: ConnectionContext, source: StreamSource): void {
|
||||
const binary = this._binary;
|
||||
if (!binary || source !== binary.source) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._committedLane = 'binary';
|
||||
this._reportCommittedLoad(context, binary.surface, source);
|
||||
}
|
||||
|
||||
private _handleBinaryFailed(context: ConnectionContext, source: StreamSource): void {
|
||||
if (source !== this._binary?.source) {
|
||||
return;
|
||||
}
|
||||
|
||||
source.stop();
|
||||
this._binary = null;
|
||||
|
||||
if (this._committedLane === 'binary') {
|
||||
this._committedLane = null;
|
||||
}
|
||||
|
||||
this._startNextBinarySource(context);
|
||||
}
|
||||
|
||||
private _teardownBinaryLane(): void {
|
||||
// Clear the current-source identity before stopping, so any callback fired
|
||||
// synchronously by stop() fails its `!== this._binary?.source` guard.
|
||||
const source = this._binary?.source;
|
||||
this._binary = null;
|
||||
this._binaryModes = [];
|
||||
|
||||
source?.stop();
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// WebRTC lane.
|
||||
// ===========================================================================
|
||||
|
||||
private _startWebRTCSource(
|
||||
context: ConnectionContext,
|
||||
useOffscreenVideo: boolean,
|
||||
): void {
|
||||
// While racing a binary lane, decode into the off-screen element; otherwise
|
||||
// decode straight onto the real video (absent it, there is nothing to do).
|
||||
let decodeVideo: HTMLVideoElement;
|
||||
if (useOffscreenVideo) {
|
||||
decodeVideo = this._offscreenVideo.get();
|
||||
} else {
|
||||
const video = context.surfaces.video.getElement();
|
||||
if (!video) {
|
||||
return;
|
||||
}
|
||||
decodeVideo = video;
|
||||
}
|
||||
|
||||
let source: WebRTCStreamSource | null = null;
|
||||
const sourceContext: StreamSourceContext<VideoStreamTarget> = {
|
||||
target: { kind: 'video', video: decodeVideo },
|
||||
channel: context.channel,
|
||||
callbacks: {
|
||||
loadedCallback: () => {
|
||||
if (source) {
|
||||
this._handleWebRTCLoaded(context, source, useOffscreenVideo, decodeVideo);
|
||||
}
|
||||
},
|
||||
failedCallback: (reason) => {
|
||||
if (source) {
|
||||
this._lastFailureReason = reason;
|
||||
this._logSourceFailure('webrtc', reason);
|
||||
this._handleWebRTCFailed(context, source);
|
||||
}
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
source = (this._options?.createWebRTCSource ?? createWebRTCSource)(sourceContext, {
|
||||
microphoneStream: this._microphoneStream,
|
||||
createPeerConnection: this._options?.createPeerConnection,
|
||||
createMediaStream: this._options?.createMediaStream,
|
||||
});
|
||||
|
||||
this._webRTCSource = source;
|
||||
source.start();
|
||||
}
|
||||
|
||||
private _handleWebRTCLoaded(
|
||||
context: ConnectionContext,
|
||||
source: WebRTCStreamSource,
|
||||
useOffscreenVideo: boolean,
|
||||
decodeVideo: HTMLVideoElement,
|
||||
): void {
|
||||
if (source !== this._webRTCSource) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!useOffscreenVideo) {
|
||||
// There is nothing in the 'binary' lane, this controller is WebRTC-only:
|
||||
// the stream already decoded on the real video element.
|
||||
this._commitWebRTC(context, source, decodeVideo);
|
||||
return;
|
||||
}
|
||||
|
||||
// There is video on both the 'binary' and 'webrtc' lane -- choose a winner.
|
||||
const binary = this._binary?.source ?? null;
|
||||
const winner = binary
|
||||
? getPreferredSource(source.getStreamProfile(), binary.getStreamProfile())
|
||||
: 'webrtc';
|
||||
|
||||
if (winner === 'binary') {
|
||||
// The binary source scored higher: keep it on the real video element and
|
||||
// tear down the WebRTC lane.
|
||||
this._teardownWebRTCLane();
|
||||
return;
|
||||
}
|
||||
|
||||
// WebRTC wins. Tear down the binary lane first: stopping the binary source
|
||||
// clears the real video element it was playing on, so the WebRTC stream is
|
||||
// attached to that element (below) only after, never before.
|
||||
this._teardownBinaryLane();
|
||||
const stream = source.getMediaStream();
|
||||
const video = context.surfaces.video.getElement();
|
||||
if (video && stream) {
|
||||
video.srcObject = stream;
|
||||
}
|
||||
|
||||
// Report the media-loaded frame size from the off-screen video element
|
||||
// (decodeVideo): it has been decoding this stream throughout the
|
||||
// lane-choice so it already knows the size, whereas the real video element
|
||||
// was only handed the stream on the line above and has not decoded a frame
|
||||
// yet (so it would report 0x0).
|
||||
this._commitWebRTC(context, source, decodeVideo);
|
||||
this._offscreenVideo.clear();
|
||||
}
|
||||
|
||||
private _handleWebRTCFailed(
|
||||
context: ConnectionContext,
|
||||
source: WebRTCStreamSource,
|
||||
): void {
|
||||
if (source !== this._webRTCSource) {
|
||||
return;
|
||||
}
|
||||
|
||||
const wasCommitted = this._committedLane === 'webrtc';
|
||||
this._teardownWebRTCLane();
|
||||
|
||||
if (wasCommitted) {
|
||||
// WebRTC was the committed (live) lane, so losing it leaves the view with
|
||||
// nothing playing: clear the real video element and force a full
|
||||
// reconnect (which restarts every configured mode, letting a binary lane
|
||||
// take over) or escalate via the error callback.
|
||||
this._committedLane = null;
|
||||
const video = context.surfaces.video.getElement();
|
||||
if (video) {
|
||||
video.srcObject = null;
|
||||
}
|
||||
|
||||
this._closeChannelAndReconnect(context);
|
||||
} else {
|
||||
// WebRTC lost a race it was still contesting; the binary lane carries on
|
||||
// if present, otherwise reconnect.
|
||||
this._maybeReconnectIfLanesDead(context);
|
||||
}
|
||||
}
|
||||
|
||||
private _commitWebRTC(
|
||||
context: ConnectionContext,
|
||||
source: WebRTCStreamSource,
|
||||
dimensionsVideo: HTMLVideoElement,
|
||||
): void {
|
||||
this._committedLane = 'webrtc';
|
||||
|
||||
// WebRTC now carries the media over the peer connection, so the socket is
|
||||
// done: signaling (offer/answer/ICE) is complete and any binary lane's
|
||||
// local consumer is torn down. go2rtc has no "stop" message, though --
|
||||
// tearing a lane down only detaches our side, so the server keeps muxing
|
||||
// that mode's binary frames down the socket (onto a now-discarded consumer)
|
||||
// until the socket itself closes. Closing it is what actually ends that
|
||||
// server stream.
|
||||
this._channel?.close();
|
||||
this._channel = null;
|
||||
|
||||
this._setupAudioMuteRedispatch(context, source);
|
||||
this._reportCommittedLoad(context, 'video', source, dimensionsVideo);
|
||||
}
|
||||
|
||||
private _setupAudioMuteRedispatch(
|
||||
context: ConnectionContext,
|
||||
source: WebRTCStreamSource,
|
||||
): void {
|
||||
this._audioTracksMuteStateUnsubscribeCallback?.();
|
||||
|
||||
// The listener is removed on teardown, so a stale source can never fire it.
|
||||
// Audio mute/unmute re-dispatches read the real video, which holds the
|
||||
// stream (and its dimensions) by the time any such change occurs.
|
||||
this._audioTracksMuteStateUnsubscribeCallback = addAudioTracksMuteStateListener(
|
||||
source.getPeerConnection(),
|
||||
() => this._dispatchMediaLoaded(context, 'video', source),
|
||||
);
|
||||
}
|
||||
|
||||
private _teardownWebRTCLane(): void {
|
||||
// Clear the current-source identity and the audio listener before stopping,
|
||||
// so any callback fired synchronously by stop() fails its guard or is gone.
|
||||
const source = this._webRTCSource;
|
||||
this._webRTCSource = null;
|
||||
|
||||
this._audioTracksMuteStateUnsubscribeCallback?.();
|
||||
this._audioTracksMuteStateUnsubscribeCallback = null;
|
||||
|
||||
this._offscreenVideo.clear();
|
||||
|
||||
source?.stop();
|
||||
}
|
||||
|
||||
// A source in one of the lanes failed.
|
||||
private _logSourceFailure(
|
||||
lane: Lane,
|
||||
reason: StreamSourceFailureReason,
|
||||
mode?: Go2RTCMode,
|
||||
): void {
|
||||
log(this._callbacks.getCardWideConfig(), 'go2rtc-experimental source failed', {
|
||||
lane,
|
||||
...(mode ? { mode } : {}),
|
||||
reason,
|
||||
});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Reconnect & escalation.
|
||||
// ===========================================================================
|
||||
|
||||
private _maybeReconnectIfLanesDead(context: ConnectionContext): void {
|
||||
if (!this._binary && !this._webRTCSource && !this._binaryModes.length) {
|
||||
this._closeChannelAndReconnect(context);
|
||||
}
|
||||
}
|
||||
|
||||
// Abandon this connection's channel and start the reconnect-or-escalate flow.
|
||||
private _closeChannelAndReconnect(context: ConnectionContext): void {
|
||||
context.channel.close();
|
||||
this._channel = null;
|
||||
|
||||
this._reconnectOrEscalateError(context);
|
||||
}
|
||||
|
||||
private _reconnectOrEscalateError(context: ConnectionContext): void {
|
||||
if (this._retryTimer.getAttempts() >= RECONNECT_MAX_ATTEMPTS) {
|
||||
this._callbacks.errorCallback(this._lastFailureReason);
|
||||
return;
|
||||
}
|
||||
this._retryTimer.schedule(() => this._connectChannel(context.url, context.surfaces));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Media-loaded reporting.
|
||||
// ===========================================================================
|
||||
|
||||
// A lane just committed with live media: the connection has proven healthy, so
|
||||
// reset the fast-reconnect budget, hand the newly live surface to the
|
||||
// component (resetting the one being left behind), briefly hide the video
|
||||
// controls so they do not flash over the freshly loaded media, then report the
|
||||
// load with the committed surface's controller.
|
||||
private _reportCommittedLoad(
|
||||
context: ConnectionContext,
|
||||
surface: SurfaceKind,
|
||||
source: StreamSource,
|
||||
dimensionsVideo?: HTMLVideoElement,
|
||||
): void {
|
||||
if (source === this._committedSource) {
|
||||
// The same source re-reporting (e.g. a mid-stream resolution change
|
||||
// re-fires the video's loadeddata): the surface and controls are already
|
||||
// set up, so only refresh the reported media dimensions. Re-running the
|
||||
// control hide would leak its loadstart listener on each repeat.
|
||||
this._dispatchMediaLoaded(context, surface, source, dimensionsVideo);
|
||||
return;
|
||||
}
|
||||
this._committedSource = source;
|
||||
|
||||
this._retryTimer.reset();
|
||||
this._lastFailureReason = null;
|
||||
|
||||
if (this._committedSurface && this._committedSurface !== surface) {
|
||||
this._resetSurface(context, this._committedSurface);
|
||||
}
|
||||
this._committedSurface = surface;
|
||||
this._callbacks.surfaceCommittedCallback(surface);
|
||||
|
||||
const video = context.surfaces.video.getElement();
|
||||
|
||||
if (surface === 'video' && this._callbacks.getControls() && video) {
|
||||
hideMediaControlsTemporarily(video, MEDIA_LOAD_CONTROLS_HIDE_SECONDS);
|
||||
}
|
||||
|
||||
this._dispatchMediaLoaded(context, surface, source, dimensionsVideo);
|
||||
}
|
||||
|
||||
private _resetSurface(context: ConnectionContext, surface: SurfaceKind): void {
|
||||
if (surface === 'image') {
|
||||
context.surfaces.image.reset();
|
||||
return;
|
||||
}
|
||||
const video = context.surfaces.video.getElement();
|
||||
if (video) {
|
||||
video.srcObject = null;
|
||||
video.src = '';
|
||||
}
|
||||
}
|
||||
|
||||
private _dispatchMediaLoaded(
|
||||
context: ConnectionContext,
|
||||
surface: SurfaceKind,
|
||||
source: StreamSource,
|
||||
// Element to read the frame size from; used only for the video surface and
|
||||
// defaults to the video-surface element. Differs when that element was just
|
||||
// handed a stream and has not decoded a frame yet (WebRTC committed from the
|
||||
// off-screen video), so its size must come from that off-screen element.
|
||||
dimensionsVideo?: HTMLVideoElement,
|
||||
): void {
|
||||
const targetSurface =
|
||||
surface === 'video' ? context.surfaces.video : context.surfaces.image;
|
||||
const dimensionsElement =
|
||||
surface === 'video'
|
||||
? dimensionsVideo ?? context.surfaces.video.getElement()
|
||||
: context.surfaces.image.getElement();
|
||||
if (!dimensionsElement) {
|
||||
return;
|
||||
}
|
||||
|
||||
const info = createMediaLoadedInfo(dimensionsElement, {
|
||||
mediaPlayerController: targetSurface.getMediaPlayer(),
|
||||
capabilities: source.getCapabilities(),
|
||||
technology: source.getTechnology(),
|
||||
});
|
||||
if (info) {
|
||||
this._callbacks.mediaLoadedCallback(info);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
import type { UnsubscribeCallback } from '../../../../types';
|
||||
import {
|
||||
go2RTCMessageSchema,
|
||||
type BinaryCallback,
|
||||
type Go2RTCMessage,
|
||||
type MessageCallback,
|
||||
} from './types';
|
||||
|
||||
export type WebSocketFactory = (url: string) => WebSocket;
|
||||
|
||||
interface SignalingChannelCallbacks {
|
||||
openCallback?: () => void;
|
||||
|
||||
// Fires when the socket disconnects (i.e. not intentional `close()`).
|
||||
disconnectCallback?: () => void;
|
||||
}
|
||||
|
||||
// Injectable test seam. Defaults to the real `WebSocket`; production passes
|
||||
// nothing here.
|
||||
interface SignalingChannelOptions {
|
||||
createWebSocket?: WebSocketFactory;
|
||||
}
|
||||
|
||||
// Owns the go2rtc WebSocket: JSON text frames are parsed and fanned out to
|
||||
// message subscribers, binary frames go to the single binary consumer. No
|
||||
// callbacks of any kind are delivered after `close()`.
|
||||
export class SignalingChannel {
|
||||
private _url: string;
|
||||
|
||||
private _ws: WebSocket | null = null;
|
||||
|
||||
private _open = false;
|
||||
|
||||
private _callbacks: SignalingChannelCallbacks;
|
||||
private _createWebSocket: WebSocketFactory;
|
||||
private _binaryCallback: BinaryCallback | null = null;
|
||||
|
||||
private _messageCallbacks = new Set<MessageCallback>();
|
||||
|
||||
constructor(
|
||||
url: string,
|
||||
callbacks: SignalingChannelCallbacks,
|
||||
options?: SignalingChannelOptions,
|
||||
) {
|
||||
this._url = url;
|
||||
this._callbacks = callbacks;
|
||||
this._createWebSocket =
|
||||
options?.createWebSocket ?? ((wsURL: string) => new WebSocket(wsURL));
|
||||
}
|
||||
|
||||
public connect(): void {
|
||||
if (this._ws) {
|
||||
return;
|
||||
}
|
||||
|
||||
const ws = this._createWebSocket(this._url);
|
||||
this._ws = ws;
|
||||
|
||||
ws.binaryType = 'arraybuffer';
|
||||
|
||||
// Each listener ignores events once this websocket is no longer current
|
||||
// (e.g. events already queued when `close()` was called).
|
||||
ws.addEventListener('open', () => {
|
||||
if (this._ws !== ws) {
|
||||
return;
|
||||
}
|
||||
this._open = true;
|
||||
this._callbacks.openCallback?.();
|
||||
});
|
||||
|
||||
ws.addEventListener('close', () => {
|
||||
if (this._ws !== ws) {
|
||||
return;
|
||||
}
|
||||
this._ws = null;
|
||||
this._open = false;
|
||||
|
||||
// This will not be called when we intentionally call close(), as the
|
||||
// guard above will have nulled out this._ws.
|
||||
this._callbacks.disconnectCallback?.();
|
||||
});
|
||||
|
||||
ws.addEventListener('message', (ev) => {
|
||||
if (this._ws !== ws) {
|
||||
return;
|
||||
}
|
||||
this._handleMessage(ev.data);
|
||||
});
|
||||
}
|
||||
|
||||
public close(): void {
|
||||
const ws = this._ws;
|
||||
this._ws = null;
|
||||
this._open = false;
|
||||
|
||||
ws?.close();
|
||||
}
|
||||
|
||||
public isOpen(): boolean {
|
||||
return this._open;
|
||||
}
|
||||
|
||||
public send(message: Go2RTCMessage): void {
|
||||
if (this._ws && this._open) {
|
||||
this._ws.send(JSON.stringify(message));
|
||||
}
|
||||
}
|
||||
|
||||
public subscribeToMessages(callback: MessageCallback): UnsubscribeCallback {
|
||||
this._messageCallbacks.add(callback);
|
||||
|
||||
return () => {
|
||||
this._messageCallbacks.delete(callback);
|
||||
};
|
||||
}
|
||||
|
||||
public setBinaryCallback(callback: BinaryCallback | null): void {
|
||||
this._binaryCallback = callback;
|
||||
}
|
||||
|
||||
private _handleMessage(data: unknown): void {
|
||||
if (data instanceof ArrayBuffer) {
|
||||
this._binaryCallback?.(data);
|
||||
return;
|
||||
}
|
||||
|
||||
if (typeof data !== 'string') {
|
||||
return;
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(data);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
const result = go2RTCMessageSchema.safeParse(parsed);
|
||||
if (!result.success) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Iterate over a copy: a callback may subscribe or unsubscribe during
|
||||
// dispatch.
|
||||
for (const callback of [...this._messageCallbacks]) {
|
||||
callback(result.data);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
import type { Go2RTCMode } from '../../../../../config/schema/cameras';
|
||||
import type { MediaSourceFactory } from '../adapters/media-source';
|
||||
import type { PeerConnectionFactory } from '../adapters/peer-connection';
|
||||
import type {
|
||||
ImageStreamTarget,
|
||||
StreamSource,
|
||||
StreamSourceCallbacks,
|
||||
StreamSourceChannel,
|
||||
StreamSourceContext,
|
||||
SurfaceKind,
|
||||
VideoStreamTarget,
|
||||
} from '../types';
|
||||
import { MJPEGStreamSource } from './mjpeg';
|
||||
import { MP4StreamSource } from './mp4';
|
||||
import { MSEStreamSource } from './mse';
|
||||
import { WebRTCStreamSource, type MediaStreamFactory } from './webrtc';
|
||||
|
||||
// ===========================================================================
|
||||
// Binary lane
|
||||
// ===========================================================================
|
||||
|
||||
interface CreateBinarySourceOptions {
|
||||
createMediaSource?: MediaSourceFactory;
|
||||
userAgent?: string;
|
||||
}
|
||||
|
||||
// The available render targets, one of which each mode is wired to. The factory
|
||||
// is the single place that maps a mode to its surface, so it reports back which
|
||||
// one it chose.
|
||||
export interface BinaryStreamTargets {
|
||||
video: VideoStreamTarget;
|
||||
image: ImageStreamTarget;
|
||||
}
|
||||
|
||||
export interface BinarySource {
|
||||
source: StreamSource;
|
||||
surface: SurfaceKind;
|
||||
}
|
||||
|
||||
// Builds a source for a mode that streams binary media over the WebSocket (only
|
||||
// one such mode runs per connection). WebRTC is created separately via
|
||||
// `createWebRTCSource` because it runs in parallel and has its own interface.
|
||||
export type BinarySourceFactory = (
|
||||
mode: Go2RTCMode,
|
||||
targets: BinaryStreamTargets,
|
||||
channel: StreamSourceChannel,
|
||||
callbacks: StreamSourceCallbacks,
|
||||
options?: CreateBinarySourceOptions,
|
||||
) => BinarySource | null;
|
||||
|
||||
// Returns null for modes with no binary source (i.e. `webrtc`, handled
|
||||
// separately via `createWebRTCSource`).
|
||||
export const createBinarySource: BinarySourceFactory = (
|
||||
mode: Go2RTCMode,
|
||||
targets: BinaryStreamTargets,
|
||||
channel: StreamSourceChannel,
|
||||
callbacks: StreamSourceCallbacks,
|
||||
options?: CreateBinarySourceOptions,
|
||||
): BinarySource | null => {
|
||||
switch (mode) {
|
||||
case 'mse':
|
||||
return {
|
||||
source: new MSEStreamSource(
|
||||
{ target: targets.video, channel, callbacks },
|
||||
options,
|
||||
),
|
||||
surface: 'video',
|
||||
};
|
||||
case 'mp4':
|
||||
return {
|
||||
source: new MP4StreamSource({ target: targets.image, channel, callbacks }),
|
||||
surface: 'image',
|
||||
};
|
||||
case 'mjpeg':
|
||||
return {
|
||||
source: new MJPEGStreamSource({ target: targets.image, channel, callbacks }),
|
||||
surface: 'image',
|
||||
};
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
// ===========================================================================
|
||||
// WebRTC lane
|
||||
// ===========================================================================
|
||||
|
||||
export interface CreateWebRTCSourceOptions {
|
||||
createPeerConnection?: PeerConnectionFactory;
|
||||
createMediaStream?: MediaStreamFactory;
|
||||
microphoneStream?: MediaStream | null;
|
||||
}
|
||||
|
||||
export type WebRTCSourceFactory = (
|
||||
context: StreamSourceContext<VideoStreamTarget>,
|
||||
options?: CreateWebRTCSourceOptions,
|
||||
) => WebRTCStreamSource;
|
||||
|
||||
export const createWebRTCSource: WebRTCSourceFactory = (
|
||||
context: StreamSourceContext<VideoStreamTarget>,
|
||||
options?: CreateWebRTCSourceOptions,
|
||||
): WebRTCStreamSource => new WebRTCStreamSource(context, options);
|
||||
@@ -0,0 +1,112 @@
|
||||
import type { Go2RTCMode } from '../../../../../config/schema/cameras';
|
||||
import type {
|
||||
MediaLoadedCapabilities,
|
||||
MediaTechnology,
|
||||
UnsubscribeCallback,
|
||||
} from '../../../../../types';
|
||||
import { Timer } from '../../../../../utils/timer';
|
||||
import type {
|
||||
Go2RTCMessage,
|
||||
ImageStreamTarget,
|
||||
StreamProfile,
|
||||
StreamSource,
|
||||
StreamSourceContext,
|
||||
} from '../types';
|
||||
import { isServerErrorForMode } from '../utils/messages';
|
||||
|
||||
// Fail if no frame arrives within this window. The channel is open and the mode
|
||||
// was requested, but the server may send neither a frame nor an error, so
|
||||
// without this the source would hang silently instead of failing over. Mirrors
|
||||
// the MSE negotiation and WebRTC connect timeouts.
|
||||
const FIRST_FRAME_TIMEOUT_SECONDS = 5;
|
||||
|
||||
// Base for the modes that present a stream as a sequence of still images fed to
|
||||
// the image surface (MJPEG, MP4): both receive binary frames and turn each into
|
||||
// a single-image Blob, differing only in the request message and the per-frame
|
||||
// conversion. These are last-resort fallbacks: no audio, no "real" playback.
|
||||
export abstract class ImageFrameStreamSource implements StreamSource {
|
||||
protected _context: StreamSourceContext<ImageStreamTarget>;
|
||||
|
||||
private _loaded = false;
|
||||
private _stopped = false;
|
||||
|
||||
private _firstFrameTimer = new Timer();
|
||||
|
||||
private _unsubscribe: UnsubscribeCallback | null = null;
|
||||
|
||||
constructor(context: StreamSourceContext<ImageStreamTarget>) {
|
||||
this._context = context;
|
||||
}
|
||||
|
||||
protected abstract _mode: Go2RTCMode;
|
||||
protected abstract _getRequestMessage(): Go2RTCMessage;
|
||||
protected abstract _handleFrame(data: ArrayBuffer): void;
|
||||
|
||||
public start(): void {
|
||||
// Arm before wiring up the channel: a synchronous channel could deliver the
|
||||
// first frame during send(), which must be able to stop an already-armed
|
||||
// timer rather than have start() arm one that never stops.
|
||||
this._firstFrameTimer.start(FIRST_FRAME_TIMEOUT_SECONDS, () =>
|
||||
this._context.callbacks.failedCallback('connect_timeout'),
|
||||
);
|
||||
|
||||
this._unsubscribe = this._context.channel.subscribeToMessages((message) => {
|
||||
if (isServerErrorForMode(message, this._mode)) {
|
||||
this._context.callbacks.failedCallback('server_error');
|
||||
}
|
||||
});
|
||||
this._context.channel.setBinaryCallback((data) => this._handleFrame(data));
|
||||
this._context.channel.send(this._getRequestMessage());
|
||||
}
|
||||
|
||||
public stop(): void {
|
||||
this._stopped = true;
|
||||
this._firstFrameTimer.stop();
|
||||
this._unsubscribe?.();
|
||||
this._unsubscribe = null;
|
||||
this._context.channel.setBinaryCallback(null);
|
||||
|
||||
this._teardown();
|
||||
}
|
||||
|
||||
public getCapabilities(): MediaLoadedCapabilities {
|
||||
return { supportsPause: false };
|
||||
}
|
||||
|
||||
public getTechnology(): MediaTechnology[] {
|
||||
return [this._mode];
|
||||
}
|
||||
|
||||
public getStreamProfile(): StreamProfile {
|
||||
return { hasVideo: true, hasH265Video: false, hasAudio: false, hasAACAudio: false };
|
||||
}
|
||||
|
||||
// Show a rendered frame and, on the first one, report the stream loaded.
|
||||
protected async _showFrame(frame: Blob): Promise<void> {
|
||||
// MP4 turns each frame into a Blob asynchronously (canvas.toBlob), so a
|
||||
// frame can arrive here after stop(). Drop it so a stopped source never
|
||||
// writes to the shared image surface or reports a stale load.
|
||||
if (this._stopped) {
|
||||
return;
|
||||
}
|
||||
|
||||
const decoded = this._context.target.showFrame(frame);
|
||||
|
||||
if (this._loaded) {
|
||||
return;
|
||||
}
|
||||
this._loaded = true;
|
||||
this._firstFrameTimer.stop();
|
||||
|
||||
// Report loaded only once the frame has decoded: the media-loaded info is
|
||||
// built by reading the <img> dimensions, and an <img> whose src was just
|
||||
// set still reports 0x0 (which is rejected as invalid), so reporting
|
||||
// synchronously would leave the media stuck "not loading".
|
||||
await decoded;
|
||||
if (!this._stopped) {
|
||||
this._context.callbacks.loadedCallback();
|
||||
}
|
||||
}
|
||||
|
||||
protected _teardown(): void {}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import type { Go2RTCMessage } from '../types';
|
||||
import { ImageFrameStreamSource } from './image-frame';
|
||||
|
||||
// Each binary frame is a complete JPEG, shown directly as an image frame.
|
||||
export class MJPEGStreamSource extends ImageFrameStreamSource {
|
||||
protected _mode = 'mjpeg' as const;
|
||||
|
||||
protected _getRequestMessage(): Go2RTCMessage {
|
||||
return { type: 'mjpeg' };
|
||||
}
|
||||
|
||||
protected _handleFrame(data: ArrayBuffer): void {
|
||||
/* istanbul ignore next: This never rejects; the catch satisfies the no-floating-promises -- @preserve */
|
||||
this._showFrame(new Blob([data], { type: 'image/jpeg' })).catch(() => {});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
import { OffscreenVideo, type VideoElementFactory } from '../offscreen-video';
|
||||
import type { Go2RTCMessage, ImageStreamTarget, StreamSourceContext } from '../types';
|
||||
import { arrayBufferToBase64 } from '../utils/base64';
|
||||
import {
|
||||
convertToCodecString,
|
||||
GO2RTC_CODECS,
|
||||
selectSupportedCodecs,
|
||||
} from '../utils/codecs';
|
||||
import { ImageFrameStreamSource } from './image-frame';
|
||||
|
||||
type CanvasElementFactory = () => HTMLCanvasElement;
|
||||
|
||||
interface MP4StreamSourceOptions {
|
||||
createVideoElement?: VideoElementFactory;
|
||||
createCanvasElement?: CanvasElementFactory;
|
||||
}
|
||||
|
||||
// Each binary frame is a standalone MP4 holding one keyframe. Unlike JPEG it
|
||||
// cannot be shown directly, so it is decoded in an off-screen video and drawn
|
||||
// to a canvas to produce the image.
|
||||
export class MP4StreamSource extends ImageFrameStreamSource {
|
||||
protected _mode = 'mp4' as const;
|
||||
|
||||
private _decoder: OffscreenVideo;
|
||||
private _createCanvasElement: CanvasElementFactory;
|
||||
private _canvas: HTMLCanvasElement | null = null;
|
||||
|
||||
constructor(
|
||||
context: StreamSourceContext<ImageStreamTarget>,
|
||||
options?: MP4StreamSourceOptions,
|
||||
) {
|
||||
super(context);
|
||||
|
||||
const createVideo =
|
||||
options?.createVideoElement ?? (() => document.createElement('video'));
|
||||
this._decoder = new OffscreenVideo(() => {
|
||||
const video = createVideo();
|
||||
video.autoplay = true;
|
||||
video.playsInline = true;
|
||||
video.muted = true;
|
||||
video.addEventListener('loadeddata', () => this._drawFrame(video));
|
||||
return video;
|
||||
});
|
||||
|
||||
this._createCanvasElement =
|
||||
options?.createCanvasElement ?? (() => document.createElement('canvas'));
|
||||
}
|
||||
|
||||
protected _getRequestMessage(): Go2RTCMessage {
|
||||
// Codec support is probed on the decoder video, since that is what plays
|
||||
// the incoming MP4 frames; the image surface onto which this is rendered
|
||||
// has has no video element of its own.
|
||||
const decoder = this._decoder.get();
|
||||
return {
|
||||
type: 'mp4',
|
||||
value: convertToCodecString(
|
||||
selectSupportedCodecs(
|
||||
GO2RTC_CODECS,
|
||||
{ audio: false, video: true },
|
||||
(mimeType) => !!decoder.canPlayType(mimeType),
|
||||
),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
protected _handleFrame(data: ArrayBuffer): void {
|
||||
this._decoder.get().src = 'data:video/mp4;base64,' + arrayBufferToBase64(data);
|
||||
}
|
||||
|
||||
protected _teardown(): void {
|
||||
this._decoder.clear();
|
||||
this._canvas = null;
|
||||
}
|
||||
|
||||
private _drawFrame(decoder: HTMLVideoElement): void {
|
||||
const canvas = (this._canvas ??= this._createCanvasElement());
|
||||
|
||||
// Reassigning a canvas dimension reallocates its backing buffer, so skip it
|
||||
// when the video size is unchanged.
|
||||
if (canvas.width !== decoder.videoWidth || canvas.height !== decoder.videoHeight) {
|
||||
canvas.width = decoder.videoWidth;
|
||||
canvas.height = decoder.videoHeight;
|
||||
}
|
||||
|
||||
const context = canvas.getContext('2d');
|
||||
if (!context) {
|
||||
return;
|
||||
}
|
||||
|
||||
context.drawImage(decoder, 0, 0, canvas.width, canvas.height);
|
||||
canvas.toBlob((frame) => {
|
||||
if (frame) {
|
||||
/* istanbul ignore next: This never rejects; the catch satisfies the no-floating-promises -- @preserve */
|
||||
this._showFrame(frame).catch(() => {});
|
||||
}
|
||||
}, 'image/jpeg');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,347 @@
|
||||
import type {
|
||||
MediaLoadedCapabilities,
|
||||
MediaTechnology,
|
||||
UnsubscribeCallback,
|
||||
} from '../../../../../types';
|
||||
import { hasAudio } from '../../../../../utils/audio';
|
||||
import { Timer } from '../../../../../utils/timer';
|
||||
import {
|
||||
createBrowserMediaSource,
|
||||
type MediaSourceFactory,
|
||||
type MediaSourceInterface,
|
||||
} from '../adapters/media-source';
|
||||
import type {
|
||||
Go2RTCMessage,
|
||||
StreamProfile,
|
||||
StreamSource,
|
||||
StreamSourceContext,
|
||||
VideoStreamTarget,
|
||||
} from '../types';
|
||||
import { BoundedBufferQueue } from '../utils/bounded-buffer-queue';
|
||||
import {
|
||||
convertToCodecString,
|
||||
getCodecsForUserAgent,
|
||||
selectSupportedCodecs,
|
||||
} from '../utils/codecs';
|
||||
import { LiveEdgeTracker } from '../utils/live-edge-tracker';
|
||||
import type { LiveEdgeAction } from '../utils/live-edge-tracker/types';
|
||||
import { isServerErrorForMode } from '../utils/messages';
|
||||
import { isWebKitUserAgent } from '../utils/user-agent';
|
||||
|
||||
// ===========================================================================
|
||||
// MSE Tuning
|
||||
// ===========================================================================
|
||||
|
||||
// These are judgement-based budgets (time, buffered seconds, staged bytes)
|
||||
// balancing latency and resilience, not derived values. The retained-buffer
|
||||
// window and the staged-bytes cap follow go2rtc's reference web client; the
|
||||
// negotiation timeout has no counterpart there and is added by this
|
||||
// implementation.
|
||||
|
||||
// Seconds of media retained behind the live edge; older media is trimmed. This
|
||||
// bounds memory even across a long pause (the buffer never grows past this
|
||||
// window), so a paused stream can hold its frame indefinitely.
|
||||
const RETAINED_BUFFER_SECONDS = 15;
|
||||
|
||||
// On resume, if the playhead is more than this far behind the live edge, jump
|
||||
// forward to rejoin live rather than replaying the buffered-behind media.
|
||||
const LIVE_EDGE_RESUME_LAG_SECONDS = 1;
|
||||
|
||||
// Where a live-edge jump lands, as a gap behind the buffered end (a small
|
||||
// cushion so playback does not immediately starve at the very edge).
|
||||
const LIVE_EDGE_JUMP_OFFSET_SECONDS = 0.75;
|
||||
|
||||
// Bound on media staged while the SourceBuffer is busy; a stream that outruns
|
||||
// this is unrecoverable without a reconnect.
|
||||
const MAX_PENDING_BUFFER_BYTES = 2 * 1024 * 1024;
|
||||
|
||||
// How long to wait for the server's codec reply before failing the source.
|
||||
const MSE_NEGOTIATION_TIMEOUT_SECONDS = 5;
|
||||
|
||||
// ===========================================================================
|
||||
// MSEStreamSource
|
||||
// ===========================================================================
|
||||
|
||||
interface MSEStreamSourceOptions {
|
||||
createMediaSource?: MediaSourceFactory;
|
||||
userAgent?: string;
|
||||
}
|
||||
|
||||
export class MSEStreamSource implements StreamSource {
|
||||
private _context: StreamSourceContext<VideoStreamTarget>;
|
||||
|
||||
private _createMediaSource: MediaSourceFactory;
|
||||
private _mediaSource: MediaSourceInterface | null = null;
|
||||
|
||||
private _sourceBuffer: SourceBuffer | null = null;
|
||||
private _pendingBuffer = new BoundedBufferQueue(MAX_PENDING_BUFFER_BYTES);
|
||||
|
||||
// The codec string the server settled on during negotiation; null until then.
|
||||
private _codecs: string | null = null;
|
||||
|
||||
// The codecs offered to the server, derived once from the user agent.
|
||||
private _codecCandidates: readonly string[];
|
||||
|
||||
private _negotiationTimer = new Timer();
|
||||
private _liveEdge: LiveEdgeTracker;
|
||||
|
||||
private _unsubscribeCallbacks: UnsubscribeCallback[] = [];
|
||||
|
||||
private _loadedHandler = (): void => {
|
||||
this._context.callbacks.loadedCallback();
|
||||
};
|
||||
|
||||
private _errorHandler = (): void => {
|
||||
this._context.callbacks.failedCallback('media_error');
|
||||
};
|
||||
|
||||
// On resume, rejoin the very live edge.
|
||||
private _playHandler = (): void => {
|
||||
const sourceBuffer = this._sourceBuffer;
|
||||
if (!sourceBuffer?.buffered.length) {
|
||||
return;
|
||||
}
|
||||
const video = this._context.target.video;
|
||||
const end = sourceBuffer.buffered.end(sourceBuffer.buffered.length - 1);
|
||||
if (end - video.currentTime > LIVE_EDGE_RESUME_LAG_SECONDS) {
|
||||
video.currentTime = end - LIVE_EDGE_JUMP_OFFSET_SECONDS;
|
||||
}
|
||||
};
|
||||
|
||||
constructor(
|
||||
context: StreamSourceContext<VideoStreamTarget>,
|
||||
options?: MSEStreamSourceOptions,
|
||||
) {
|
||||
this._context = context;
|
||||
this._createMediaSource = options?.createMediaSource ?? createBrowserMediaSource;
|
||||
|
||||
const userAgent = options?.userAgent ?? navigator.userAgent;
|
||||
this._liveEdge = new LiveEdgeTracker({ webkit: isWebKitUserAgent(userAgent) });
|
||||
this._codecCandidates = getCodecsForUserAgent(userAgent);
|
||||
}
|
||||
|
||||
public start(): void {
|
||||
const mediaSource = this._createMediaSource();
|
||||
if (!mediaSource) {
|
||||
this._context.callbacks.failedCallback('unsupported');
|
||||
return;
|
||||
}
|
||||
this._mediaSource = mediaSource;
|
||||
|
||||
this._unsubscribeCallbacks.push(
|
||||
mediaSource.subscribeToSourceOpen(() => this._negotiate(mediaSource)),
|
||||
this._context.channel.subscribeToMessages((message) =>
|
||||
this._handleMessage(mediaSource, message),
|
||||
),
|
||||
);
|
||||
this._context.target.video.addEventListener('loadeddata', this._loadedHandler);
|
||||
this._context.target.video.addEventListener('error', this._errorHandler);
|
||||
this._context.target.video.addEventListener('play', this._playHandler);
|
||||
|
||||
mediaSource.attach(this._context.target.video);
|
||||
}
|
||||
|
||||
public stop(): void {
|
||||
this._negotiationTimer.stop();
|
||||
|
||||
this._unsubscribeCallbacks.forEach((unsubscribe) => unsubscribe());
|
||||
this._unsubscribeCallbacks = [];
|
||||
|
||||
this._context.target.video.removeEventListener('loadeddata', this._loadedHandler);
|
||||
this._context.target.video.removeEventListener('error', this._errorHandler);
|
||||
this._context.target.video.removeEventListener('play', this._playHandler);
|
||||
|
||||
this._context.channel.setBinaryCallback(null);
|
||||
|
||||
this._mediaSource?.detach(this._context.target.video);
|
||||
this._mediaSource = null;
|
||||
|
||||
this._sourceBuffer = null;
|
||||
|
||||
this._pendingBuffer.clear();
|
||||
}
|
||||
|
||||
public getCapabilities(): MediaLoadedCapabilities {
|
||||
return {
|
||||
supportsPause: true,
|
||||
hasAudio: hasAudio(this._context.target.video, { mseCodecs: this._codecs }),
|
||||
has2WayAudio: false,
|
||||
};
|
||||
}
|
||||
|
||||
public getTechnology(): MediaTechnology[] {
|
||||
return ['mse'];
|
||||
}
|
||||
|
||||
public getStreamProfile(): StreamProfile {
|
||||
const codecs = this._codecs ?? '';
|
||||
return {
|
||||
hasVideo: codecs.includes('avc1') || codecs.includes('hvc1'),
|
||||
hasH265Video: codecs.includes('hvc1'),
|
||||
hasAudio:
|
||||
codecs.includes('mp4a') || codecs.includes('opus') || codecs.includes('flac'),
|
||||
hasAACAudio: codecs.includes('mp4a'),
|
||||
};
|
||||
}
|
||||
|
||||
private _negotiate(mediaSource: MediaSourceInterface): void {
|
||||
const codecs = selectSupportedCodecs(
|
||||
this._codecCandidates,
|
||||
{ audio: true, video: true },
|
||||
(mimeType) => mediaSource.isTypeSupported(mimeType),
|
||||
);
|
||||
|
||||
this._context.channel.send({ type: 'mse', value: convertToCodecString(codecs) });
|
||||
this._negotiationTimer.start(MSE_NEGOTIATION_TIMEOUT_SECONDS, () =>
|
||||
this._context.callbacks.failedCallback('negotiation_timeout'),
|
||||
);
|
||||
}
|
||||
|
||||
private _handleMessage(
|
||||
mediaSource: MediaSourceInterface,
|
||||
message: Go2RTCMessage,
|
||||
): void {
|
||||
if (isServerErrorForMode(message, 'mse')) {
|
||||
this._negotiationTimer.stop();
|
||||
this._context.callbacks.failedCallback('server_error');
|
||||
return;
|
||||
}
|
||||
|
||||
if (
|
||||
message.type !== 'mse' ||
|
||||
typeof message.value !== 'string' ||
|
||||
this._sourceBuffer
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._negotiationTimer.stop();
|
||||
|
||||
// go2rtc answers the codec offer with an `mse` text message whose value is
|
||||
// the codec string for addSourceBuffer. It arrives once, before any
|
||||
// SourceBuffer exists (the guard above returns for every later message), so
|
||||
// this value is the negotiated codecs.
|
||||
this._codecs = message.value;
|
||||
|
||||
let sourceBuffer: SourceBuffer;
|
||||
try {
|
||||
sourceBuffer = mediaSource.addSourceBuffer(message.value);
|
||||
} catch {
|
||||
this._context.callbacks.failedCallback('media_error');
|
||||
return;
|
||||
}
|
||||
|
||||
// Segments mode: order media by its fMP4 timestamps rather than by
|
||||
// arrival order.
|
||||
sourceBuffer.mode = 'segments';
|
||||
|
||||
const updateEndListener = (): void =>
|
||||
this._handleUpdateEnd(mediaSource, sourceBuffer);
|
||||
sourceBuffer.addEventListener('updateend', updateEndListener);
|
||||
this._unsubscribeCallbacks.push(() =>
|
||||
sourceBuffer.removeEventListener('updateend', updateEndListener),
|
||||
);
|
||||
|
||||
this._sourceBuffer = sourceBuffer;
|
||||
|
||||
this._context.channel.setBinaryCallback((data) =>
|
||||
this._handleData(sourceBuffer, data),
|
||||
);
|
||||
}
|
||||
|
||||
private _handleData(sourceBuffer: SourceBuffer, data: ArrayBuffer): void {
|
||||
if (sourceBuffer.updating || !this._pendingBuffer.isEmpty) {
|
||||
if (!this._pendingBuffer.push(data)) {
|
||||
this._context.callbacks.failedCallback('buffer_overflow');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
this._append(sourceBuffer, data);
|
||||
}
|
||||
|
||||
private _append(sourceBuffer: SourceBuffer, data: ArrayBuffer): void {
|
||||
try {
|
||||
sourceBuffer.appendBuffer(data);
|
||||
} catch {
|
||||
// Append failures during teardown races are recoverable by later appends;
|
||||
// fatal conditions surface through the video element's error event
|
||||
// instead.
|
||||
}
|
||||
}
|
||||
|
||||
private _handleUpdateEnd(
|
||||
mediaSource: MediaSourceInterface,
|
||||
sourceBuffer: SourceBuffer,
|
||||
): void {
|
||||
if (sourceBuffer.updating) {
|
||||
return;
|
||||
}
|
||||
|
||||
// A queued updateend can fire after a reconnect or teardown has closed the
|
||||
// MediaSource (detached from the video).
|
||||
if (!mediaSource.isOpen()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const pending = this._pendingBuffer.shift();
|
||||
if (pending) {
|
||||
this._append(sourceBuffer, pending);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!sourceBuffer.buffered.length) {
|
||||
return;
|
||||
}
|
||||
|
||||
const video = this._context.target.video;
|
||||
const end = sourceBuffer.buffered.end(sourceBuffer.buffered.length - 1);
|
||||
this._trimSourceBuffer(mediaSource, sourceBuffer, end);
|
||||
|
||||
// While paused the user is holding a frame: leave the playhead alone (the
|
||||
// trim above still bounds memory) and do not sample the growing lag, which
|
||||
// would otherwise bias the catch-up rate once playback resumes. Resuming
|
||||
// jumps back to the live edge (see the play handler).
|
||||
if (video.paused) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Keep playback tracking the live edge as new media arrives.
|
||||
this._applyLiveEdgeAction(
|
||||
video,
|
||||
this._liveEdge.next({
|
||||
bufferedEndSeconds: end,
|
||||
currentTimeSeconds: video.currentTime,
|
||||
playbackRate: video.playbackRate,
|
||||
now: new Date(),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
private _applyLiveEdgeAction(video: HTMLVideoElement, action: LiveEdgeAction): void {
|
||||
if (action.action === 'seek') {
|
||||
video.currentTime = action.seconds;
|
||||
} else if (action.action === 'rate' && video.playbackRate !== action.rate) {
|
||||
video.playbackRate = action.rate;
|
||||
}
|
||||
}
|
||||
|
||||
// Keep only the most recent RETAINED_BUFFER_SECONDS of media, trimming older
|
||||
// media and re-declaring the seekable range. The playhead is never snapped
|
||||
// here: while playing the live-edge tracker keeps it near the edge, and while
|
||||
// paused it is left holding its frame (a playhead that falls past the lag
|
||||
// bound is handled by a reconnect, not by dragging it forward).
|
||||
private _trimSourceBuffer(
|
||||
mediaSource: MediaSourceInterface,
|
||||
sourceBuffer: SourceBuffer,
|
||||
end: number,
|
||||
): void {
|
||||
const retainedStart = end - RETAINED_BUFFER_SECONDS;
|
||||
const bufferedStart = sourceBuffer.buffered.start(0);
|
||||
|
||||
if (retainedStart > bufferedStart) {
|
||||
sourceBuffer.remove(bufferedStart, retainedStart);
|
||||
mediaSource.setLiveSeekableRange(retainedStart, end);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
import type {
|
||||
MediaLoadedCapabilities,
|
||||
MediaTechnology,
|
||||
UnsubscribeCallback,
|
||||
} from '../../../../../types';
|
||||
import { has2WayAudio, hasAudio } from '../../../../../utils/audio';
|
||||
import { Timer } from '../../../../../utils/timer';
|
||||
import {
|
||||
createBrowserPeerConnection,
|
||||
GO2RTC_PEER_CONNECTION_CONFIG,
|
||||
type PeerConnectionFactory,
|
||||
} from '../adapters/peer-connection';
|
||||
import type {
|
||||
Go2RTCMessage,
|
||||
StreamProfile,
|
||||
StreamSource,
|
||||
StreamSourceContext,
|
||||
VideoStreamTarget,
|
||||
} from '../types';
|
||||
import { isServerErrorForMode } from '../utils/messages';
|
||||
import { sdpHasH265 } from '../utils/webrtc-sdp';
|
||||
|
||||
// ===========================================================================
|
||||
// WebRTC Tuning
|
||||
// ===========================================================================
|
||||
|
||||
// Fail if no decoded frame arrives within this window. Covers the case where
|
||||
// ICE connects and packets flow but frames never decode (the connection looks
|
||||
// healthy yet nothing plays), so the session can fall back instead of hanging.
|
||||
// See https://github.com/dermotduffy/advanced-camera-card/issues/1699
|
||||
// The window length is a tuning value, not derived: long enough to avoid
|
||||
// failing a slow-but-healthy start, short enough to fall back promptly.
|
||||
const WEBRTC_CONNECT_TIMEOUT_SECONDS = 5;
|
||||
|
||||
// ===========================================================================
|
||||
// WebRTCStreamSource
|
||||
// ===========================================================================
|
||||
|
||||
export type MediaStreamFactory = (tracks: MediaStreamTrack[]) => MediaStream;
|
||||
|
||||
interface WebRTCStreamSourceOptions {
|
||||
createPeerConnection?: PeerConnectionFactory;
|
||||
createMediaStream?: MediaStreamFactory;
|
||||
microphoneStream?: MediaStream | null;
|
||||
}
|
||||
|
||||
export class WebRTCStreamSource implements StreamSource {
|
||||
private _context: StreamSourceContext<VideoStreamTarget>;
|
||||
private _stream: MediaStream | null = null;
|
||||
private _pc: RTCPeerConnection | null = null;
|
||||
|
||||
private _createPeerConnection: PeerConnectionFactory;
|
||||
private _createMediaStream: MediaStreamFactory;
|
||||
private _microphoneStream: MediaStream | null;
|
||||
|
||||
private _microphoneTransceiver: RTCRtpTransceiver | null = null;
|
||||
|
||||
private _connectTimer = new Timer();
|
||||
|
||||
private _unsubscribeCallbacks: UnsubscribeCallback[] = [];
|
||||
|
||||
private _loadedHandler = (): void => {
|
||||
this._connectTimer.stop();
|
||||
this._context.callbacks.loadedCallback();
|
||||
};
|
||||
|
||||
constructor(
|
||||
context: StreamSourceContext<VideoStreamTarget>,
|
||||
options?: WebRTCStreamSourceOptions,
|
||||
) {
|
||||
this._context = context;
|
||||
this._createPeerConnection =
|
||||
options?.createPeerConnection ?? createBrowserPeerConnection;
|
||||
this._createMediaStream =
|
||||
options?.createMediaStream ?? ((tracks) => new MediaStream(tracks));
|
||||
|
||||
this._microphoneStream = options?.microphoneStream ?? null;
|
||||
}
|
||||
|
||||
public start(): void {
|
||||
const pc = this._createPeerConnection(GO2RTC_PEER_CONNECTION_CONFIG);
|
||||
this._pc = pc;
|
||||
|
||||
// Always pre-arm exactly one outbound audio slot so the microphone track
|
||||
// can be attached later via `replaceTrack` with no renegotiation. The
|
||||
// kind-only `addTransceiver('audio', ...)` form never calls getUserMedia,
|
||||
// so it never raises permission prompt for users.
|
||||
const microphoneTrack = this._microphoneStream?.getAudioTracks()[0] ?? null;
|
||||
this._microphoneTransceiver = pc.addTransceiver(microphoneTrack ?? 'audio', {
|
||||
direction: 'sendonly',
|
||||
});
|
||||
pc.addTransceiver('video', { direction: 'recvonly' });
|
||||
pc.addTransceiver('audio', { direction: 'recvonly' });
|
||||
|
||||
pc.addEventListener('icecandidate', (ev) => {
|
||||
// A late candidate from a superseded connection (e.g. after stop(), if
|
||||
// WebRTC has lost the race) must not send on the channel a binary lane
|
||||
// may still be using.
|
||||
if (this._pc !== pc) {
|
||||
return;
|
||||
}
|
||||
|
||||
// An empty value signals end-of-candidates, which the server accepts.
|
||||
this._context.channel.send({
|
||||
type: 'webrtc/candidate',
|
||||
value: ev.candidate ? ev.candidate.candidate : '',
|
||||
});
|
||||
});
|
||||
pc.addEventListener('connectionstatechange', () =>
|
||||
this._handleConnectionStateChange(pc),
|
||||
);
|
||||
|
||||
this._unsubscribeCallbacks.push(
|
||||
this._context.channel.subscribeToMessages((message) =>
|
||||
this._handleMessage(pc, message),
|
||||
),
|
||||
);
|
||||
|
||||
this._connectTimer.start(WEBRTC_CONNECT_TIMEOUT_SECONDS, () =>
|
||||
this._context.callbacks.failedCallback('connect_timeout'),
|
||||
);
|
||||
|
||||
this._negotiate(pc).catch(() => {
|
||||
// A rejection from a superseded connection (e.g. after `stop`) is not
|
||||
// this source's concern.
|
||||
if (this._pc === pc) {
|
||||
this._context.callbacks.failedCallback('media_error');
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
public stop(): void {
|
||||
this._connectTimer.stop();
|
||||
|
||||
this._unsubscribeCallbacks.forEach((unsubscribe) => unsubscribe());
|
||||
this._unsubscribeCallbacks = [];
|
||||
|
||||
this._context.target.video.removeEventListener('loadeddata', this._loadedHandler);
|
||||
if (this._pc) {
|
||||
// pc.close() does not stop the sender's tracks, so the outbound microphone
|
||||
// track keeps running. That is deliberate: MicrophoneManager owns the mic
|
||||
// (it is shared across cameras), so stopping it here would break it
|
||||
// elsewhere -- do not add a track.stop() here. See
|
||||
// https://github.com/dermotduffy/advanced-camera-card/issues/1810
|
||||
this._pc.close();
|
||||
this._pc = null;
|
||||
}
|
||||
|
||||
// The transceiver belonged to the now-closed peer connection.
|
||||
this._microphoneTransceiver = null;
|
||||
this._context.target.video.srcObject = null;
|
||||
this._stream = null;
|
||||
}
|
||||
|
||||
public getMediaStream(): MediaStream | null {
|
||||
return this._stream;
|
||||
}
|
||||
|
||||
public getPeerConnection(): RTCPeerConnection | null {
|
||||
return this._pc;
|
||||
}
|
||||
|
||||
public getCapabilities(): MediaLoadedCapabilities {
|
||||
return {
|
||||
supportsPause: true,
|
||||
hasAudio: hasAudio(this._context.target.video, { pc: this._pc }),
|
||||
has2WayAudio: has2WayAudio(this._pc),
|
||||
};
|
||||
}
|
||||
|
||||
public getTechnology(): MediaTechnology[] {
|
||||
return ['webrtc'];
|
||||
}
|
||||
|
||||
public getStreamProfile(): StreamProfile {
|
||||
const sdp = this._pc?.remoteDescription?.sdp ?? null;
|
||||
|
||||
return {
|
||||
hasVideo: (this._stream?.getVideoTracks().length ?? 0) > 0,
|
||||
hasH265Video: sdp ? sdpHasH265(sdp) : false,
|
||||
hasAudio: (this._stream?.getAudioTracks().length ?? 0) > 0,
|
||||
hasAACAudio: false,
|
||||
};
|
||||
}
|
||||
|
||||
// Swap the outbound microphone track without renegotiating. Guards against a
|
||||
// late rejection from a superseded call (a newer stream, or teardown)
|
||||
// bringing a retired connection back or overwriting a fresher request.
|
||||
public async setMicrophoneStream(stream: MediaStream | null): Promise<void> {
|
||||
if (this._microphoneStream === stream) {
|
||||
return;
|
||||
}
|
||||
this._microphoneStream = stream;
|
||||
|
||||
const transceiver = this._microphoneTransceiver;
|
||||
if (!transceiver) {
|
||||
// No peer connection yet; the next `start()` reads the current stream and
|
||||
// pre-arms the transceiver with it.
|
||||
return;
|
||||
}
|
||||
|
||||
// A microphone stream carries a single audio track; null detaches the sender.
|
||||
const desiredTrack = stream?.getAudioTracks()[0] ?? null;
|
||||
try {
|
||||
await transceiver.sender.replaceTrack(desiredTrack);
|
||||
} catch {
|
||||
const stillCurrent =
|
||||
transceiver === this._microphoneTransceiver &&
|
||||
this._microphoneStream === stream &&
|
||||
this._pc !== null;
|
||||
if (stillCurrent) {
|
||||
this._context.callbacks.failedCallback('two_way_audio_error');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async _negotiate(pc: RTCPeerConnection): Promise<void> {
|
||||
const offer = await pc.createOffer();
|
||||
if (this._pc !== pc) {
|
||||
return;
|
||||
}
|
||||
|
||||
await pc.setLocalDescription(offer);
|
||||
if (this._pc !== pc) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._context.channel.send({ type: 'webrtc/offer', value: offer.sdp ?? '' });
|
||||
}
|
||||
|
||||
private _handleMessage(pc: RTCPeerConnection, message: Go2RTCMessage): void {
|
||||
// Every message type handled here carries a string value.
|
||||
if (this._pc !== pc || typeof message.value !== 'string') {
|
||||
return;
|
||||
}
|
||||
|
||||
if (isServerErrorForMode(message, 'webrtc')) {
|
||||
this._context.callbacks.failedCallback('server_error');
|
||||
return;
|
||||
}
|
||||
|
||||
switch (message.type) {
|
||||
case 'webrtc/answer':
|
||||
pc.setRemoteDescription({ type: 'answer', sdp: message.value }).catch(() => {});
|
||||
break;
|
||||
|
||||
case 'webrtc/candidate':
|
||||
if (message.value) {
|
||||
// The server sends no sdpMid; max-bundle puts every track on m-line 0.
|
||||
pc.addIceCandidate({ candidate: message.value, sdpMid: '0' }).catch(() => {});
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private _handleConnectionStateChange(pc: RTCPeerConnection): void {
|
||||
if (this._pc !== pc) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (pc.connectionState === 'connected') {
|
||||
this._attachStream(pc);
|
||||
} else if (pc.connectionState === 'failed') {
|
||||
// Only 'failed' is terminal. 'disconnected' is a recoverable ICE blip
|
||||
// that usually returns to 'connected' on its own, so failing on it would
|
||||
// turn a brief network hiccup into a needless reconnect; a disconnect
|
||||
// that never recovers stops delivering frames and is caught by the
|
||||
// frame-stall watchdog instead.
|
||||
this._context.callbacks.failedCallback('media_error');
|
||||
}
|
||||
}
|
||||
|
||||
private _attachStream(pc: RTCPeerConnection): void {
|
||||
if (this._stream) {
|
||||
return;
|
||||
}
|
||||
|
||||
const tracks = pc
|
||||
.getTransceivers()
|
||||
.filter((transceiver) => transceiver.currentDirection === 'recvonly')
|
||||
.map((transceiver) => transceiver.receiver.track);
|
||||
|
||||
const stream = this._createMediaStream(tracks);
|
||||
this._stream = stream;
|
||||
|
||||
this._context.target.video.addEventListener('loadeddata', this._loadedHandler, {
|
||||
once: true,
|
||||
});
|
||||
this._context.target.video.srcObject = stream;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import type {
|
||||
MediaLoadedCapabilities,
|
||||
MediaTechnology,
|
||||
UnsubscribeCallback,
|
||||
} from '../../../../types';
|
||||
|
||||
// ===========================================================================
|
||||
// Control messages
|
||||
// ===========================================================================
|
||||
|
||||
// go2rtc control messages are JSON text frames of this shape; media flows as
|
||||
// separate binary frames.
|
||||
export const go2RTCMessageSchema = z.object({
|
||||
type: z.string(),
|
||||
|
||||
// Per-type payload (a codec list, an SDP, an ICE candidate, error text, ...):
|
||||
// absent for some types (e.g. mjpeg) and not always a string, so it is typed
|
||||
// `unknown` and each handler narrows it before use.
|
||||
value: z.unknown().optional(),
|
||||
});
|
||||
export type Go2RTCMessage = z.infer<typeof go2RTCMessageSchema>;
|
||||
|
||||
export type MessageCallback = (message: Go2RTCMessage) => void;
|
||||
export type BinaryCallback = (data: ArrayBuffer) => void;
|
||||
|
||||
// ===========================================================================
|
||||
// Signaling channel
|
||||
// ===========================================================================
|
||||
|
||||
// A session's two parallel delivery paths: 'binary' (MSE/MP4/MJPEG media over
|
||||
// the WebSocket) and 'webrtc' (media over the peer connection).
|
||||
export type Lane = 'binary' | 'webrtc';
|
||||
|
||||
// The narrow view of the signaling channel that stream sources receive: they
|
||||
// may exchange messages but not manage the connection.
|
||||
export interface StreamSourceChannel {
|
||||
send(message: Go2RTCMessage): void;
|
||||
subscribeToMessages(callback: MessageCallback): UnsubscribeCallback;
|
||||
setBinaryCallback(callback: BinaryCallback | null): void;
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Render targets
|
||||
// ===========================================================================
|
||||
|
||||
// Surface vs Target: A "target" is where a source (binary vs webrtc) puts its
|
||||
// frames for one mode: a <video> element, or an <img> element (via a showFrame
|
||||
// callback). The component adds a media-player controller to a target to make a
|
||||
// fuller "surface" (see SessionSurfaces in session-controller.ts); it hands a
|
||||
// source only the target, never the surface, so a source cannot reach the
|
||||
// controller.
|
||||
|
||||
// What a source renders onto: the real <video> for modes the browser plays
|
||||
// (MSE, WebRTC), or an image sink fed decoded frames for modes presented as a
|
||||
// sequence of still images (MP4, MJPEG).
|
||||
export interface VideoStreamTarget {
|
||||
kind: 'video';
|
||||
video: HTMLVideoElement;
|
||||
}
|
||||
|
||||
// A frame is handed over as a Blob rather than exposing the raw <img> so the
|
||||
// object-URL lifecycle (create, revoke the previous) stays owned by the image
|
||||
// surface, not each source. The returned promise resolves once the frame has
|
||||
// decoded, so a source can defer reporting itself loaded until the surface has
|
||||
// real dimensions.
|
||||
export interface ImageStreamTarget {
|
||||
kind: 'image';
|
||||
showFrame(frame: Blob): Promise<void>;
|
||||
}
|
||||
|
||||
type StreamSourceTarget = VideoStreamTarget | ImageStreamTarget;
|
||||
|
||||
// Which of the two render surfaces media is shown on: the real <video> or the
|
||||
// <img>. Matches the `kind` of the corresponding stream target.
|
||||
export type SurfaceKind = 'video' | 'image';
|
||||
|
||||
// ===========================================================================
|
||||
// Stream sources
|
||||
// ===========================================================================
|
||||
|
||||
// Describes a source's negotiated media, used to arbitrate the MSE-vs-WebRTC
|
||||
// race. The bit-weighted comparison lives in `utils/source-priority.ts`.
|
||||
export interface StreamProfile {
|
||||
hasVideo: boolean;
|
||||
hasH265Video: boolean;
|
||||
hasAudio: boolean;
|
||||
|
||||
// The MSE audio score is inferred from the negotiated codec string, not an
|
||||
// observed track, so it counts only AAC -- the audio codec browsers decode
|
||||
// reliably via MSE. Opus and FLAC over MSE are inconsistently supported, so
|
||||
// crediting them could prefer a stream whose audio never plays.
|
||||
hasAACAudio: boolean;
|
||||
}
|
||||
|
||||
export type StreamSourceFailureReason =
|
||||
| 'two_way_audio_error'
|
||||
| 'buffer_overflow'
|
||||
| 'connect_timeout'
|
||||
| 'media_error'
|
||||
| 'negotiation_timeout'
|
||||
| 'server_error'
|
||||
| 'unsupported';
|
||||
|
||||
export interface StreamSourceCallbacks {
|
||||
loadedCallback: () => void;
|
||||
failedCallback: (reason: StreamSourceFailureReason) => void;
|
||||
}
|
||||
|
||||
export interface StreamSourceContext<T extends StreamSourceTarget = StreamSourceTarget> {
|
||||
target: T;
|
||||
channel: StreamSourceChannel;
|
||||
callbacks: StreamSourceCallbacks;
|
||||
}
|
||||
|
||||
// A single streaming mode implementation (e.g. MSE). Sources negotiate over the
|
||||
// shared channel, attach media to the provided video element, and report
|
||||
// readiness or failure via callbacks; connection management and failure
|
||||
// recovery belong to the session not the source.
|
||||
export interface StreamSource {
|
||||
// Single-use: start() begins the source and stop() tears it down for good.
|
||||
start(): void;
|
||||
stop(): void;
|
||||
|
||||
getCapabilities(): MediaLoadedCapabilities;
|
||||
getTechnology(): MediaTechnology[];
|
||||
getStreamProfile(): StreamProfile;
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
export const arrayBufferToBase64 = (buffer: ArrayBuffer): string => {
|
||||
const bytes = new Uint8Array(buffer);
|
||||
let binary = '';
|
||||
|
||||
// `btoa` needs bytes as a binary string, build char-by-char.
|
||||
for (let i = 0; i < bytes.byteLength; ++i) {
|
||||
binary += String.fromCharCode(bytes[i]);
|
||||
}
|
||||
|
||||
return window.btoa(binary);
|
||||
};
|
||||
@@ -0,0 +1,42 @@
|
||||
// A byte-bounded FIFO queue of ArrayBuffer chunks: a push that would exceed the
|
||||
// byte cap is rejected.
|
||||
export class BoundedBufferQueue {
|
||||
private _buffers: ArrayBuffer[] = [];
|
||||
private _bytes = 0;
|
||||
|
||||
private _maxBytes: number;
|
||||
|
||||
constructor(maxBytes: number) {
|
||||
this._maxBytes = maxBytes;
|
||||
}
|
||||
|
||||
public get isEmpty(): boolean {
|
||||
return this._buffers.length === 0;
|
||||
}
|
||||
|
||||
// Stage a chunk at the back, or return false (staging nothing) if it would
|
||||
// exceed the byte cap.
|
||||
public push(data: ArrayBuffer): boolean {
|
||||
if (this._bytes + data.byteLength > this._maxBytes) {
|
||||
return false;
|
||||
}
|
||||
this._buffers.push(data);
|
||||
this._bytes += data.byteLength;
|
||||
return true;
|
||||
}
|
||||
|
||||
// Remove and return the oldest staged chunk, or null when empty.
|
||||
public shift(): ArrayBuffer | null {
|
||||
const data = this._buffers.shift();
|
||||
if (!data) {
|
||||
return null;
|
||||
}
|
||||
this._bytes -= data.byteLength;
|
||||
return data;
|
||||
}
|
||||
|
||||
public clear(): void {
|
||||
this._buffers = [];
|
||||
this._bytes = 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
import { getSafariMajorVersion } from './user-agent';
|
||||
|
||||
// Codec strings offered to the go2rtc server when negotiating an MSE or MP4
|
||||
// stream. The go2rtc server recognizes only a fixed set of exact strings (one
|
||||
// canonical spelling per codec, e.g. `avc1.640029` for all H.264) and silently
|
||||
// ignores the rest; it then matches the camera's tracks by codec name alone,
|
||||
// never comparing profile or level.
|
||||
//
|
||||
// So membership here is not correctness-critical: an unrecognized entry (a
|
||||
// different H.264 level, AAC-HE) has no effect, and a camera streaming any
|
||||
// level still matches. The list can stay a broad superset rather than tracking
|
||||
// one server version's exact set.
|
||||
//
|
||||
// Order matters as the offer's preference order: video before audio, and audio
|
||||
// most to least reliably supported.
|
||||
//
|
||||
// H.264/H.265 video, most-compatible first. Offered to every browser.
|
||||
const GO2RTC_VIDEO_CODECS: readonly string[] = [
|
||||
// H.264 high 4.1
|
||||
'avc1.640029',
|
||||
|
||||
// H.264 high 4.2
|
||||
'avc1.64002A',
|
||||
|
||||
// H.264 high 5.1
|
||||
'avc1.640033',
|
||||
|
||||
// H.265 main 5.1
|
||||
'hvc1.1.6.L153.B0',
|
||||
];
|
||||
|
||||
// Audio is layered by the oldest Safari major version that can decode each
|
||||
// codec via MSE (see `getCodecsForUserAgent`). Each tier extends the previous
|
||||
// one with the next-most-reliable audio codec.
|
||||
const GO2RTC_SAFARI_13_CODECS: readonly string[] = [
|
||||
...GO2RTC_VIDEO_CODECS,
|
||||
// AAC LC
|
||||
'mp4a.40.2',
|
||||
|
||||
// AAC HE
|
||||
'mp4a.40.5',
|
||||
];
|
||||
|
||||
const GO2RTC_SAFARI_14_CODECS: readonly string[] = [
|
||||
...GO2RTC_SAFARI_13_CODECS,
|
||||
|
||||
// FLAC
|
||||
'flac',
|
||||
];
|
||||
|
||||
// Full codec set: offered to non-Safari browsers.
|
||||
export const GO2RTC_CODECS: readonly string[] = [
|
||||
...GO2RTC_SAFARI_14_CODECS,
|
||||
|
||||
// Opus
|
||||
'opus',
|
||||
];
|
||||
|
||||
// Safari's `isTypeSupported` over-reports audio codec support, so offer only
|
||||
// the codec set the running major version can *actually* play via MSE. The
|
||||
// version boundaries (AAC from 13, FLAC from 14, OPUS never) are empirical
|
||||
// browser-compatibility observations, not derived values -- treat them as a
|
||||
// best-known table that may need revising for future Safari versions.
|
||||
export const getCodecsForUserAgent = (userAgent: string): readonly string[] => {
|
||||
const version = getSafariMajorVersion(userAgent);
|
||||
if (version === null) {
|
||||
return GO2RTC_CODECS;
|
||||
}
|
||||
if (version < 13) {
|
||||
return GO2RTC_VIDEO_CODECS;
|
||||
}
|
||||
if (version < 14) {
|
||||
return GO2RTC_SAFARI_13_CODECS;
|
||||
}
|
||||
return GO2RTC_SAFARI_14_CODECS;
|
||||
};
|
||||
|
||||
interface CodecMediaSelection {
|
||||
audio: boolean;
|
||||
video: boolean;
|
||||
}
|
||||
|
||||
// A codec is a video codec if it contains 'vc1' beyond position zero (avc1.*,
|
||||
// hvc1.*); everything else in the go2rtc codec list is audio.
|
||||
const isVideoCodec = (codec: string): boolean => codec.indexOf('vc1') > 0;
|
||||
|
||||
// Filter the codec list to the requested media kinds and to what the given
|
||||
// support callback accepts.
|
||||
export const selectSupportedCodecs = (
|
||||
codecs: readonly string[],
|
||||
media: CodecMediaSelection,
|
||||
isSupported: (mimeType: string) => boolean,
|
||||
): readonly string[] =>
|
||||
codecs
|
||||
.filter((codec) => (isVideoCodec(codec) ? media.video : media.audio))
|
||||
.filter((codec) => isSupported(`video/mp4; codecs="${codec}"`));
|
||||
|
||||
// Encode a codec list as the comma-joined string go2rtc expects on the wire.
|
||||
export const convertToCodecString = (codecs: readonly string[]): string =>
|
||||
codecs.join(',');
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { MediaUnavailableIssueReason } from '../../../../../card-controller/issues/issues/media-unavailable';
|
||||
import type { StreamSourceFailureReason } from '../types';
|
||||
|
||||
// The card's media-unavailable causes are user-facing; a source's failure
|
||||
// reasons are technical. Only a media error and a buffer overflow have no
|
||||
// distinct user story, so they map to a generic playback error; the rest each
|
||||
// keep a meaningful cause -- a server rejection, an unsupported stream, a failed
|
||||
// two-way-audio call, or a timeout that means the stream never got going.
|
||||
const FAILURE_TO_ISSUE_REASON: Record<
|
||||
StreamSourceFailureReason,
|
||||
MediaUnavailableIssueReason
|
||||
> = {
|
||||
two_way_audio_error: 'two_way_audio_error',
|
||||
buffer_overflow: 'playback_error',
|
||||
connect_timeout: 'not_loading',
|
||||
media_error: 'playback_error',
|
||||
negotiation_timeout: 'not_loading',
|
||||
server_error: 'server_error',
|
||||
unsupported: 'unsupported',
|
||||
};
|
||||
|
||||
// A null reason is a connection-level failure with no source detail (e.g. the
|
||||
// socket dropped), which reads as a generic playback error.
|
||||
export const mapFailureReasonToIssueReason = (
|
||||
reason: StreamSourceFailureReason | null,
|
||||
): MediaUnavailableIssueReason =>
|
||||
reason === null ? 'playback_error' : FAILURE_TO_ISSUE_REASON[reason];
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
// A GOP ("group of pictures") is the span from one keyframe to just before the
|
||||
// next: a full keyframe followed by the smaller frames that only encode what
|
||||
// changed since it. Streamed video arrives one GOP at a time, so a camera with
|
||||
// a 2s keyframe interval delivers media in roughly 2s bursts. That cadence sets
|
||||
// how far behind the live edge playback must sit to ride over the bursts
|
||||
// without starving, so this file estimates the GOP length by timing the
|
||||
// wall-clock interval between buffer advances (each advance is about one GOP of
|
||||
// new media) and averaging it over a short rolling window.
|
||||
//
|
||||
// Measuring, rather than assuming a fixed length, is necessary here: the stream
|
||||
// carries no declared GOP length (it's raw fragments arriving over a
|
||||
// WebSocket), and camera GOPs vary widely -- a doorbell at ~1s, an NVR tuned
|
||||
// for bandwidth at 4s+ -- so any single constant is either too small (high-GOP
|
||||
// cameras stall) or needlessly laggy for everyone else. Measuring per stream
|
||||
// right-sizes it without a configuration knob.
|
||||
//
|
||||
// This is necessary to adaptively calculate the buffer (and thus inflicted
|
||||
// latency) to maintain for Safari, in order to avoid it "hitting the end" and
|
||||
// thus pausing, requiring human intervention to play again.
|
||||
|
||||
// The assumed GOP length (seconds) until enough delivery cadence is measured,
|
||||
// and how many recent cadence samples the rolling average spans.
|
||||
const DEFAULT_GOP_SECONDS = 1;
|
||||
export const GOP_SAMPLE_WINDOW_SIZE = 5;
|
||||
|
||||
// Estimates the GOP length (the delivery cadence) as a rolling average of the
|
||||
// interval between buffer advances. A trim updates the buffer without growing
|
||||
// it, so only advances are sampled.
|
||||
export class GOPCadenceEstimator {
|
||||
private _samples: number[] = [];
|
||||
private _lastBufferedEnd: number | null = null;
|
||||
private _lastAdvanceTime: Date | null = null;
|
||||
|
||||
public sample(bufferedEndSeconds: number, now: Date): void {
|
||||
const advanced =
|
||||
this._lastBufferedEnd === null || bufferedEndSeconds > this._lastBufferedEnd;
|
||||
if (!advanced) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._lastAdvanceTime !== null) {
|
||||
const interval = (now.getTime() - this._lastAdvanceTime.getTime()) / 1000;
|
||||
if (interval > 0) {
|
||||
if (this._samples.length >= GOP_SAMPLE_WINDOW_SIZE) {
|
||||
this._samples.shift();
|
||||
}
|
||||
this._samples.push(interval);
|
||||
}
|
||||
}
|
||||
|
||||
this._lastAdvanceTime = now;
|
||||
this._lastBufferedEnd = bufferedEndSeconds;
|
||||
}
|
||||
|
||||
public estimateSeconds(): number {
|
||||
if (!this._samples.length) {
|
||||
return DEFAULT_GOP_SECONDS;
|
||||
}
|
||||
return this._samples.reduce((sum, sample) => sum + sample, 0) / this._samples.length;
|
||||
}
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
// Keeps MSE playback near the live edge. The strategy is browser-specific: see
|
||||
// ./webkit.ts (seek-based, because WebKit stutters on playbackRate changes) and
|
||||
// ./non-webkit.ts (playback-rate-based).
|
||||
|
||||
import { NonWebKitLiveEdgeStrategy } from './non-webkit';
|
||||
import type { LiveEdgeAction, LiveEdgeStatus, LiveEdgeStrategy } from './types';
|
||||
import { WebKitLiveEdgeStrategy } from './webkit';
|
||||
|
||||
// Picks the browser-appropriate live-edge strategy and delegates to it.
|
||||
export class LiveEdgeTracker {
|
||||
private _strategy: LiveEdgeStrategy;
|
||||
|
||||
constructor(options: { webkit: boolean }) {
|
||||
this._strategy = options.webkit
|
||||
? new WebKitLiveEdgeStrategy()
|
||||
: new NonWebKitLiveEdgeStrategy();
|
||||
}
|
||||
|
||||
public next(status: LiveEdgeStatus): LiveEdgeAction {
|
||||
return this._strategy.next(status);
|
||||
}
|
||||
}
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
// Everything that is not WebKit: keep playback near the live edge by nudging the
|
||||
// playback rate (the approach WebKit cannot use; see ./webkit.ts).
|
||||
|
||||
import type { LiveEdgeAction, LiveEdgeStatus, LiveEdgeStrategy } from './types';
|
||||
|
||||
// The lag (seconds) below which playback stays at realtime, how much the
|
||||
// adaptive threshold scales with the stream's normal lag, the exponential rate
|
||||
// curve's scale and steepness, and the rate ceiling.
|
||||
const CATCH_UP_MIN_LAG_SECONDS = 3;
|
||||
const CATCH_UP_AVERAGE_LAG_MULTIPLIER = 1.5;
|
||||
const CATCH_UP_RATE_SCALE = 0.2;
|
||||
const CATCH_UP_RATE_STEEPNESS = 0.5;
|
||||
const CATCH_UP_MAX_RATE = 2;
|
||||
|
||||
// How many recent lag samples the rolling average spans; a tuning value trading
|
||||
// responsiveness against noise, not a derived one.
|
||||
export const LAG_SAMPLE_WINDOW_SIZE = 10;
|
||||
|
||||
// Nudge the playback rate up on an exponential curve that stays gentle for
|
||||
// ordinary drift and only accelerates when playback is far behind, clamped so it
|
||||
// never drops below realtime.
|
||||
export class NonWebKitLiveEdgeStrategy implements LiveEdgeStrategy {
|
||||
private _samples: number[] = [];
|
||||
private _nextSampleIndex = 0;
|
||||
|
||||
public next(status: LiveEdgeStatus): LiveEdgeAction {
|
||||
const lag = status.bufferedEndSeconds - status.currentTimeSeconds;
|
||||
this._sampleLag(lag, status.playbackRate);
|
||||
|
||||
if (lag < CATCH_UP_MIN_LAG_SECONDS) {
|
||||
return { action: 'rate', rate: 1 };
|
||||
}
|
||||
|
||||
// The threshold adapts to the stream's normal lag (e.g. long keyframe
|
||||
// intervals make a constant multi-second lag healthy), so the rate only
|
||||
// rises when playback is genuinely falling behind.
|
||||
const threshold = (this._averageLag() ?? 0) * CATCH_UP_AVERAGE_LAG_MULTIPLIER;
|
||||
const rate = Math.min(
|
||||
1 + CATCH_UP_RATE_SCALE * Math.exp(CATCH_UP_RATE_STEEPNESS * lag - threshold),
|
||||
CATCH_UP_MAX_RATE,
|
||||
);
|
||||
return { action: 'rate', rate };
|
||||
}
|
||||
|
||||
private _sampleLag(lagSeconds: number, playbackRate: number): void {
|
||||
// Samples taken while deliberately catching up (rate above realtime and
|
||||
// still far behind) would inflate the average the threshold derives from,
|
||||
// so they are excluded.
|
||||
if (playbackRate !== 1 && lagSeconds >= CATCH_UP_MIN_LAG_SECONDS) {
|
||||
return;
|
||||
}
|
||||
if (this._samples.length < LAG_SAMPLE_WINDOW_SIZE) {
|
||||
this._samples.push(lagSeconds);
|
||||
} else {
|
||||
this._samples[this._nextSampleIndex] = lagSeconds;
|
||||
this._nextSampleIndex = (this._nextSampleIndex + 1) % LAG_SAMPLE_WINDOW_SIZE;
|
||||
}
|
||||
}
|
||||
|
||||
private _averageLag(): number | null {
|
||||
if (!this._samples.length) {
|
||||
return null;
|
||||
}
|
||||
return this._samples.reduce((sum, sample) => sum + sample, 0) / this._samples.length;
|
||||
}
|
||||
}
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
export type LiveEdgeAction =
|
||||
| { action: 'none' }
|
||||
| { action: 'rate'; rate: number }
|
||||
| { action: 'seek'; seconds: number };
|
||||
|
||||
export interface LiveEdgeStatus {
|
||||
bufferedEndSeconds: number;
|
||||
currentTimeSeconds: number;
|
||||
playbackRate: number;
|
||||
now: Date;
|
||||
}
|
||||
|
||||
// Given the current live-edge lag, returns the action needed to stay near the
|
||||
// edge (a seek, a playback-rate nudge, or nothing).
|
||||
export interface LiveEdgeStrategy {
|
||||
next(status: LiveEdgeStatus): LiveEdgeAction;
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
// WebKit re-buffers whenever `playbackRate` changes, which in practice locks
|
||||
// playback into 1fps slow-motion stutter (see
|
||||
// https://github.com/dermotduffy/advanced-camera-card/issues/2450), so this
|
||||
// strategy never touches the rate and manages position with seeks instead.
|
||||
|
||||
import { GOPCadenceEstimator } from './gop-cadence-estimator';
|
||||
import type { LiveEdgeAction, LiveEdgeStatus, LiveEdgeStrategy } from './types';
|
||||
|
||||
// How many GOPs behind the live edge to hold playback, and the band (seconds)
|
||||
// that hold-back is clamped to so an anomalous GOP estimate cannot pin playback
|
||||
// at the edge or strand it far behind live.
|
||||
const HOLDBACK_GOP_MULTIPLIER = 3;
|
||||
const MIN_HOLDBACK_SECONDS = 1.5;
|
||||
const MAX_HOLDBACK_SECONDS = 8;
|
||||
|
||||
// How far past the hold-back (in GOPs) playback must fall before a forward
|
||||
// catch-up seek, and the minimum gap between such seeks.
|
||||
const CATCHUP_EXCESS_GOP_MULTIPLIER = 2;
|
||||
const JUMP_COOLDOWN_SECONDS = 5;
|
||||
|
||||
// Hold playback a few GOPs behind the live edge. Sitting too close lets the
|
||||
// playhead outrun the bursty per-keyframe delivery and reach the buffered end,
|
||||
// at which point WebKit stalls into a pause a muted stream cannot auto-resume.
|
||||
// The hold-back is sized to the measured GOP cadence, since staying under
|
||||
// roughly three GOPs behind live lets a delivery gap drain the buffer and
|
||||
// stall. Playback is seeked forward only when it has fallen well behind the
|
||||
// hold-back, and back toward it when it drifts within one GOP of the edge.
|
||||
export class WebKitLiveEdgeStrategy implements LiveEdgeStrategy {
|
||||
private _cadence = new GOPCadenceEstimator();
|
||||
private _lastJumpTime: Date | null = null;
|
||||
|
||||
public next(status: LiveEdgeStatus): LiveEdgeAction {
|
||||
this._cadence.sample(status.bufferedEndSeconds, status.now);
|
||||
|
||||
const lag = status.bufferedEndSeconds - status.currentTimeSeconds;
|
||||
const gop = this._cadence.estimateSeconds();
|
||||
const holdback = this._holdbackSeconds(gop);
|
||||
const target = status.bufferedEndSeconds - holdback;
|
||||
|
||||
// Playback has drifted within one GOP of the edge and is about to starve.
|
||||
// WebKit cannot slow down with playbackRate to avoid it, so seek back to the
|
||||
// hold-back to restore runway before it stalls into a pause. Not
|
||||
// cooldown-gated: avoiding the stall takes priority over jump spacing.
|
||||
if (lag < gop) {
|
||||
return { action: 'seek', seconds: target };
|
||||
}
|
||||
|
||||
// Playback has fallen well behind the hold-back (e.g. after a background
|
||||
// tab): jump forward to the hold-back, rate-limited by a cooldown.
|
||||
const secondsSinceLastJump =
|
||||
this._lastJumpTime === null
|
||||
? null
|
||||
: (status.now.getTime() - this._lastJumpTime.getTime()) / 1000;
|
||||
|
||||
if (
|
||||
lag > holdback + CATCHUP_EXCESS_GOP_MULTIPLIER * gop &&
|
||||
(secondsSinceLastJump === null || secondsSinceLastJump >= JUMP_COOLDOWN_SECONDS)
|
||||
) {
|
||||
this._lastJumpTime = status.now;
|
||||
return { action: 'seek', seconds: target };
|
||||
}
|
||||
|
||||
return { action: 'none' };
|
||||
}
|
||||
|
||||
private _holdbackSeconds(gopSeconds: number): number {
|
||||
return Math.min(
|
||||
Math.max(HOLDBACK_GOP_MULTIPLIER * gopSeconds, MIN_HOLDBACK_SECONDS),
|
||||
MAX_HOLDBACK_SECONDS,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
import type { Go2RTCMessage } from '../types';
|
||||
|
||||
// The go2rtc server reports a mode failure as `{ type: 'error', value: '<mode>: ...' }`
|
||||
// (e.g. `mse: stream not found`), so an error is for a given mode when its value
|
||||
// starts with that mode's name.
|
||||
export const isServerErrorForMode = (message: Go2RTCMessage, mode: string): boolean =>
|
||||
message.type === 'error' &&
|
||||
typeof message.value === 'string' &&
|
||||
message.value.startsWith(mode);
|
||||
@@ -0,0 +1,31 @@
|
||||
import type { Lane, StreamProfile } from '../types';
|
||||
|
||||
// Choose which lane to present when both a WebRTC stream and a binary-lane
|
||||
// stream (MSE, MP4, MJPEG) are available for the same camera. The factors are
|
||||
// compared in order of decreasing significance -- the first one that differs
|
||||
// decides, regardless of the rest:
|
||||
// 1. Has video -- a camera view needs a picture before anything else.
|
||||
// 2. Has audio -- a complete stream over a silent one.
|
||||
// 3. H.265 over H.264 -- better quality per bitrate.
|
||||
// If all three are equal the tie goes to WebRTC, which plays as it arrives
|
||||
// (lower latency for a live view) rather than from a buffer.
|
||||
export const getPreferredSource = (
|
||||
webrtc: StreamProfile,
|
||||
binary: StreamProfile,
|
||||
): Lane => {
|
||||
// WebRTC audio is observed from live tracks; the binary side's audio comes
|
||||
// from MSE's negotiated codec string, where only AAC plays reliably, so only
|
||||
// AAC counts as audio on the binary side here (see StreamProfile).
|
||||
const factors: [boolean, boolean][] = [
|
||||
[webrtc.hasVideo, binary.hasVideo],
|
||||
[webrtc.hasAudio, binary.hasAACAudio],
|
||||
[webrtc.hasH265Video, binary.hasH265Video],
|
||||
];
|
||||
|
||||
for (const [webRTCHas, binaryHas] of factors) {
|
||||
if (webRTCHas !== binaryHas) {
|
||||
return webRTCHas ? 'webrtc' : 'binary';
|
||||
}
|
||||
}
|
||||
return 'webrtc';
|
||||
};
|
||||
@@ -0,0 +1,13 @@
|
||||
// Whether the user agent runs the WebKit engine (Safari, and every browser or
|
||||
// WebView on iOS regardless of branding, e.g. Chrome on iOS is `CriOS`).
|
||||
// Blink-based browsers also advertise `AppleWebKit`, but are distinguishable by
|
||||
// their `Chrome/`, `Chromium` or `Android` tokens.
|
||||
export const isWebKitUserAgent = (userAgent: string): boolean =>
|
||||
userAgent.includes('AppleWebKit') && !/Chrome\/|Chromium|Android/.test(userAgent);
|
||||
|
||||
// The Safari major version (from the `Version/<n>` token), or null when the user
|
||||
// agent is not Safari.
|
||||
export const getSafariMajorVersion = (userAgent: string): number | null => {
|
||||
const match = userAgent.match(/Version\/(\d+).+Safari/);
|
||||
return match ? Number(match[1]) : null;
|
||||
};
|
||||
@@ -0,0 +1,6 @@
|
||||
// The SDP (Session Description Protocol) is the text blob describing a WebRTC
|
||||
// session's media. go2rtc advertises an H.265 track with this rtpmap encoding
|
||||
// name, so its presence in the answer SDP means the browser negotiated H.265 --
|
||||
// preferred over H.264 when choosing between the WebRTC and MSE streams (see
|
||||
// https://github.com/dermotduffy/advanced-camera-card/issues/2200).
|
||||
export const sdpHasH265 = (sdp: string): boolean => sdp.includes('H265/90000');
|
||||
@@ -1,5 +1,20 @@
|
||||
import type { MediaUnavailableIssueReason } from '../../../card-controller/issues/issues/media-unavailable';
|
||||
import { fireAdvancedCameraCardEvent } from '../../../utils/fire-advanced-camera-card-event';
|
||||
|
||||
export function dispatchLiveErrorEvent(element: EventTarget): void {
|
||||
fireAdvancedCameraCardEvent(element, 'live:error');
|
||||
declare global {
|
||||
interface HTMLElementEventMap {
|
||||
'advanced-camera-card:live:error': CustomEvent<
|
||||
MediaUnavailableIssueReason | undefined
|
||||
>;
|
||||
}
|
||||
}
|
||||
|
||||
// The optional reason lets a provider that knows why it failed drive a specific
|
||||
// media-unavailable message; absent, the liveness detector falls back to a
|
||||
// generic playback error.
|
||||
export function dispatchLiveErrorEvent(
|
||||
element: EventTarget,
|
||||
reason?: MediaUnavailableIssueReason,
|
||||
): void {
|
||||
fireAdvancedCameraCardEvent(element, 'live:error', reason);
|
||||
}
|
||||
|
||||
@@ -148,7 +148,7 @@ export class MediaActionsController {
|
||||
}
|
||||
}
|
||||
private async _play(index: number): Promise<void> {
|
||||
await (await this._children[index]?.getMediaPlayerController())?.play();
|
||||
await (await this._children[index]?.getMediaPlayerController())?.playback?.play();
|
||||
}
|
||||
private async _unmuteTargetIfConfigured(
|
||||
condition: AutoUnmuteCondition,
|
||||
@@ -204,7 +204,7 @@ export class MediaActionsController {
|
||||
}
|
||||
}
|
||||
private async _pause(index: number): Promise<void> {
|
||||
await (await this._children[index]?.getMediaPlayerController())?.pause();
|
||||
await (await this._children[index]?.getMediaPlayerController())?.playback?.pause();
|
||||
}
|
||||
|
||||
private async _muteAllIfConfigured(condition: AutoMuteCondition): Promise<void> {
|
||||
|
||||
@@ -23,6 +23,11 @@ export interface FrameStallWatchdogConfig {
|
||||
// Stop receiving frames. Defaults to a no-op, for a source that needs no
|
||||
// teardown.
|
||||
stopSource?: () => void;
|
||||
|
||||
// Seconds without a frame (while playback is expected) before a stall is
|
||||
// reported. Defaults to FRAME_STALL_SECONDS; a slow source (e.g. a snapshot
|
||||
// that refreshes every N seconds) needs a window at least as long as N.
|
||||
stallAfterSeconds?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -39,6 +44,7 @@ export interface FrameStallWatchdogConfig {
|
||||
*/
|
||||
export class FrameStallWatchdog {
|
||||
private _config: FrameStallWatchdogConfig;
|
||||
private _stallAfterSeconds: number;
|
||||
|
||||
private _timer = new Timer();
|
||||
private _callbacks = new Set<LivenessCallback>();
|
||||
@@ -51,6 +57,7 @@ export class FrameStallWatchdog {
|
||||
|
||||
constructor(config: FrameStallWatchdogConfig) {
|
||||
this._config = config;
|
||||
this._stallAfterSeconds = config.stallAfterSeconds ?? FRAME_STALL_SECONDS;
|
||||
}
|
||||
|
||||
public subscribe(callback: LivenessCallback): UnsubscribeCallback {
|
||||
@@ -74,7 +81,7 @@ export class FrameStallWatchdog {
|
||||
if (!this._sourceActive) {
|
||||
return;
|
||||
}
|
||||
this._timer.start(FRAME_STALL_SECONDS, () => this._onStall());
|
||||
this._timer.start(this._stallAfterSeconds, () => this._onStall());
|
||||
|
||||
// Notify last: if this recovery notification prompts the final subscriber
|
||||
// to unsubscribe, `_stop` then clears the timer just armed instead of
|
||||
@@ -90,7 +97,7 @@ export class FrameStallWatchdog {
|
||||
// watching begins is still detected. With no source there is nothing to
|
||||
// arm, so nothing is ever reported.
|
||||
if (this._sourceActive) {
|
||||
this._timer.start(FRAME_STALL_SECONDS, () => this._onStall());
|
||||
this._timer.start(this._stallAfterSeconds, () => this._onStall());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -113,7 +120,7 @@ export class FrameStallWatchdog {
|
||||
// Legitimately idle (paused / seeking / ended). Re-arm rather than stop: a
|
||||
// source that later resumes already frozen delivers no frame to kick the
|
||||
// timer, so a freeze that only becomes actionable later is still caught.
|
||||
this._timer.start(FRAME_STALL_SECONDS, () => this._onStall());
|
||||
this._timer.start(this._stallAfterSeconds, () => this._onStall());
|
||||
}
|
||||
|
||||
private _setLive(isLive: boolean): void {
|
||||
|
||||
@@ -1,31 +1,102 @@
|
||||
import type { LitElement } from 'lit';
|
||||
|
||||
import type { FullscreenElement, MediaPlayerController, PIPElement } from '../../types';
|
||||
import type {
|
||||
FullscreenElement,
|
||||
LivenessCallback,
|
||||
MediaPlayerController,
|
||||
PIPElement,
|
||||
PlaybackControl,
|
||||
UnsubscribeCallback,
|
||||
} from '../../types';
|
||||
import { screenshotImage } from '../../utils/screenshot';
|
||||
import { FrameStallWatchdog } from './frame-stall-watchdog';
|
||||
|
||||
// A pausable update loop the controller owns (e.g. an image refreshed on a
|
||||
// timer). Its presence makes the image player pausable.
|
||||
export interface ImageUpdateControl {
|
||||
start(): void;
|
||||
stop(): void;
|
||||
isRunning(): boolean;
|
||||
}
|
||||
|
||||
// Liveness for an image stream: each <img> `load` is a frame; a gap longer than
|
||||
// the window while frames are expected is a stall. `stallWindowSeconds`
|
||||
// defaults to the standard frame-stall window (suits a push-fed stream); a
|
||||
// timer-refreshed image may set a different one, should be at least its refresh
|
||||
// interval.
|
||||
interface ImageLivenessOptions {
|
||||
isFrameExpected: () => boolean;
|
||||
stallWindowSeconds?: number;
|
||||
}
|
||||
|
||||
// Obtaining a screenshot. Defaults to drawing the current <img>; an image
|
||||
// refreshed on a timer can hand back its cached URL instead.
|
||||
type ImageScreenshotProvider = () => Promise<string | null>;
|
||||
|
||||
interface ImageMediaPlayerControllerOptions {
|
||||
updateControl?: ImageUpdateControl;
|
||||
livenessOptions?: ImageLivenessOptions;
|
||||
screenshotProvider?: ImageScreenshotProvider;
|
||||
}
|
||||
|
||||
// Image player composed from opt-in capabilities.
|
||||
export class ImageMediaPlayerController implements MediaPlayerController {
|
||||
private _host: LitElement;
|
||||
private _getImageCallback: () => HTMLImageElement | null;
|
||||
private _screenshotProvider: ImageScreenshotProvider | null;
|
||||
|
||||
constructor(host: LitElement, getImageCallback: () => HTMLImageElement | null) {
|
||||
private _stallWatchdog: FrameStallWatchdog | null = null;
|
||||
private _loadListener: (() => void) | null = null;
|
||||
|
||||
public readonly playback?: PlaybackControl;
|
||||
public readonly subscribeLiveness?: (
|
||||
callback: LivenessCallback,
|
||||
) => UnsubscribeCallback;
|
||||
|
||||
constructor(
|
||||
host: LitElement,
|
||||
getImageCallback: () => HTMLImageElement | null,
|
||||
options?: ImageMediaPlayerControllerOptions,
|
||||
) {
|
||||
this._host = host;
|
||||
this._getImageCallback = getImageCallback;
|
||||
}
|
||||
this._screenshotProvider = options?.screenshotProvider ?? null;
|
||||
|
||||
public async play(): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
const updateControl = options?.updateControl;
|
||||
if (updateControl) {
|
||||
this.playback = {
|
||||
play: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
updateControl.start();
|
||||
},
|
||||
pause: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
updateControl.stop();
|
||||
},
|
||||
isPaused: (): boolean => !updateControl.isRunning(),
|
||||
};
|
||||
}
|
||||
|
||||
public async pause(): Promise<void> {
|
||||
// Not implemented.
|
||||
const livenessOptions = options?.livenessOptions;
|
||||
if (livenessOptions) {
|
||||
const stallWatchdog = new FrameStallWatchdog({
|
||||
isPlaybackExpected: livenessOptions.isFrameExpected,
|
||||
stallAfterSeconds: livenessOptions.stallWindowSeconds,
|
||||
startSource: () => this._startFrameSource(),
|
||||
stopSource: () => this._stopFrameSource(),
|
||||
});
|
||||
this._stallWatchdog = stallWatchdog;
|
||||
this.subscribeLiveness = (callback): UnsubscribeCallback =>
|
||||
stallWatchdog.subscribe(callback);
|
||||
}
|
||||
}
|
||||
|
||||
public async mute(): Promise<void> {
|
||||
// Not implemented.
|
||||
// No audio.
|
||||
}
|
||||
|
||||
public async unmute(): Promise<void> {
|
||||
// Not implemented.
|
||||
// No audio.
|
||||
}
|
||||
|
||||
public isMuted(): boolean {
|
||||
@@ -33,26 +104,17 @@ export class ImageMediaPlayerController implements MediaPlayerController {
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async seek(_seconds: number): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async setControls(_controls: boolean): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
public isPaused(): boolean {
|
||||
// The image could be an MJPEG, so it is always reported unpaused.
|
||||
return false;
|
||||
public async setControls(_controls?: boolean): Promise<void> {
|
||||
// No playback controls to show.
|
||||
}
|
||||
|
||||
public async getScreenshotURL(): Promise<string | null> {
|
||||
await this._host.updateComplete;
|
||||
|
||||
if (this._screenshotProvider) {
|
||||
return this._screenshotProvider();
|
||||
}
|
||||
const image = this._getImageCallback();
|
||||
|
||||
// It might an MJPEG so still need to screenshot it.
|
||||
return image ? screenshotImage(image) : null;
|
||||
}
|
||||
|
||||
@@ -61,6 +123,30 @@ export class ImageMediaPlayerController implements MediaPlayerController {
|
||||
}
|
||||
|
||||
public getPIPElement(): PIPElement | null {
|
||||
// Picture-in-picture is video-only.
|
||||
return null;
|
||||
}
|
||||
|
||||
// The frame source is the image's `load` event: each newly displayed frame
|
||||
// fires it. With no image element there is nothing to observe, so the watchdog
|
||||
// is told there is no source and reports no stall.
|
||||
private _startFrameSource(): boolean {
|
||||
const image = this._getImageCallback();
|
||||
if (!image) {
|
||||
return false;
|
||||
}
|
||||
this._loadListener = (): void => {
|
||||
this._stallWatchdog?.notifyFrame();
|
||||
};
|
||||
image.addEventListener('load', this._loadListener);
|
||||
return true;
|
||||
}
|
||||
|
||||
private _stopFrameSource(): void {
|
||||
const image = this._getImageCallback();
|
||||
if (image && this._loadListener) {
|
||||
image.removeEventListener('load', this._loadListener);
|
||||
}
|
||||
this._loadListener = null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import type {
|
||||
LivenessCallback,
|
||||
MediaPlayerController,
|
||||
PIPElement,
|
||||
PlaybackControl,
|
||||
UnsubscribeCallback,
|
||||
} from '../../types';
|
||||
import { FrameStallWatchdog } from './frame-stall-watchdog';
|
||||
@@ -20,7 +21,7 @@ export class JSMPEGMediaPlayerController implements MediaPlayerController {
|
||||
// watchdog's always-available defaults apply. Playback is expected whenever
|
||||
// not paused; a decode gap for the frame-stall window is then a stall.
|
||||
private _stallWatchdog = new FrameStallWatchdog({
|
||||
isPlaybackExpected: () => !this.isPaused(),
|
||||
isPlaybackExpected: () => !this.playback.isPaused(),
|
||||
});
|
||||
|
||||
constructor(
|
||||
@@ -42,15 +43,22 @@ export class JSMPEGMediaPlayerController implements MediaPlayerController {
|
||||
this._stallWatchdog.notifyFrame();
|
||||
}
|
||||
|
||||
public async play(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
return this._getJSMPEGVideoElementCallback()?.play();
|
||||
}
|
||||
// JSMpeg decodes a live stream it can start and stop.
|
||||
public readonly playback: PlaybackControl = {
|
||||
play: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
return this._getJSMPEGVideoElementCallback()?.play();
|
||||
},
|
||||
|
||||
public async pause(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
return this._getJSMPEGVideoElementCallback()?.stop();
|
||||
}
|
||||
pause: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
return this._getJSMPEGVideoElementCallback()?.stop();
|
||||
},
|
||||
|
||||
isPaused: (): boolean => {
|
||||
return this._getJSMPEGVideoElementCallback()?.player?.paused ?? true;
|
||||
},
|
||||
};
|
||||
|
||||
public async mute(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
@@ -73,20 +81,11 @@ export class JSMPEGMediaPlayerController implements MediaPlayerController {
|
||||
return player ? player.volume === 0 : true;
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async seek(_seconds: number): Promise<void> {
|
||||
// JSMPEG does not support seeking.
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async setControls(_controls: boolean): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
public isPaused(): boolean {
|
||||
return this._getJSMPEGVideoElementCallback()?.player?.paused ?? true;
|
||||
}
|
||||
|
||||
public async getScreenshotURL(): Promise<string | null> {
|
||||
await this._host.updateComplete;
|
||||
return this._getCanvasElementCallback()?.toDataURL('image/jpeg') ?? null;
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
import type { LitElement } from 'lit';
|
||||
|
||||
import type { FullscreenElement, MediaPlayerController, PIPElement } from '../../types';
|
||||
import type { CachedValueController } from '../cached-value-controller';
|
||||
|
||||
export class UpdatingImageMediaPlayerController implements MediaPlayerController {
|
||||
private _host: LitElement;
|
||||
private _getImageCallback: () => HTMLImageElement | null;
|
||||
private _getCachedValueController: () => CachedValueController<string> | null;
|
||||
|
||||
constructor(
|
||||
host: LitElement,
|
||||
getImageCallback: () => HTMLImageElement | null,
|
||||
getCachedValueController: () => CachedValueController<string> | null,
|
||||
) {
|
||||
this._host = host;
|
||||
this._getImageCallback = getImageCallback;
|
||||
this._getCachedValueController = getCachedValueController;
|
||||
}
|
||||
|
||||
public async play(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
this._getCachedValueController()?.startTimer();
|
||||
}
|
||||
|
||||
public async pause(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
this._getCachedValueController()?.stopTimer();
|
||||
}
|
||||
|
||||
public async mute(): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
public async unmute(): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
public isMuted(): boolean {
|
||||
return true;
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async seek(_seconds: number): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
public async setControls(_controls: boolean): Promise<void> {
|
||||
// Not implemented.
|
||||
}
|
||||
|
||||
public isPaused(): boolean {
|
||||
return !this._getCachedValueController()?.hasTimer();
|
||||
}
|
||||
|
||||
public async getScreenshotURL(): Promise<string | null> {
|
||||
await this._host.updateComplete;
|
||||
return this._getCachedValueController()?.getValue() ?? null;
|
||||
}
|
||||
|
||||
public getFullscreenElement(): FullscreenElement | null {
|
||||
return this._getImageCallback() ?? null;
|
||||
}
|
||||
|
||||
public getPIPElement(): PIPElement | null {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import type {
|
||||
LivenessCallback,
|
||||
MediaPlayerController,
|
||||
PIPElement,
|
||||
PlaybackControl,
|
||||
UnsubscribeCallback,
|
||||
} from '../../types';
|
||||
import { hideMediaControlsTemporarily, setControlsOnVideo } from '../../utils/controls';
|
||||
@@ -19,15 +20,22 @@ export class VideoMediaPlayerController implements MediaPlayerController {
|
||||
private _rvfcHandle: number | null = null;
|
||||
private _stallWatchdog = new FrameStallWatchdog({
|
||||
// Playback is expected unless the video is legitimately idle. Seeking /
|
||||
// ended is idle. A paused video is idle only if it holds a current frame --
|
||||
// a genuine user pause; paused with no current frame is not a real pause but
|
||||
// a source mid-reconnect or buffering (nothing to pause on), so playback is
|
||||
// still expected and a missing frame is a stall.
|
||||
// ended is idle. A poster shown with no media loaded is a still-image
|
||||
// surface (e.g. an MJPEG/MP4 poster slideshow) that never presents video
|
||||
// frames, so no frame is ever expected -- distinct from a poster shown over
|
||||
// real media (a loading placeholder), which is still watched. A paused
|
||||
// video is idle only if it holds a current frame -- a genuine user pause;
|
||||
// paused with no current frame is not a real pause but a source
|
||||
// mid-reconnect or buffering (nothing to pause on), so playback is still
|
||||
// expected and a missing frame is a stall.
|
||||
isPlaybackExpected: () => {
|
||||
const video = this._getVideoCallback();
|
||||
if (!video || video.seeking || video.ended) {
|
||||
return false;
|
||||
}
|
||||
if (video.poster && !video.currentSrc && !video.srcObject) {
|
||||
return false;
|
||||
}
|
||||
return !video.paused || video.readyState < HTMLMediaElement.HAVE_CURRENT_DATA;
|
||||
},
|
||||
startSource: () => this._startFrameSource(),
|
||||
@@ -44,36 +52,42 @@ export class VideoMediaPlayerController implements MediaPlayerController {
|
||||
this._getControlsDefaultCallback = getControlsDefaultCallback ?? null;
|
||||
}
|
||||
|
||||
public async play(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
public readonly playback: PlaybackControl = {
|
||||
play: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
|
||||
const video = this._getVideoCallback();
|
||||
if (!video?.play) {
|
||||
return;
|
||||
}
|
||||
const video = this._getVideoCallback();
|
||||
if (!video?.play) {
|
||||
return;
|
||||
}
|
||||
|
||||
// If the play call fails, and the media is not already muted, mute it first
|
||||
// and then try again. This works around some browsers that prevent
|
||||
// auto-play unless the video is muted.
|
||||
try {
|
||||
await video.play();
|
||||
} catch (err: unknown) {
|
||||
if ((err as Error).name === 'NotAllowedError' && !this.isMuted()) {
|
||||
await this.mute();
|
||||
try {
|
||||
await video.play();
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
} catch (e) {
|
||||
// Pass.
|
||||
// If the play call fails, and the media is not already muted, mute it
|
||||
// first and then try again. This works around some browsers that prevent
|
||||
// auto-play unless the video is muted.
|
||||
try {
|
||||
await video.play();
|
||||
} catch (err: unknown) {
|
||||
if ((err as Error).name === 'NotAllowedError' && !this.isMuted()) {
|
||||
await this.mute();
|
||||
try {
|
||||
await video.play();
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
} catch (e) {
|
||||
// Pass.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
public async pause(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
this._getVideoCallback()?.pause();
|
||||
}
|
||||
pause: async (): Promise<void> => {
|
||||
await this._host.updateComplete;
|
||||
this._getVideoCallback()?.pause();
|
||||
},
|
||||
|
||||
isPaused: (): boolean => {
|
||||
return this._getVideoCallback()?.paused ?? true;
|
||||
},
|
||||
};
|
||||
|
||||
public async mute(): Promise<void> {
|
||||
await this._host.updateComplete;
|
||||
@@ -119,10 +133,6 @@ export class VideoMediaPlayerController implements MediaPlayerController {
|
||||
}
|
||||
}
|
||||
|
||||
public isPaused(): boolean {
|
||||
return this._getVideoCallback()?.paused ?? true;
|
||||
}
|
||||
|
||||
public async getScreenshotURL(): Promise<string | null> {
|
||||
await this._host.updateComplete;
|
||||
|
||||
|
||||
@@ -713,7 +713,7 @@ export class MenuButtonController {
|
||||
currentMediaLoadedInfo.mediaPlayerController &&
|
||||
currentMediaLoadedInfo.capabilities?.supportsPause
|
||||
) {
|
||||
const paused = currentMediaLoadedInfo.mediaPlayerController?.isPaused();
|
||||
const paused = currentMediaLoadedInfo.mediaPlayerController?.playback?.isPaused();
|
||||
return {
|
||||
icon: paused ? 'mdi:play' : 'mdi:pause',
|
||||
...config.menu.buttons.play,
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
import type { Notification } from '../../config/schema/actions/types.js';
|
||||
import { TROUBLESHOOTING_URL } from '../../const.js';
|
||||
import { localize } from '../../localize/localize.js';
|
||||
|
||||
// A narrower type (than Notification) for UX consistency across the notifications
|
||||
// shown over a media surface: provider errors, the awaiting-live placeholder, and
|
||||
// the viewer/gallery no-media state.
|
||||
export interface MediaNotificationOptions {
|
||||
// A short heading. A longer explanation goes in `detail`.
|
||||
title: string;
|
||||
|
||||
// The heading icon. Defaults to a generic alert icon.
|
||||
icon?: string;
|
||||
|
||||
// Appended to the heading to identify the media, when it has a title (e.g. the
|
||||
// camera title `: Front Door`). Absent for untitled media (a url or
|
||||
// screensaver image).
|
||||
targetTitle?: string;
|
||||
|
||||
// A longer explanation, shown as the body when the title alone is not enough.
|
||||
detail?: string;
|
||||
|
||||
// Whether to show the retry spinner. A failing media surface is retried, so
|
||||
// this defaults to true.
|
||||
inProgress?: boolean;
|
||||
|
||||
// Whether to show the troubleshooting link. Defaults to true.
|
||||
troubleshooting?: boolean;
|
||||
}
|
||||
|
||||
// The standard notification block shown over a media surface (a camera stream, a
|
||||
// non-camera url/screensaver image, or an empty viewer/gallery): a short titled
|
||||
// heading (with the media title when there is one), an optional longer detail, an
|
||||
// optional troubleshooting link, and a retry spinner.
|
||||
export const createMediaNotification = (
|
||||
options: MediaNotificationOptions,
|
||||
): Notification => ({
|
||||
heading: {
|
||||
icon: options.icon ?? 'mdi:alert-circle',
|
||||
text: options.targetTitle
|
||||
? `${options.title}: ${options.targetTitle}`
|
||||
: options.title,
|
||||
},
|
||||
...(options.detail && { body: { text: options.detail } }),
|
||||
...((options.troubleshooting ?? true) && {
|
||||
link: {
|
||||
url: TROUBLESHOOTING_URL,
|
||||
title: localize('error.troubleshooting'),
|
||||
},
|
||||
}),
|
||||
...((options.inProgress ?? true) && { in_progress: true }),
|
||||
});
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
} from '../ha/web-proxy.js';
|
||||
import type { Endpoint } from '../types.js';
|
||||
import { errorToConsole } from '../utils/basic.js';
|
||||
import { Generation } from '../utils/concurrency/generation.js';
|
||||
|
||||
const PROXY_URL_SIGN_EXPIRY_SECONDS = 24 * 60 * 60;
|
||||
|
||||
@@ -40,11 +41,11 @@ export class SignedURLController implements ReactiveController {
|
||||
|
||||
// Caching and race-condition state.
|
||||
// The targetURL and proxy config are tracked to detect when inputs change and
|
||||
// invalidate the cache. The requestID tracks the most recent valid fetch, to
|
||||
// ensure that older, slower in-flight requests do not overwrite newer ones.
|
||||
// invalidate the cache. The request generation tracks the most recent valid
|
||||
// fetch, so older, slower in-flight requests do not overwrite newer ones.
|
||||
private _targetURL: string | null = null;
|
||||
private _targetProxyConfig: EnabledProxyConfig | null = null;
|
||||
private _requestID = 0;
|
||||
private _requestGeneration = new Generation();
|
||||
|
||||
constructor(
|
||||
host: ReactiveControllerHost,
|
||||
@@ -74,7 +75,7 @@ export class SignedURLController implements ReactiveController {
|
||||
}
|
||||
|
||||
public hostDisconnected(): void {
|
||||
++this._requestID;
|
||||
this._requestGeneration.invalidate();
|
||||
this._value = null;
|
||||
this._error = null;
|
||||
this._cachedAt = null;
|
||||
@@ -88,7 +89,7 @@ export class SignedURLController implements ReactiveController {
|
||||
if (!hass || !endpoint || (!proxyConfig?.enabled && !endpoint.sign)) {
|
||||
// Invalidate any in-flight async work so a stale proxy/sign result cannot
|
||||
// repopulate the controller after inputs have been cleared or disabled.
|
||||
++this._requestID;
|
||||
this._requestGeneration.invalidate();
|
||||
this._value = null;
|
||||
this._error = null;
|
||||
this._targetURL = null;
|
||||
@@ -137,7 +138,7 @@ export class SignedURLController implements ReactiveController {
|
||||
// Mark as in-flight so the `!this._cachedAt` guard above prevents
|
||||
// subsequent hostUpdate() calls from restarting the async work.
|
||||
this._cachedAt = null;
|
||||
const requestID = ++this._requestID;
|
||||
const requestID = this._requestGeneration.next();
|
||||
|
||||
const resolvedEndpoint = await this._proxy(
|
||||
hass,
|
||||
@@ -146,7 +147,7 @@ export class SignedURLController implements ReactiveController {
|
||||
proxyConfig,
|
||||
proxyEndpointOptions,
|
||||
);
|
||||
if (this._isStale(requestID)) {
|
||||
if (!this._requestGeneration.isCurrent(requestID)) {
|
||||
return;
|
||||
}
|
||||
if (!resolvedEndpoint) {
|
||||
@@ -155,7 +156,7 @@ export class SignedURLController implements ReactiveController {
|
||||
}
|
||||
|
||||
const signedURL = await this._sign(hass, resolvedEndpoint);
|
||||
if (this._isStale(requestID)) {
|
||||
if (!this._requestGeneration.isCurrent(requestID)) {
|
||||
return;
|
||||
}
|
||||
if (!signedURL) {
|
||||
@@ -213,10 +214,6 @@ export class SignedURLController implements ReactiveController {
|
||||
}
|
||||
}
|
||||
|
||||
private _isStale(requestID: number): boolean {
|
||||
return this._requestID !== requestID;
|
||||
}
|
||||
|
||||
private _applySuccess(url: string): void {
|
||||
this._value = url;
|
||||
this._error = null;
|
||||
|
||||
@@ -38,7 +38,7 @@ import { getReviewedQueryFilterFromQuery } from '../../view/utils/query-filter.j
|
||||
|
||||
import '../media-filter.js';
|
||||
|
||||
import { renderNoMedia } from '../notification/no-media.js';
|
||||
import { renderNoMediaNotification } from '../notification/media.js';
|
||||
|
||||
import '../surround-basic.js';
|
||||
import '../thumbnail/thumbnail.js';
|
||||
@@ -219,11 +219,13 @@ export class AdvancedCameraCardGallery extends LitElement {
|
||||
</advanced-camera-card-media-filter>`
|
||||
: ''}
|
||||
${!hasItems
|
||||
? renderNoMedia({
|
||||
cameraID: this.viewManagerEpoch?.manager.getView()?.camera ?? null,
|
||||
cameraManager: this.cameraManager ?? null,
|
||||
loading: isLoading,
|
||||
})
|
||||
? renderNoMediaNotification(
|
||||
{
|
||||
cameraID: this.viewManagerEpoch?.manager.getView()?.camera ?? null,
|
||||
inProgress: isLoading,
|
||||
},
|
||||
this.cameraManager,
|
||||
)
|
||||
: html`<advanced-camera-card-gallery-core
|
||||
.hass=${this.hass}
|
||||
.columnWidth=${this._controller.getColumnWidth(
|
||||
|
||||
@@ -12,16 +12,17 @@ import { live } from 'lit/directives/live.js';
|
||||
import { createRef, ref, type Ref } from 'lit/directives/ref.js';
|
||||
|
||||
import { getCameraEntityFromConfig } from '../camera-manager/utils/camera-entity-from-config.js';
|
||||
import type { MediaUnavailableIssueReason } from '../card-controller/issues/issues/media-unavailable.js';
|
||||
import type { IssueTriggerEventData } from '../card-controller/issues/types.js';
|
||||
import { CachedValueController } from '../components-lib/cached-value-controller.js';
|
||||
import { MediaLoadedInfoSourceController } from '../components-lib/media-loaded-info-source-controller.js';
|
||||
import { UpdatingImageMediaPlayerController } from '../components-lib/media-player/updating-image.js';
|
||||
import { ImageMediaPlayerController } from '../components-lib/media-player/image.js';
|
||||
import { createMediaNotification } from '../components-lib/notification/media.js';
|
||||
import { SignedURLController } from '../components-lib/signed-url-controller.js';
|
||||
import type { Notification } from '../config/schema/actions/types.js';
|
||||
import type { CameraConfig } from '../config/schema/cameras.js';
|
||||
import { type ImageBaseConfig, type ImageMode } from '../config/schema/common/image.js';
|
||||
import type { EnabledProxyConfig } from '../config/schema/common/proxy.js';
|
||||
import { TROUBLESHOOTING_URL } from '../const.js';
|
||||
import { isHassDifferent } from '../ha/is-hass-different.js';
|
||||
import type { HomeAssistant } from '../ha/types.js';
|
||||
import defaultImage from '../images/iris-screensaver.jpg';
|
||||
@@ -38,6 +39,14 @@ import {
|
||||
import type { View } from '../view/view.js';
|
||||
import { renderNotificationBlock } from './notification/block.js';
|
||||
|
||||
declare global {
|
||||
interface HTMLElementEventMap {
|
||||
// A private signal to the immediate parent that the media failed.
|
||||
// Non-bubbling.
|
||||
'advanced-camera-card:image-updating-player:error': CustomEvent<MediaUnavailableIssueReason>;
|
||||
}
|
||||
}
|
||||
|
||||
// See TOKEN_CHANGE_INTERVAL in https://github.com/home-assistant/core/blob/dev/homeassistant/components/camera/__init__.py .
|
||||
const HASS_REJECTION_CUTOFF_MS = 5 * 60 * 1000;
|
||||
|
||||
@@ -91,6 +100,10 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
@property({ attribute: false })
|
||||
public targetID?: string;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
@property({ attribute: false, hasChanged: contentsChanged })
|
||||
public proxyConfig?: EnabledProxyConfig;
|
||||
|
||||
@@ -103,6 +116,10 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
@state()
|
||||
private _imageLoadError = false;
|
||||
|
||||
// Tracks the signed/proxy error so it is reported once on the transition into
|
||||
// failure, not on every update.
|
||||
private _hasSignError = false;
|
||||
|
||||
private _refImage: Ref<HTMLImageElement> = createRef();
|
||||
|
||||
private _cachedValueController = new CachedValueController(
|
||||
@@ -144,10 +161,19 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
|
||||
private _boundVisibilityHandler = this._visibilityHandler.bind(this);
|
||||
|
||||
private _mediaPlayerController = new UpdatingImageMediaPlayerController(
|
||||
// A poll-refreshed snapshot: the cached-value timer is the pausable update
|
||||
// loop, and its cached URL is the screenshot.
|
||||
private _mediaPlayerController = new ImageMediaPlayerController(
|
||||
this,
|
||||
() => this._refImage.value ?? null,
|
||||
() => this._cachedValueController,
|
||||
{
|
||||
updateControl: {
|
||||
start: () => this._cachedValueController.startTimer(),
|
||||
stop: () => this._cachedValueController.stopTimer(),
|
||||
isRunning: () => this._cachedValueController.hasTimer(),
|
||||
},
|
||||
screenshotProvider: async () => this._cachedValueController.getValue(),
|
||||
},
|
||||
);
|
||||
|
||||
private _mediaLoadedInfoSourceController = new MediaLoadedInfoSourceController(this, {
|
||||
@@ -218,6 +244,19 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
if (!this._cachedValueController?.getValue()) {
|
||||
this._cachedValueController?.updateValue();
|
||||
}
|
||||
|
||||
const hasSignError = !!this._signedURLController.getError();
|
||||
if (hasSignError && !this._hasSignError) {
|
||||
this._dispatchError('server_error');
|
||||
}
|
||||
this._hasSignError = hasSignError;
|
||||
}
|
||||
|
||||
private _dispatchError(reason: MediaUnavailableIssueReason): void {
|
||||
fireAdvancedCameraCardEvent(this, 'image-updating-player:error', reason, {
|
||||
bubbles: false,
|
||||
composed: false,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -411,24 +450,16 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
private _getDisplayNotification(): Notification | null {
|
||||
const error = this._signedURLController.getError();
|
||||
if (error) {
|
||||
return {
|
||||
heading: {
|
||||
text: localize(error === 'proxy' ? 'error.failed_proxy' : 'error.failed_sign'),
|
||||
icon: 'mdi:alert-circle',
|
||||
},
|
||||
link: { url: TROUBLESHOOTING_URL, title: localize('error.troubleshooting') },
|
||||
context: this.proxyConfig ? [this.proxyConfig] : undefined,
|
||||
};
|
||||
return createMediaNotification({
|
||||
title: localize(error === 'proxy' ? 'error.failed_proxy' : 'error.failed_sign'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
if (this._imageLoadError) {
|
||||
return {
|
||||
heading: {
|
||||
text: localize('error.image_load_error'),
|
||||
icon: 'mdi:alert-circle',
|
||||
},
|
||||
link: { url: TROUBLESHOOTING_URL, title: localize('error.troubleshooting') },
|
||||
context: this.imageConfig ? [this.imageConfig] : undefined,
|
||||
};
|
||||
return createMediaNotification({
|
||||
title: localize('error.image_load_error'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -468,6 +499,11 @@ export class AdvancedCameraCardImageUpdatingPlayer
|
||||
this._forceSafeImage(true);
|
||||
} else if (mode === 'url') {
|
||||
this._imageLoadError = true;
|
||||
|
||||
// Report the failure to the parent. A live context marks the
|
||||
// stream not-live so its wrapper stops covering the error with a
|
||||
// loading overlay; the plain image view ignores it.
|
||||
this._dispatchError('not_loading');
|
||||
}
|
||||
if (this.targetID) {
|
||||
fireAdvancedCameraCardEvent<IssueTriggerEventData>(
|
||||
|
||||
@@ -275,7 +275,7 @@ export class AdvancedCameraCardLiveCarousel extends LitElement {
|
||||
.microphoneStream=${microphoneStream}
|
||||
.camera=${resolvedCamera}
|
||||
.targetID=${cameraID}
|
||||
.label=${cameraMetadata?.title ?? ''}
|
||||
.cameraTitle=${cameraMetadata?.title}
|
||||
.liveConfig=${this.liveConfig}
|
||||
.hass=${this.hass}
|
||||
.stateWatcher=${this.stateWatcher}
|
||||
@@ -284,6 +284,7 @@ export class AdvancedCameraCardLiveCarousel extends LitElement {
|
||||
.zoom=${!this._isGesturesPTZActive(view, cameraID)}
|
||||
.forceSelected=${isSelectedSlide}
|
||||
.locked=${this.locked}
|
||||
.suppressLoadingImage=${mediaEpoch > 0}
|
||||
@advanced-camera-card:zoom:change=${(
|
||||
ev: CustomEvent<ZoomSettingsObserved>,
|
||||
) =>
|
||||
|
||||
@@ -34,7 +34,7 @@ import { getResolvedLiveProvider } from '../../utils/live-provider.js';
|
||||
|
||||
import '../icon.js';
|
||||
|
||||
import { renderNotificationBlockFromText } from '../notification/block.js';
|
||||
import { renderMediaNotification } from '../notification/media.js';
|
||||
|
||||
import './../media-dimensions-container';
|
||||
|
||||
@@ -56,9 +56,10 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
@property({ attribute: false })
|
||||
public liveConfig?: LiveConfig;
|
||||
|
||||
// Label that is used for ARIA support and as tooltip.
|
||||
// The camera's title, used for ARIA support, as tooltip, and to identify the
|
||||
// camera in error messages.
|
||||
@property({ attribute: false })
|
||||
public label = '';
|
||||
public cameraTitle?: string;
|
||||
|
||||
@property({ attribute: false })
|
||||
public cardWideConfig?: CardWideConfig;
|
||||
@@ -85,6 +86,12 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
@property({ attribute: false })
|
||||
public locked?: boolean;
|
||||
|
||||
// When true, suppress the loading snapshot (show_image_during_load). Set on a
|
||||
// media reload after a failure so the snapshot doesn't flash back in on every
|
||||
// retry; a first load still shows it.
|
||||
@property({ attribute: false })
|
||||
public suppressLoadingImage = false;
|
||||
|
||||
private _mediaLoadedInfoSinkController = new MediaLoadedInfoSinkController(this, {
|
||||
getTargetID: () => this.targetID ?? null,
|
||||
});
|
||||
@@ -127,6 +134,7 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
*/
|
||||
private _shouldShowImageDuringLoading(): boolean {
|
||||
return (
|
||||
!this.suppressLoadingImage &&
|
||||
!this._mediaLoadedInfoSinkController.has() &&
|
||||
!!this.camera?.getConfig()?.camera_entity &&
|
||||
!!this.hass &&
|
||||
@@ -167,6 +175,8 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
this._importPromises.push(import('./providers/image.js'));
|
||||
} else if (provider === 'go2rtc') {
|
||||
this._importPromises.push(import('./providers/go2rtc/index.js'));
|
||||
} else if (provider === 'go2rtc-experimental') {
|
||||
this._importPromises.push(import('./providers/go2rtc-experimental/index.js'));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -239,57 +249,70 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
// being initialized. This can cause spurious errors (e.g. lack of resolved
|
||||
// endpoints). Instead, simply never render uninitialized cameras.
|
||||
if (!this.camera.isInitialized()) {
|
||||
return renderNotificationBlockFromText(
|
||||
`${localize('error.awaiting_live')}${this.label ? `: ${this.label}` : ''}`,
|
||||
{ icon: 'mdi:progress-helper', in_progress: true },
|
||||
);
|
||||
return renderMediaNotification({
|
||||
icon: 'mdi:progress-helper',
|
||||
title: localize('error.awaiting_live'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
|
||||
// Set title and ariaLabel from the provided label property.
|
||||
this.title = this.label;
|
||||
this.ariaLabel = this.label;
|
||||
this.title = this.cameraTitle ?? '';
|
||||
this.ariaLabel = this.cameraTitle ?? '';
|
||||
|
||||
const provider = getResolvedLiveProvider(this.camera?.getConfig());
|
||||
|
||||
// `ha`/`image` cannot stream without a camera entity, so validate that
|
||||
// here. Entity *availability* (including the always_error immediate path)
|
||||
// is owned by the liveness controller's EntityAvailabilityDetector and
|
||||
// surfaces via getPlaceholder() below, for all providers.
|
||||
// surfaces via getFailure() below, for all providers.
|
||||
if (
|
||||
provider === 'ha' ||
|
||||
provider === 'image' ||
|
||||
(cameraConfig?.camera_entity && cameraConfig.always_error_if_entity_unavailable)
|
||||
) {
|
||||
if (!cameraConfig?.camera_entity) {
|
||||
return renderNotificationBlockFromText(localize('error.no_live_camera'), {
|
||||
return renderMediaNotification({
|
||||
icon: 'mdi:camera',
|
||||
context: cameraConfig,
|
||||
title: localize('error.configuration_error'),
|
||||
detail: localize('error.no_live_camera'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
if (!this.hass.states[cameraConfig.camera_entity]) {
|
||||
return renderNotificationBlockFromText(localize('error.live_camera_not_found'), {
|
||||
return renderMediaNotification({
|
||||
icon: 'mdi:camera',
|
||||
context: cameraConfig,
|
||||
title: localize('error.configuration_error'),
|
||||
detail: localize('error.live_camera_not_found'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const failure = this._streamLivenessController.getFailure();
|
||||
|
||||
// A detector reports the stream is silently lost (the camera entity is
|
||||
// unavailable, or the stream stalled): render a reconnecting placeholder,
|
||||
// which unmounts the provider and unloads it via the existing media-loaded
|
||||
// abort. The message names the specific cause.
|
||||
const placeholder = this._streamLivenessController.getPlaceholder();
|
||||
if (placeholder) {
|
||||
if (failure?.renderPlaceholder) {
|
||||
const { localizationKey: textKey, icon } =
|
||||
MEDIA_UNAVAILABLE_REASONS[placeholder.reason];
|
||||
return renderNotificationBlockFromText(
|
||||
`${localize(textKey)}${this.label ? `: ${this.label}` : ''}`,
|
||||
{ icon, in_progress: true },
|
||||
);
|
||||
MEDIA_UNAVAILABLE_REASONS[failure.reason];
|
||||
return renderMediaNotification({
|
||||
icon,
|
||||
title: localize(textKey),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
|
||||
const showImageDuringLoading = this._shouldShowImageDuringLoading();
|
||||
const showLoadingIcon = !this._mediaLoadedInfoSinkController.has();
|
||||
const mediaLoaded = this._mediaLoadedInfoSinkController.has();
|
||||
|
||||
// Loaded media or a snapshot gives the frame a size; mark the host `sized`
|
||||
// when one is present. In its absence CSS reserves an aspect ratio so the
|
||||
// frame (whose loading/error fill is absolutely positioned) doesn't
|
||||
// collapse.
|
||||
this.toggleAttribute('sized', mediaLoaded || showImageDuringLoading);
|
||||
|
||||
const classes = {
|
||||
hidden: showImageDuringLoading,
|
||||
@@ -302,6 +325,7 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
.hass=${this.hass}
|
||||
.camera=${this.camera}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
class=${classMap({
|
||||
...classes,
|
||||
// The image provider is providing the temporary loading image,
|
||||
@@ -340,42 +364,75 @@ export class AdvancedCameraCardLiveProvider extends LitElement implements MediaP
|
||||
.hass=${this.hass}
|
||||
.camera=${this.camera}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
.microphoneStream=${this.microphoneStream}
|
||||
.microphoneConfig=${this.liveConfig.microphone}
|
||||
?controls=${this._getEffectiveBuiltinControls()}
|
||||
>
|
||||
</advanced-camera-card-live-go2rtc>`
|
||||
: provider === 'webrtc-card'
|
||||
? html`<advanced-camera-card-live-webrtc-card
|
||||
: provider === 'go2rtc-experimental'
|
||||
? html`<advanced-camera-card-live-go2rtc-experimental
|
||||
${ref(this._refProvider)}
|
||||
class=${classMap(classes)}
|
||||
.hass=${this.hass}
|
||||
.camera=${this.camera}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
.microphoneStream=${this.microphoneStream}
|
||||
.microphoneConfig=${this.liveConfig.microphone}
|
||||
.cardWideConfig=${this.cardWideConfig}
|
||||
?controls=${this._getEffectiveBuiltinControls()}
|
||||
>
|
||||
</advanced-camera-card-live-webrtc-card>`
|
||||
: provider === 'jsmpeg'
|
||||
? html` <advanced-camera-card-live-jsmpeg
|
||||
</advanced-camera-card-live-go2rtc-experimental>`
|
||||
: provider === 'webrtc-card'
|
||||
? html`<advanced-camera-card-live-webrtc-card
|
||||
${ref(this._refProvider)}
|
||||
class=${classMap(classes)}
|
||||
.hass=${this.hass}
|
||||
.camera=${this.camera}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
.cardWideConfig=${this.cardWideConfig}
|
||||
?controls=${this._getEffectiveBuiltinControls()}
|
||||
>
|
||||
</advanced-camera-card-live-jsmpeg>`
|
||||
: html``}
|
||||
</advanced-camera-card-live-webrtc-card>`
|
||||
: provider === 'jsmpeg'
|
||||
? html` <advanced-camera-card-live-jsmpeg
|
||||
${ref(this._refProvider)}
|
||||
class=${classMap(classes)}
|
||||
.hass=${this.hass}
|
||||
.camera=${this.camera}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
.cardWideConfig=${this.cardWideConfig}
|
||||
>
|
||||
</advanced-camera-card-live-jsmpeg>`
|
||||
: html``}
|
||||
`)}
|
||||
${showLoadingIcon
|
||||
? html`<advanced-camera-card-icon
|
||||
title=${localize('error.awaiting_live')}
|
||||
.icon=${{ icon: 'mdi:progress-helper' }}
|
||||
@click=${() =>
|
||||
fireAdvancedCameraCardEvent(this, 'issue:notify', 'media_unavailable')}
|
||||
></advanced-camera-card-icon>`
|
||||
: ''}`;
|
||||
${failure || mediaLoaded ? '' : this._renderLoadingOverlay(showImageDuringLoading)}`;
|
||||
}
|
||||
|
||||
// The loading status drawn on top of the mounted provider while its media has
|
||||
// not loaded: a subtle corner spinner over a snapshot that is already filling
|
||||
// the frame, or a full "waiting for live" state. The cases that render nothing
|
||||
// (a failure, or media already loaded) are handled at the call site.
|
||||
private _renderLoadingOverlay(showImageDuringLoading: boolean): TemplateResult {
|
||||
if (showImageDuringLoading) {
|
||||
return html`<advanced-camera-card-icon
|
||||
title=${localize('error.awaiting_live')}
|
||||
.icon=${{ icon: 'mdi:progress-helper' }}
|
||||
@click=${() =>
|
||||
fireAdvancedCameraCardEvent(this, 'issue:notify', 'media_unavailable')}
|
||||
></advanced-camera-card-icon>`;
|
||||
}
|
||||
|
||||
return html`<div class="fill">
|
||||
${renderMediaNotification({
|
||||
icon: 'mdi:progress-helper',
|
||||
title: localize('error.awaiting_live'),
|
||||
targetTitle: this.cameraTitle,
|
||||
})}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
static get styles(): CSSResultGroup {
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
import {
|
||||
html,
|
||||
LitElement,
|
||||
unsafeCSS,
|
||||
type CSSResultGroup,
|
||||
type PropertyValues,
|
||||
type TemplateResult,
|
||||
} from 'lit';
|
||||
import { customElement, property, state } from 'lit/decorators.js';
|
||||
import { createRef, ref, type Ref } from 'lit/directives/ref.js';
|
||||
|
||||
import type { Camera } from '../../../../camera-manager/camera.js';
|
||||
import {
|
||||
MEDIA_UNAVAILABLE_REASONS,
|
||||
type MediaUnavailableIssueReason,
|
||||
} from '../../../../card-controller/issues/issues/media-unavailable.js';
|
||||
import { ImageSurfaceController } from '../../../../components-lib/live/providers/go2rtc-experimental/image-surface-controller.js';
|
||||
import {
|
||||
Go2RTCSessionController,
|
||||
type SessionSurfaces,
|
||||
type VideoSurface,
|
||||
} from '../../../../components-lib/live/providers/go2rtc-experimental/session-controller.js';
|
||||
import type { SurfaceKind } from '../../../../components-lib/live/providers/go2rtc-experimental/types.js';
|
||||
import { mapFailureReasonToIssueReason } from '../../../../components-lib/live/providers/go2rtc-experimental/utils/failure-reason.js';
|
||||
import { dispatchLiveErrorEvent } from '../../../../components-lib/live/utils/dispatch-live-error.js';
|
||||
import { MediaLoadedInfoSourceController } from '../../../../components-lib/media-loaded-info-source-controller.js';
|
||||
import { VideoMediaPlayerController } from '../../../../components-lib/media-player/video.js';
|
||||
import { SignedURLController } from '../../../../components-lib/signed-url-controller.js';
|
||||
import type { MicrophoneConfig } from '../../../../config/schema/live.js';
|
||||
import type { CardWideConfig } from '../../../../config/schema/types.js';
|
||||
import type { HomeAssistant } from '../../../../ha/types.js';
|
||||
import { localize } from '../../../../localize/localize.js';
|
||||
import liveGo2RTCExperimentalStyle from '../../../../scss/live-go2rtc-experimental.scss';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../../../../types.js';
|
||||
import {
|
||||
dispatchMediaPauseEvent,
|
||||
dispatchMediaPlayEvent,
|
||||
dispatchMediaVolumeChangeEvent,
|
||||
} from '../../../../utils/media-info.js';
|
||||
import { renderMediaNotification } from '../../../notification/media.js';
|
||||
|
||||
@customElement('advanced-camera-card-live-go2rtc-experimental')
|
||||
export class AdvancedCameraCardGo2RTCExperimental
|
||||
extends LitElement
|
||||
implements MediaPlayer
|
||||
{
|
||||
// Not a reactive property to avoid resetting the video.
|
||||
public hass?: HomeAssistant;
|
||||
|
||||
@property({ attribute: false })
|
||||
public camera?: Camera;
|
||||
|
||||
// The BASE camera ID (camera property may be a substream)
|
||||
@property({ attribute: false })
|
||||
public targetID?: string;
|
||||
|
||||
@property({ attribute: false })
|
||||
public microphoneStream?: MediaStream | null;
|
||||
|
||||
@property({ attribute: false })
|
||||
public microphoneConfig?: MicrophoneConfig;
|
||||
|
||||
@property({ attribute: false })
|
||||
public cardWideConfig?: CardWideConfig;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
@property({ attribute: true, type: Boolean })
|
||||
public controls = false;
|
||||
|
||||
private _hasLiveError = false;
|
||||
|
||||
// ===========================================================================
|
||||
// Surface: Video
|
||||
// ===========================================================================
|
||||
private _refVideo: Ref<HTMLVideoElement> = createRef();
|
||||
|
||||
private _videoMediaPlayerController = new VideoMediaPlayerController(
|
||||
this,
|
||||
() => this._refVideo.value ?? null,
|
||||
() => this.controls,
|
||||
);
|
||||
|
||||
private _videoSurface: VideoSurface = {
|
||||
getElement: () => this._refVideo.value ?? null,
|
||||
getMediaPlayer: () => this._videoMediaPlayerController,
|
||||
};
|
||||
|
||||
// ===========================================================================
|
||||
// Surface: Image
|
||||
// ===========================================================================
|
||||
|
||||
private _refImage: Ref<HTMLImageElement> = createRef();
|
||||
|
||||
// A controller rather than a plain object (unlike the video surface): the
|
||||
// image surface owns state, the object-URL lifecycle -- each frame's
|
||||
// createObjectURL and revoking the previous one.
|
||||
private _imageSurface = new ImageSurfaceController(
|
||||
this,
|
||||
() => this._refImage.value ?? null,
|
||||
{
|
||||
livenessOptions: {
|
||||
isFrameExpected: () => true,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
// ===========================================================================
|
||||
// Surface Management
|
||||
// ===========================================================================
|
||||
|
||||
// The surface currently showing committed media, or null before anything has
|
||||
// committed (both surfaces hidden). Driven by the session's
|
||||
// surfaceCommittedCallback.
|
||||
@state()
|
||||
private _activeSurface: SurfaceKind | null = null;
|
||||
|
||||
@state()
|
||||
private _streamError: MediaUnavailableIssueReason | null = null;
|
||||
|
||||
// Built once and kept stable: the session compares this object by identity,
|
||||
// so handing it a new one will trigger a reconnect.
|
||||
private _surfaces: SessionSurfaces = {
|
||||
video: this._videoSurface,
|
||||
image: this._imageSurface,
|
||||
};
|
||||
|
||||
private _signedURLController = new SignedURLController(this, () => {
|
||||
const endpoint = this.camera?.getEndpoints()?.go2rtc;
|
||||
if (!this.hass || !endpoint) {
|
||||
return {};
|
||||
}
|
||||
return {
|
||||
hass: this.hass,
|
||||
endpoint,
|
||||
proxyConfig: this.camera?.getLiveProxyConfig(),
|
||||
proxyEndpointOptions: { websocket: true },
|
||||
};
|
||||
});
|
||||
|
||||
private _mediaLoadedInfoSourceController = new MediaLoadedInfoSourceController(this, {
|
||||
getTargetID: () => this.targetID ?? null,
|
||||
});
|
||||
|
||||
private _session = new Go2RTCSessionController({
|
||||
getControls: () => this.controls,
|
||||
getCardWideConfig: () => this.cardWideConfig ?? null,
|
||||
mediaLoadedCallback: (info) => this._mediaLoadedInfoSourceController.set(info),
|
||||
|
||||
surfaceCommittedCallback: (surface) => {
|
||||
this._activeSurface = surface;
|
||||
|
||||
// A commit means the stream recovered: drop any prior error.
|
||||
this._streamError = null;
|
||||
},
|
||||
|
||||
// The session could not recover the stream on its own; surface it (with the
|
||||
// failure's user-facing cause) so the card's media-load retry (reconnecting
|
||||
// indicator, backoff, give-up) runs and can name why. The provider renders
|
||||
// the error itself (below); the event drives the liveness verdict + retry.
|
||||
errorCallback: (reason) => {
|
||||
this._streamError = mapFailureReasonToIssueReason(reason);
|
||||
dispatchLiveErrorEvent(this, this._streamError);
|
||||
},
|
||||
});
|
||||
|
||||
public async getMediaPlayerController(): Promise<MediaPlayerController | null> {
|
||||
return this._activeSurface === 'image'
|
||||
? this._imageSurface.getMediaPlayer()
|
||||
: this._videoMediaPlayerController;
|
||||
}
|
||||
|
||||
connectedCallback(): void {
|
||||
super.connectedCallback();
|
||||
|
||||
// Re-render (and thus re-establish the session) when reconnected to the
|
||||
// DOM. https://github.com/dermotduffy/advanced-camera-card/issues/996
|
||||
this.requestUpdate();
|
||||
}
|
||||
|
||||
disconnectedCallback(): void {
|
||||
// Tear down synchronously so streams (e.g. 2-way audio backchannels)
|
||||
// release immediately.
|
||||
this._session.reset();
|
||||
this._activeSurface = null;
|
||||
super.disconnectedCallback();
|
||||
}
|
||||
|
||||
protected willUpdate(changedProps: PropertyValues): void {
|
||||
if (changedProps.has('camera')) {
|
||||
// The session is re-established by `updated()` once the new camera's
|
||||
// signed URL resolves; the next commit picks the live surface. Blank the
|
||||
// view meanwhile so the previous camera's last frame is not shown.
|
||||
this._session.reset();
|
||||
this._activeSurface = null;
|
||||
this._streamError = null;
|
||||
}
|
||||
|
||||
// Only treat a missing go2rtc endpoint as an error after the camera's
|
||||
// endpoints have been explicitly set (not undefined / still loading).
|
||||
const endpoints = this.camera?.getEndpoints();
|
||||
const hasLiveError =
|
||||
!!this._signedURLController.getError() || (!!endpoints && !endpoints.go2rtc);
|
||||
|
||||
if (hasLiveError && !this._hasLiveError) {
|
||||
dispatchLiveErrorEvent(this);
|
||||
}
|
||||
this._hasLiveError = hasLiveError;
|
||||
|
||||
if (changedProps.has('controls')) {
|
||||
// Only the video surface has native controls; the image surface has none.
|
||||
this._videoMediaPlayerController.setControls(this.controls).catch(() => {});
|
||||
}
|
||||
|
||||
if (changedProps.has('microphoneStream')) {
|
||||
// The WebRTC lane swaps the outbound track in place; no visible reload.
|
||||
this._session.setMicrophoneStream(this.microphoneStream ?? null);
|
||||
}
|
||||
}
|
||||
|
||||
protected updated(): void {
|
||||
const url = this._signedURLController.getValue();
|
||||
if (url) {
|
||||
this._session.connect(
|
||||
url,
|
||||
this._surfaces,
|
||||
this.camera?.getConfig()?.go2rtc?.modes,
|
||||
);
|
||||
} else {
|
||||
// No usable URL: a signing/proxy error, or no go2rtc endpoint (which
|
||||
// includes endpoints still loading). The render omits the surfaces, so
|
||||
// drop the session -- otherwise a later URL, even an identical unsigned
|
||||
// endpoint, would be skipped by connect()'s identity check and leave the
|
||||
// session bound to the removed elements.
|
||||
this._session.reset();
|
||||
this._activeSurface = null;
|
||||
}
|
||||
}
|
||||
|
||||
protected render(): TemplateResult | void {
|
||||
const error = this._signedURLController.getError();
|
||||
if (error) {
|
||||
return renderMediaNotification({
|
||||
title: localize(error === 'proxy' ? 'error.failed_proxy' : 'error.failed_sign'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
if (!this.camera?.getEndpoints()?.go2rtc) {
|
||||
return renderMediaNotification({
|
||||
title: localize('error.configuration_error'),
|
||||
detail: localize('error.live_camera_no_endpoint'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
if (this._streamError) {
|
||||
// A stream-level failure the session gave up on: the provider must render
|
||||
// its own error (marked in-progress, since the card keeps retrying).
|
||||
return renderMediaNotification({
|
||||
icon: MEDIA_UNAVAILABLE_REASONS[this._streamError].icon,
|
||||
title: localize(MEDIA_UNAVAILABLE_REASONS[this._streamError].localizationKey),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
|
||||
// Both image and video surfaces are always rendered; only the committed one
|
||||
// is shown (the other, and both before anything commits, are hidden). MSE
|
||||
// and WebRTC play on the <video>; MP4 and MJPEG show frames on the <img>.
|
||||
//
|
||||
// Muted is bound as a property: Chrome ignores the `muted` content
|
||||
// attribute on videos instantiated from cloned templates (as Lit does), so
|
||||
// an attribute would not actually start the video muted. Media may be
|
||||
// unmuted later in accordance with user configuration.
|
||||
return html`
|
||||
<video
|
||||
${ref(this._refVideo)}
|
||||
.muted=${true}
|
||||
?hidden=${this._activeSurface !== 'video'}
|
||||
playsinline
|
||||
preload="auto"
|
||||
@play=${() => dispatchMediaPlayEvent(this)}
|
||||
@pause=${() => dispatchMediaPauseEvent(this)}
|
||||
@volumechange=${() => dispatchMediaVolumeChangeEvent(this)}
|
||||
></video>
|
||||
<img ${ref(this._refImage)} ?hidden=${this._activeSurface !== 'image'} alt="" />
|
||||
`;
|
||||
}
|
||||
|
||||
static get styles(): CSSResultGroup {
|
||||
return unsafeCSS(liveGo2RTCExperimentalStyle);
|
||||
}
|
||||
}
|
||||
|
||||
declare global {
|
||||
interface HTMLElementTagNameMap {
|
||||
'advanced-camera-card-live-go2rtc-experimental': AdvancedCameraCardGo2RTCExperimental;
|
||||
}
|
||||
}
|
||||
@@ -17,7 +17,7 @@ import type { HomeAssistant } from '../../../../ha/types.js';
|
||||
import { localize } from '../../../../localize/localize.js';
|
||||
import liveGo2RTCStyle from '../../../../scss/live-go2rtc.scss';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../../../../types.js';
|
||||
import { renderNotificationBlockFromText } from '../../../notification/block.js';
|
||||
import { renderMediaNotification } from '../../../notification/media.js';
|
||||
import { VideoRTC } from './video-rtc.js';
|
||||
|
||||
customElements.define('advanced-camera-card-live-go2rtc-player', VideoRTC);
|
||||
@@ -40,6 +40,10 @@ export class AdvancedCameraCardGo2RTC extends LitElement implements MediaPlayer
|
||||
@property({ attribute: false })
|
||||
public microphoneConfig?: MicrophoneConfig;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
@property({ attribute: true, type: Boolean })
|
||||
public controls = false;
|
||||
|
||||
@@ -148,14 +152,16 @@ export class AdvancedCameraCardGo2RTC extends LitElement implements MediaPlayer
|
||||
protected render(): TemplateResult | void {
|
||||
const error = this._signedURLController.getError();
|
||||
if (error) {
|
||||
return renderNotificationBlockFromText(
|
||||
localize(error === 'proxy' ? 'error.failed_proxy' : 'error.failed_sign'),
|
||||
{ context: this.camera?.getConfig() },
|
||||
);
|
||||
return renderMediaNotification({
|
||||
title: localize(error === 'proxy' ? 'error.failed_proxy' : 'error.failed_sign'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
if (!this.camera?.getEndpoints()?.go2rtc) {
|
||||
return renderNotificationBlockFromText(localize('error.live_camera_no_endpoint'), {
|
||||
context: this.camera?.getConfig(),
|
||||
return renderMediaNotification({
|
||||
title: localize('error.configuration_error'),
|
||||
detail: localize('error.live_camera_no_endpoint'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
}
|
||||
return html`${this._player}`;
|
||||
|
||||
@@ -9,6 +9,8 @@ import { customElement, property } from 'lit/decorators.js';
|
||||
import { createRef, ref, type Ref } from 'lit/directives/ref.js';
|
||||
|
||||
import type { Camera } from '../../../camera-manager/camera.js';
|
||||
import type { MediaUnavailableIssueReason } from '../../../card-controller/issues/issues/media-unavailable.js';
|
||||
import { dispatchLiveErrorEvent } from '../../../components-lib/live/utils/dispatch-live-error.js';
|
||||
import type { HomeAssistant } from '../../../ha/types';
|
||||
import basicBlockStyle from '../../../scss/basic-block.scss';
|
||||
import type {
|
||||
@@ -31,6 +33,10 @@ export class AdvancedCameraCardLiveImage extends LitElement implements MediaPlay
|
||||
@property({ attribute: false })
|
||||
public targetID?: string;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
private _refImage: Ref<MediaPlayerElement> = createRef();
|
||||
|
||||
public async getMediaPlayerController(): Promise<MediaPlayerController | null> {
|
||||
@@ -51,7 +57,11 @@ export class AdvancedCameraCardLiveImage extends LitElement implements MediaPlay
|
||||
.imageConfig=${cameraConfig.image}
|
||||
.cameraConfig=${cameraConfig}
|
||||
.targetID=${this.targetID}
|
||||
.cameraTitle=${this.cameraTitle}
|
||||
.proxyConfig=${this.camera?.getLiveProxyConfig()}
|
||||
@advanced-camera-card:image-updating-player:error=${(
|
||||
ev: CustomEvent<MediaUnavailableIssueReason>,
|
||||
) => dispatchLiveErrorEvent(this, ev.detail)}
|
||||
>
|
||||
</advanced-camera-card-image-updating-player>
|
||||
`;
|
||||
|
||||
@@ -14,7 +14,7 @@ import type { Camera } from '../../../camera-manager/camera.js';
|
||||
import { dispatchLiveErrorEvent } from '../../../components-lib/live/utils/dispatch-live-error.js';
|
||||
import { MediaLoadedInfoSourceController } from '../../../components-lib/media-loaded-info-source-controller.js';
|
||||
import { JSMPEGMediaPlayerController } from '../../../components-lib/media-player/jsmpeg.js';
|
||||
import { createNotificationFromText } from '../../../components-lib/notification/factory.js';
|
||||
import { createMediaNotification } from '../../../components-lib/notification/media.js';
|
||||
import type { Notification } from '../../../config/schema/actions/types.js';
|
||||
import type { CardWideConfig } from '../../../config/schema/types.js';
|
||||
import { homeAssistantGetSignedURLIfNecessary } from '../../../ha/sign-path.js';
|
||||
@@ -22,13 +22,14 @@ import type { HomeAssistant } from '../../../ha/types.js';
|
||||
import { localize } from '../../../localize/localize.js';
|
||||
import liveJSMPEGStyle from '../../../scss/live-jsmpeg.scss';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../../../types.js';
|
||||
import { convertHTTPAdressToWebsocket, errorToConsole } from '../../../utils/basic.js';
|
||||
import { errorToConsole } from '../../../utils/basic.js';
|
||||
import {
|
||||
createMediaLoadedInfo,
|
||||
dispatchMediaPauseEvent,
|
||||
dispatchMediaPlayEvent,
|
||||
} from '../../../utils/media-info.js';
|
||||
import { Timer } from '../../../utils/timer.js';
|
||||
import { convertToWebSocketURL } from '../../../utils/websocket-url.js';
|
||||
|
||||
import '../../notification/block.js';
|
||||
|
||||
@@ -58,6 +59,10 @@ export class AdvancedCameraCardLiveJSMPEG extends LitElement implements MediaPla
|
||||
@property({ attribute: false })
|
||||
public cardWideConfig?: CardWideConfig;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
@state()
|
||||
private _notification: Notification | null = null;
|
||||
|
||||
@@ -193,10 +198,11 @@ export class AdvancedCameraCardLiveJSMPEG extends LitElement implements MediaPla
|
||||
|
||||
const endpoint = this.camera?.getEndpoints()?.jsmpeg;
|
||||
if (!endpoint) {
|
||||
this._notification = createNotificationFromText(
|
||||
localize('error.live_camera_no_endpoint'),
|
||||
{ context: this.camera?.getConfig() },
|
||||
);
|
||||
this._notification = createMediaNotification({
|
||||
title: localize('error.configuration_error'),
|
||||
detail: localize('error.live_camera_no_endpoint'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
dispatchLiveErrorEvent(this);
|
||||
return;
|
||||
}
|
||||
@@ -211,11 +217,12 @@ export class AdvancedCameraCardLiveJSMPEG extends LitElement implements MediaPla
|
||||
} catch (e) {
|
||||
errorToConsole(e);
|
||||
}
|
||||
const address = response ? convertHTTPAdressToWebsocket(response) : null;
|
||||
const address = response ? convertToWebSocketURL(response) : null;
|
||||
|
||||
if (!address) {
|
||||
this._notification = createNotificationFromText(localize('error.failed_sign'), {
|
||||
context: this.camera?.getConfig(),
|
||||
this._notification = createMediaNotification({
|
||||
title: localize('error.failed_sign'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
dispatchLiveErrorEvent(this);
|
||||
return;
|
||||
@@ -238,10 +245,10 @@ export class AdvancedCameraCardLiveJSMPEG extends LitElement implements MediaPla
|
||||
|
||||
if (!this._jsmpegVideoPlayer || !this._jsmpegCanvasElement) {
|
||||
if (!this._notification) {
|
||||
this._notification = createNotificationFromText(
|
||||
localize('error.jsmpeg_no_player'),
|
||||
{ context: this.camera?.getConfig() },
|
||||
);
|
||||
this._notification = createMediaNotification({
|
||||
title: localize('error.jsmpeg_no_player'),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
dispatchLiveErrorEvent(this);
|
||||
}
|
||||
return;
|
||||
|
||||
@@ -14,24 +14,19 @@ import { dispatchLiveErrorEvent } from '../../../components-lib/live/utils/dispa
|
||||
import { getTechnologyForVideoRTC } from '../../../components-lib/live/utils/get-technology-for-video-rtc.js';
|
||||
import { MediaLoadedInfoSourceController } from '../../../components-lib/media-loaded-info-source-controller.js';
|
||||
import { VideoMediaPlayerController } from '../../../components-lib/media-player/video.js';
|
||||
import { createNotificationFromText } from '../../../components-lib/notification/factory.js';
|
||||
import { createMediaNotification } from '../../../components-lib/notification/media.js';
|
||||
import type { Notification } from '../../../config/schema/actions/types.js';
|
||||
import type { CardWideConfig } from '../../../config/schema/types.js';
|
||||
import type { HomeAssistant } from '../../../ha/types.js';
|
||||
import { localize } from '../../../localize/localize.js';
|
||||
import liveWebRTCCardStyle from '../../../scss/live-webrtc-card.scss';
|
||||
import {
|
||||
AdvancedCameraCardError,
|
||||
type MediaPlayer,
|
||||
type MediaPlayerController,
|
||||
} from '../../../types.js';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../../../types.js';
|
||||
import { mayHaveAudio } from '../../../utils/audio.js';
|
||||
import {
|
||||
hideMediaControlsTemporarily,
|
||||
MEDIA_LOAD_CONTROLS_HIDE_SECONDS,
|
||||
setControlsOnVideo,
|
||||
} from '../../../utils/controls.js';
|
||||
import { getContextFromError } from '../../../utils/error-context.js';
|
||||
import {
|
||||
createMediaLoadedInfo,
|
||||
dispatchMediaPauseEvent,
|
||||
@@ -63,6 +58,10 @@ export class AdvancedCameraCardLiveWebRTCCard extends LitElement implements Medi
|
||||
@property({ attribute: false })
|
||||
public cardWideConfig?: CardWideConfig;
|
||||
|
||||
// The camera's title, shown in error messages to identify the camera.
|
||||
@property({ attribute: false })
|
||||
public cameraTitle?: string;
|
||||
|
||||
@property({ attribute: true, type: Boolean })
|
||||
public controls = false;
|
||||
|
||||
@@ -169,15 +168,11 @@ export class AdvancedCameraCardLiveWebRTCCard extends LitElement implements Medi
|
||||
try {
|
||||
webrtcElement = this._createWebRTC();
|
||||
} catch (e) {
|
||||
const context = getContextFromError(e);
|
||||
this._notification = createNotificationFromText(
|
||||
e instanceof AdvancedCameraCardError
|
||||
? e.message
|
||||
: localize('error.webrtc_card_reported_error') + ': ' + (e as Error).message,
|
||||
{
|
||||
...(context && { context }),
|
||||
},
|
||||
);
|
||||
this._notification = createMediaNotification({
|
||||
title: localize('error.webrtc_card_reported_error'),
|
||||
detail: e instanceof Error ? e.message : String(e),
|
||||
targetTitle: this.cameraTitle,
|
||||
});
|
||||
dispatchLiveErrorEvent(this);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import type { TemplateResult } from 'lit';
|
||||
|
||||
import type { CameraManager } from '../../camera-manager/manager.js';
|
||||
import {
|
||||
createMediaNotification,
|
||||
type MediaNotificationOptions,
|
||||
} from '../../components-lib/notification/media.js';
|
||||
import { localize } from '../../localize/localize.js';
|
||||
import { renderNotificationBlock } from './block.js';
|
||||
|
||||
// Render the standard media notification block: a short titled heading (with
|
||||
// the camera name when there is one), an optional longer detail, a
|
||||
// troubleshooting link, and a retry spinner. See `createMediaNotification`.
|
||||
export function renderMediaNotification(
|
||||
options: MediaNotificationOptions,
|
||||
): TemplateResult {
|
||||
return renderNotificationBlock(createMediaNotification(options));
|
||||
}
|
||||
|
||||
interface NoMediaOptions {
|
||||
cameraID: string | null;
|
||||
inProgress?: boolean;
|
||||
}
|
||||
|
||||
// The viewer/gallery no-media (or awaiting-media) state.
|
||||
export function renderNoMediaNotification(
|
||||
options: NoMediaOptions,
|
||||
cameraManager?: CameraManager,
|
||||
): TemplateResult {
|
||||
const cameraID =
|
||||
options.cameraID ?? cameraManager?.getStore().getDefaultCameraID() ?? null;
|
||||
const targetTitle = cameraID
|
||||
? cameraManager?.getCameraMetadata(cameraID)?.title ?? cameraID
|
||||
: undefined;
|
||||
|
||||
return renderMediaNotification({
|
||||
title: localize(options.inProgress ? 'error.awaiting_media' : 'common.no_media'),
|
||||
icon: 'mdi:multimedia',
|
||||
targetTitle,
|
||||
inProgress: !!options.inProgress,
|
||||
troubleshooting: false,
|
||||
});
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
import type { TemplateResult } from 'lit';
|
||||
|
||||
import type { CameraManager } from '../../camera-manager/manager.js';
|
||||
import { localize } from '../../localize/localize.js';
|
||||
import { renderNotificationBlock } from './block.js';
|
||||
|
||||
interface NoMediaOptions {
|
||||
cameraID: string | null;
|
||||
cameraManager: CameraManager | null;
|
||||
loading?: boolean;
|
||||
}
|
||||
|
||||
export function renderNoMedia(options: NoMediaOptions): TemplateResult {
|
||||
const cameraID =
|
||||
options.cameraID ?? options.cameraManager?.getStore().getDefaultCameraID() ?? null;
|
||||
const cameraTitle = cameraID
|
||||
? options.cameraManager?.getCameraMetadata(cameraID)?.title ?? cameraID
|
||||
: null;
|
||||
|
||||
return renderNotificationBlock({
|
||||
heading: {
|
||||
text: options.loading
|
||||
? localize('error.awaiting_media')
|
||||
: localize('common.no_media'),
|
||||
icon: 'mdi:multimedia',
|
||||
},
|
||||
in_progress: options.loading,
|
||||
...(cameraTitle && {
|
||||
metadata: [{ text: cameraTitle, icon: 'mdi:cctv' }],
|
||||
}),
|
||||
});
|
||||
}
|
||||
@@ -38,7 +38,7 @@ import type { ViewMedia } from '../../view/item.js';
|
||||
import '../carousel';
|
||||
import '../next-prev-control.js';
|
||||
|
||||
import { renderNoMedia } from '../notification/no-media.js';
|
||||
import { renderNoMediaNotification } from '../notification/media.js';
|
||||
|
||||
import '../ptz.js';
|
||||
import './provider.js';
|
||||
@@ -321,13 +321,15 @@ export class AdvancedCameraCardViewerCarousel extends LitElement {
|
||||
protected render(): TemplateResult | void {
|
||||
const mediaCount = this._media?.length ?? 0;
|
||||
if (!this._media || !mediaCount) {
|
||||
return renderNoMedia({
|
||||
cameraID:
|
||||
this.viewFilterCameraID ??
|
||||
this.viewManagerEpoch?.manager.getView()?.camera ??
|
||||
null,
|
||||
cameraManager: this.cameraManager ?? null,
|
||||
});
|
||||
return renderNoMediaNotification(
|
||||
{
|
||||
cameraID:
|
||||
this.viewFilterCameraID ??
|
||||
this.viewManagerEpoch?.manager.getView()?.camera ??
|
||||
null,
|
||||
},
|
||||
this.cameraManager,
|
||||
);
|
||||
}
|
||||
|
||||
if (!this.hass || !this.cameraManager || this._selected === null) {
|
||||
@@ -443,17 +445,17 @@ export class AdvancedCameraCardViewerCarousel extends LitElement {
|
||||
|
||||
const seekTimeInMedia = selectedMedia.includesTime(seek);
|
||||
this.toggleAttribute('unseekable', !seekTimeInMedia);
|
||||
if (!seekTimeInMedia && !mediaPlayerController.isPaused()) {
|
||||
void mediaPlayerController.pause();
|
||||
} else if (seekTimeInMedia && mediaPlayerController.isPaused()) {
|
||||
void mediaPlayerController.play();
|
||||
if (!seekTimeInMedia && !mediaPlayerController.playback?.isPaused()) {
|
||||
void mediaPlayerController.playback?.pause();
|
||||
} else if (seekTimeInMedia && mediaPlayerController.playback?.isPaused()) {
|
||||
void mediaPlayerController.playback?.play();
|
||||
}
|
||||
|
||||
const seekTime =
|
||||
(await this.cameraManager?.getMediaSeekTime(selectedMedia, seek)) ?? null;
|
||||
|
||||
if (seekTime !== null) {
|
||||
void mediaPlayerController.seek(seekTime);
|
||||
void mediaPlayerController.seek?.(seekTime);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ import '../../patches/ha-hls-player.js';
|
||||
|
||||
import viewerStyle from '../../scss/viewer.scss';
|
||||
import { ViewItemClassifier } from '../../view/item-classifier.js';
|
||||
import { renderNoMedia } from '../notification/no-media.js';
|
||||
import { renderNoMediaNotification } from '../notification/media.js';
|
||||
|
||||
import './grid';
|
||||
|
||||
@@ -84,11 +84,13 @@ export class AdvancedCameraCardViewer extends LitElement {
|
||||
// Directly render an error message (instead of dispatching it upwards)
|
||||
// to preserve the mini-timeline if the user pans into an area with no
|
||||
// media.
|
||||
return renderNoMedia({
|
||||
cameraID: this.viewManagerEpoch.manager.getView()?.camera ?? null,
|
||||
cameraManager: this.cameraManager ?? null,
|
||||
loading: !!this.viewManagerEpoch.manager.getView()?.context?.loading?.query,
|
||||
});
|
||||
return renderNoMediaNotification(
|
||||
{
|
||||
cameraID: this.viewManagerEpoch.manager.getView()?.camera ?? null,
|
||||
inProgress: !!this.viewManagerEpoch.manager.getView()?.context?.loading?.query,
|
||||
},
|
||||
this.cameraManager,
|
||||
);
|
||||
}
|
||||
|
||||
return html` <advanced-camera-card-viewer-grid
|
||||
|
||||
@@ -31,6 +31,7 @@ const LIVE_PROVIDERS = [
|
||||
'ha',
|
||||
'jsmpeg',
|
||||
'go2rtc',
|
||||
'go2rtc-experimental',
|
||||
'webrtc-card',
|
||||
] as const;
|
||||
export type LiveProvider = (typeof LIVE_PROVIDERS)[number];
|
||||
@@ -40,12 +41,15 @@ const go2rtcConfigDefault = {
|
||||
metadata_fetch_timeout_seconds: 2,
|
||||
};
|
||||
|
||||
export const GO2RTC_MODES = ['webrtc', 'mse', 'mp4', 'mjpeg'] as const;
|
||||
export type Go2RTCMode = (typeof GO2RTC_MODES)[number];
|
||||
|
||||
const go2rtcConfigSchema = z.object({
|
||||
url: z
|
||||
.string()
|
||||
.transform((input) => input.replace(/\/+$/, ''))
|
||||
.optional(),
|
||||
modes: z.enum(['webrtc', 'mse', 'mp4', 'mjpeg']).array().optional(),
|
||||
modes: z.enum(GO2RTC_MODES).array().optional(),
|
||||
stream: z.string().optional(),
|
||||
metadata_fetch_timeout_seconds: z
|
||||
.number()
|
||||
|
||||
@@ -2595,6 +2595,10 @@ export class AdvancedCameraCardEditor extends LitElement implements LovelaceCard
|
||||
value: 'go2rtc',
|
||||
label: localize('config.cameras.live_providers.go2rtc'),
|
||||
},
|
||||
{
|
||||
value: 'go2rtc-experimental',
|
||||
label: localize('config.cameras.live_providers.go2rtc-experimental'),
|
||||
},
|
||||
{
|
||||
value: 'webrtc-card',
|
||||
label: localize('config.cameras.live_providers.webrtc-card'),
|
||||
|
||||
@@ -382,7 +382,6 @@
|
||||
"awaiting_media": "Warte auf Medium ...",
|
||||
"camera_initialization": "Kamera-Initialisierung fehlgeschlagen",
|
||||
"camera_initialization_reolink": "Konnte Reolink-Kamera nicht initialisieren",
|
||||
"configuration": "Konfiguration überprüfen",
|
||||
"duplicate_folder_id": "Doppelte Ordner-ID für den folgen Ordner, benutzte Parameter 'id' um Ordner eindeutig zu identifizieren",
|
||||
"failed_proxy": "Konnte mich nicht über Home Assistant verbinden",
|
||||
"fetching_diagnostics": "Rufe Diagnosedaten ab",
|
||||
|
||||
@@ -141,6 +141,7 @@
|
||||
"live_providers": {
|
||||
"auto": "Automatic",
|
||||
"go2rtc": "go2rtc",
|
||||
"go2rtc-experimental": "go2rtc (experimental implementation)",
|
||||
"ha": "Home Assistant video stream (i.e. HLS, LL-HLS, WebRTC via HA)",
|
||||
"image": "Image",
|
||||
"jsmpeg": "JSMpeg",
|
||||
@@ -781,7 +782,7 @@
|
||||
"call_no_two_way_audio": "This camera does not support two-way audio.",
|
||||
"camera_initialization": "Camera initialization failed",
|
||||
"camera_initialization_reolink": "Could not initialize Reolink camera",
|
||||
"configuration": "Check configuration",
|
||||
"configuration_error": "Configuration error",
|
||||
"could_not_create_elements": "Could not create picture elements",
|
||||
"diagnostics": "Card diagnostics. Please review for confidential information prior to sharing",
|
||||
"download_failed": "Download failed",
|
||||
@@ -866,7 +867,10 @@
|
||||
"entity_unavailable": "Camera entity unavailable",
|
||||
"not_loading": "Media not loading",
|
||||
"playback_error": "Playback error",
|
||||
"stalled": "Stream stalled"
|
||||
"server_error": "Streaming server error",
|
||||
"stalled": "Stream stalled",
|
||||
"two_way_audio_error": "Two-way audio error",
|
||||
"unsupported": "Stream not supported"
|
||||
},
|
||||
"text": "The media is not currently available. This can happen for several reasons, for example a camera becoming unavailable, a stream stalling or failing, or media that has not finished loading. The card keeps retrying automatically. For live views, a still image may be shown while a stream is loading (if configured, and by default)"
|
||||
},
|
||||
|
||||
@@ -654,7 +654,6 @@
|
||||
"awaiting_media": "Oczekiwanie na załadowanie mediów",
|
||||
"camera_initialization": "Inicjalizacja kamery nie powiodła się",
|
||||
"camera_initialization_reolink": "Nie można zainicjować kamery Reolink",
|
||||
"configuration": "Sprawdź konfigurację",
|
||||
"could_not_create_elements": "Nie można wyrenderować elementów obrazu",
|
||||
"diagnostics": "Diagnostyka karty. Przejrzyj pod kątem poufnych informacji przed udostępnieniem",
|
||||
"download_no_media": "Brak mediów do pobrania",
|
||||
|
||||
@@ -16,7 +16,8 @@ import { query } from 'lit/decorators/query.js';
|
||||
import { dispatchLiveErrorEvent } from '../components-lib/live/utils/dispatch-live-error.js';
|
||||
import { MediaLoadedInfoSourceController } from '../components-lib/media-loaded-info-source-controller.js';
|
||||
import { VideoMediaPlayerController } from '../components-lib/media-player/video.js';
|
||||
import { renderNotificationBlockFromText } from '../components/notification/block.js';
|
||||
import { renderMediaNotification } from '../components/notification/media.js';
|
||||
import { localize } from '../localize/localize.js';
|
||||
import liveHAComponentsStyle from '../scss/live-ha-components.scss';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../types.js';
|
||||
import { mayHaveAudio } from '../utils/audio.js';
|
||||
@@ -72,8 +73,10 @@ void customElements.whenDefined('ha-hls-player').then(() => {
|
||||
if (this._error) {
|
||||
if (this._errorIsFatal) {
|
||||
dispatchLiveErrorEvent(this);
|
||||
return renderNotificationBlockFromText(this._error, {
|
||||
metadata: [{ text: this.entityid, icon: 'mdi:cctv' }],
|
||||
return renderMediaNotification({
|
||||
title: localize('issues.media_unavailable.reasons.playback_error'),
|
||||
detail: this._error,
|
||||
targetTitle: this.entityid,
|
||||
});
|
||||
} else {
|
||||
errorToConsole(this._error, console.error);
|
||||
|
||||
@@ -16,7 +16,8 @@ import { ifDefined } from 'lit/directives/if-defined.js';
|
||||
import { dispatchLiveErrorEvent } from '../components-lib/live/utils/dispatch-live-error.js';
|
||||
import { MediaLoadedInfoSourceController } from '../components-lib/media-loaded-info-source-controller.js';
|
||||
import { VideoMediaPlayerController } from '../components-lib/media-player/video.js';
|
||||
import { renderNotificationBlockFromText } from '../components/notification/block.js';
|
||||
import { renderMediaNotification } from '../components/notification/media.js';
|
||||
import { localize } from '../localize/localize.js';
|
||||
import liveHAComponentsStyle from '../scss/live-ha-components.scss';
|
||||
import type { MediaPlayer, MediaPlayerController } from '../types.js';
|
||||
import {
|
||||
@@ -112,8 +113,10 @@ void customElements.whenDefined('ha-web-rtc-player').then(() => {
|
||||
protected render(): TemplateResult | void {
|
||||
if (this._error) {
|
||||
dispatchLiveErrorEvent(this);
|
||||
return renderNotificationBlockFromText(this._error, {
|
||||
metadata: [{ text: this.entityid, icon: 'mdi:cctv' }],
|
||||
return renderMediaNotification({
|
||||
title: localize('issues.media_unavailable.reasons.playback_error'),
|
||||
detail: this._error,
|
||||
targetTitle: this.entityid,
|
||||
});
|
||||
}
|
||||
return html`
|
||||
@@ -146,7 +149,7 @@ void customElements.whenDefined('ha-web-rtc-player').then(() => {
|
||||
mediaPlayerController: this._mediaPlayerController,
|
||||
capabilities: {
|
||||
supportsPause: true,
|
||||
hasAudio: hasAudio(this._videoEl, this._peerConnection),
|
||||
hasAudio: hasAudio(this._videoEl, { pc: this._peerConnection }),
|
||||
},
|
||||
technology: ['webrtc'],
|
||||
});
|
||||
@@ -164,7 +167,7 @@ void customElements.whenDefined('ha-web-rtc-player').then(() => {
|
||||
mediaPlayerController: this._mediaPlayerController,
|
||||
capabilities: {
|
||||
supportsPause: true,
|
||||
hasAudio: hasAudio(this._videoEl, this._peerConnection),
|
||||
hasAudio: hasAudio(this._videoEl, { pc: this._peerConnection }),
|
||||
},
|
||||
technology: ['webrtc'],
|
||||
});
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
@use 'media-layout.scss';
|
||||
|
||||
:host {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
display: block;
|
||||
}
|
||||
|
||||
video,
|
||||
img {
|
||||
@include media-layout.media-layout();
|
||||
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
display: block;
|
||||
|
||||
// The inactive surface is `?hidden` in the template (only the committed one
|
||||
// is shown).
|
||||
&[hidden] {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
// Hide the native seek bar (and its time readouts) on the live video: the
|
||||
// stream is live, so scrubbing back only fights the live-edge chase and snaps
|
||||
// to live. Play/pause/volume/fullscreen remain. WebKit/Blink only (Chrome,
|
||||
// Safari, Edge, Android); Firefox does not expose these controls for styling,
|
||||
// so its scrubber stays.
|
||||
video::-webkit-media-controls-timeline,
|
||||
video::-webkit-media-controls-current-time-display,
|
||||
video::-webkit-media-controls-time-remaining-display {
|
||||
display: none;
|
||||
}
|
||||
@@ -4,10 +4,26 @@
|
||||
position: relative;
|
||||
}
|
||||
|
||||
// Until loaded media or a snapshot sizes the frame (the `sized` attribute),
|
||||
// nothing gives it a size, so reserve a default ratio to stop it collapsing.
|
||||
// Real media is never letterboxed into this ratio: the attribute is set the
|
||||
// moment it sizes the frame.
|
||||
:host(:not([sized])) {
|
||||
aspect-ratio: 16 / 9;
|
||||
}
|
||||
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
// Fills the frame with a loading or error status when neither media nor a
|
||||
// snapshot is present (a provider renders only its own media, so the frame
|
||||
// would otherwise be blank).
|
||||
.fill {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
}
|
||||
|
||||
advanced-camera-card-icon {
|
||||
position: absolute;
|
||||
top: 10px;
|
||||
|
||||
+19
-3
@@ -89,20 +89,36 @@ export type UnsubscribeCallback = () => void;
|
||||
// Reports each live/stalled transition of a player's media stream.
|
||||
export type LivenessCallback = (isLive: boolean) => void;
|
||||
|
||||
export interface MediaPlayerController {
|
||||
// Control over a pausable playback loop. Optional on the player: present only
|
||||
// when the player owns a stream it can start and stop (e.g. a real video, or an
|
||||
// image refreshed on a timer). A static image, or one fed frames as they arrive
|
||||
// where the client has no control (e.g. MP4, MJPEG), has no pausable loop and
|
||||
// omits it.
|
||||
export interface PlaybackControl {
|
||||
play(): Promise<void>;
|
||||
pause(): Promise<void>;
|
||||
isPaused(): boolean;
|
||||
}
|
||||
|
||||
export interface MediaPlayerController {
|
||||
mute(): Promise<void>;
|
||||
unmute(): Promise<void>;
|
||||
isMuted(): boolean;
|
||||
seek(seconds: number): Promise<void>;
|
||||
getScreenshotURL(): Promise<string | null>;
|
||||
|
||||
// If no value for controls if specified, the player should use the default.
|
||||
setControls(controls?: boolean): Promise<void>;
|
||||
isPaused(): boolean;
|
||||
getFullscreenElement(): FullscreenElement | null;
|
||||
getPIPElement(): PIPElement | null;
|
||||
|
||||
// Jump to a time position, if the media has a seekable timeline. Optional:
|
||||
// implemented only by players over seekable media.
|
||||
seek?(seconds: number): Promise<void>;
|
||||
|
||||
// Start/pause the media, if it is pausable. Optional: implemented only by
|
||||
// players that own a pausable playback loop.
|
||||
playback?: PlaybackControl;
|
||||
|
||||
// Observe whether the player is actively delivering media, so a silent freeze
|
||||
// (frames stop advancing while playing) can be detected. Optional:
|
||||
// implemented only by players that can observe their own frame progress. The
|
||||
|
||||
+11
-6
@@ -25,15 +25,17 @@ export const mayHaveAudio = (video: HTMLVideoElement & AudioProperties): boolean
|
||||
|
||||
/**
|
||||
* Determine if audio is available for a go2rtc stream.
|
||||
* @param pc The RTCPeerConnection (for WebRTC streams).
|
||||
* @param mseCodecs The negotiated MSE codecs string (for MSE streams).
|
||||
* @param video The video element (fallback for browser-based detection).
|
||||
* @param options.pc The RTCPeerConnection (for WebRTC streams).
|
||||
* @param options.mseCodecs The negotiated MSE codecs string (for MSE streams).
|
||||
* @returns True if audio is available.
|
||||
*/
|
||||
export const hasAudio = (
|
||||
video: HTMLVideoElement & AudioProperties,
|
||||
pc?: RTCPeerConnection | null,
|
||||
mseCodecs?: string,
|
||||
options?: {
|
||||
pc?: RTCPeerConnection | null;
|
||||
mseCodecs?: string | null;
|
||||
},
|
||||
): boolean => {
|
||||
// For WebRTC: Check if there's an audio receiver with an active track. We
|
||||
// check that the track is not muted because muted means no media data is
|
||||
@@ -43,15 +45,17 @@ export const hasAudio = (
|
||||
// (e.g. WebRTC failed, fell back to MSE) will have receivers with muted
|
||||
// tracks that don't reflect actual media availability.
|
||||
// See: https://github.com/dermotduffy/advanced-camera-card/issues/2417
|
||||
if (pc && pc.connectionState === 'connected') {
|
||||
const receivers = pc.getReceivers();
|
||||
if (options?.pc && options.pc.connectionState === 'connected') {
|
||||
const receivers = options.pc.getReceivers();
|
||||
if (receivers.length > 0) {
|
||||
return receivers.some(
|
||||
(receiver) => receiver.track?.kind === 'audio' && !receiver.track?.muted,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// For MSE: Check negotiated codecs for audio codecs
|
||||
const mseCodecs = options?.mseCodecs;
|
||||
if (mseCodecs) {
|
||||
return (
|
||||
mseCodecs.includes('mp4a') ||
|
||||
@@ -59,6 +63,7 @@ export const hasAudio = (
|
||||
mseCodecs.includes('flac')
|
||||
);
|
||||
}
|
||||
|
||||
// Fallback to browser-based detection (unreliable in Chrome)
|
||||
return mayHaveAudio(video);
|
||||
};
|
||||
|
||||
@@ -352,10 +352,6 @@ export const ignoreFunctionIdentity = (a: unknown, b: unknown): boolean | undefi
|
||||
? typeof a === 'function' && typeof b === 'function'
|
||||
: undefined;
|
||||
|
||||
export const convertHTTPAdressToWebsocket = (url: string): string => {
|
||||
return url.replace(/^http/i, 'ws');
|
||||
};
|
||||
|
||||
export const forceReflow = (element: HTMLElement): void => {
|
||||
// Force reflow by measuring the height.
|
||||
void element.offsetHeight;
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
// A monotonic counter for guarding against stale asynchronous results. Snapshot
|
||||
// the current generation before an `await`; afterwards, `isCurrent()` reports
|
||||
// whether anything has invalidated that snapshot since (a reset, a teardown, or
|
||||
// a newer operation), so a superseded result can be dropped.
|
||||
export class Generation {
|
||||
private _value = 0;
|
||||
|
||||
// Invalidate all outstanding snapshots (e.g. on reset or teardown).
|
||||
public invalidate(): void {
|
||||
this._value++;
|
||||
}
|
||||
|
||||
// Start a new latest-wins operation -- invalidating any outstanding snapshot --
|
||||
// and return its token.
|
||||
public next(): number {
|
||||
return ++this._value;
|
||||
}
|
||||
|
||||
// Snapshot the current generation to compare after an await.
|
||||
public current(): number {
|
||||
return this._value;
|
||||
}
|
||||
|
||||
// Whether the token is still current (nothing has invalidated it since it was
|
||||
// taken).
|
||||
public isCurrent(token: number): boolean {
|
||||
return token === this._value;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
/**
|
||||
* Runs an async operation for the latest submitted value, one run at a time.
|
||||
*
|
||||
* While a run is in flight, only the most recently submitted value is kept;
|
||||
* values submitted in between are dropped. The kept value runs when the current
|
||||
* run finishes, so the operation always converges on the newest input without
|
||||
* running more than once at a time.
|
||||
*
|
||||
* Each `submit` returns a promise that resolves after the next run completes, so
|
||||
* a caller can wait for a run to have happened without tracking which value ran
|
||||
* (under load its own value may have been superseded by a newer one).
|
||||
*
|
||||
* The operation owns its own errors: a run that rejects still counts as done
|
||||
* (its waiters resolve and draining continues), so a single bad value cannot
|
||||
* strand later ones.
|
||||
*/
|
||||
export class LatestValueRunner<T> {
|
||||
private _run: (value: T) => Promise<void>;
|
||||
|
||||
private _running = false;
|
||||
|
||||
private _pending: { value: T } | null = null;
|
||||
private _waiters: Array<() => void> = [];
|
||||
|
||||
constructor(run: (value: T) => Promise<void>) {
|
||||
this._run = run;
|
||||
}
|
||||
|
||||
// Submit a value to run. Returns a promise that resolves once the next run
|
||||
// has completed.
|
||||
public submit(value: T): Promise<void> {
|
||||
this._pending = { value };
|
||||
const ran = new Promise<void>((resolve) => this._waiters.push(resolve));
|
||||
if (!this._running) {
|
||||
this._running = true;
|
||||
// The drain loop cannot reject (the operation's errors are caught within
|
||||
// it); the catch only satisfies the no-floating-promises rule.
|
||||
/* istanbul ignore next -- @preserve */
|
||||
this._drain().catch(() => {});
|
||||
}
|
||||
return ran;
|
||||
}
|
||||
|
||||
// Drop any value still waiting to run.
|
||||
public clear(): void {
|
||||
this._pending = null;
|
||||
}
|
||||
|
||||
private async _drain(): Promise<void> {
|
||||
try {
|
||||
while (this._pending) {
|
||||
const { value } = this._pending;
|
||||
this._pending = null;
|
||||
|
||||
try {
|
||||
await this._run(value);
|
||||
} catch {
|
||||
// The operation owns its errors; the runner only guarantees progress.
|
||||
}
|
||||
|
||||
// Release everyone waiting on a run; values submitted during the run
|
||||
// were queued after this snapshot and wait for the next one.
|
||||
const waiters = this._waiters;
|
||||
this._waiters = [];
|
||||
waiters.forEach((resolve) => resolve());
|
||||
}
|
||||
} finally {
|
||||
this._running = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -22,6 +22,13 @@ export const getResolvedLiveProvider = (
|
||||
return config?.live_provider ?? 'image';
|
||||
};
|
||||
|
||||
// Live providers that stream from a go2rtc server and share the `go2rtc`
|
||||
// config block and endpoints.
|
||||
const GO2RTC_LIVE_PROVIDERS: readonly LiveProvider[] = ['go2rtc', 'go2rtc-experimental'];
|
||||
|
||||
export const isGo2RTCLiveProvider = (provider: LiveProvider): boolean =>
|
||||
GO2RTC_LIVE_PROVIDERS.includes(provider);
|
||||
|
||||
export const liveProviderSupports2WayAudio = async (
|
||||
hass: HomeAssistant,
|
||||
config: CameraConfig,
|
||||
@@ -29,7 +36,7 @@ export const liveProviderSupports2WayAudio = async (
|
||||
go2rtcMetadataEndpoint?: Endpoint | null,
|
||||
proxyConfig?: EnabledProxyConfig,
|
||||
): Promise<boolean> => {
|
||||
if (getResolvedLiveProvider(config) !== 'go2rtc') {
|
||||
if (!isGo2RTCLiveProvider(getResolvedLiveProvider(config))) {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
// Convert an HTTP(S) or origin-relative URL to its WS(S) equivalent.
|
||||
export const convertToWebSocketURL = (url: string, origin?: string): string => {
|
||||
if (/^http/i.test(url)) {
|
||||
return 'ws' + url.substring(4);
|
||||
}
|
||||
if (url.startsWith('/')) {
|
||||
return 'ws' + (origin ?? location.origin).substring(4) + url;
|
||||
}
|
||||
return url;
|
||||
};
|
||||
@@ -7,6 +7,7 @@ import type { CameraProxyConfig } from '../../src/camera-manager/types.js';
|
||||
import type { EventWatcherSubscriptionInterface } from '../../src/card-controller/hass/event-watcher.js';
|
||||
import type { StateWatcherSubscriptionInterface } from '../../src/card-controller/hass/state-watcher.js';
|
||||
import { liveProviderSupports2WayAudio } from '../../src/utils/live-provider.js';
|
||||
import type * as LiveProviderUtils from '../../src/utils/live-provider.js';
|
||||
import { EntityRegistryManagerMock } from '../ha/registry/entity/mock.js';
|
||||
import {
|
||||
callEventWatcherCallback,
|
||||
@@ -21,7 +22,12 @@ import {
|
||||
createStateEntity,
|
||||
} from '../test-utils.js';
|
||||
|
||||
vi.mock('../../src/utils/live-provider.js');
|
||||
// Partially mock to keep the real pure helpers (e.g. `isGo2RTCLiveProvider`
|
||||
// used by `getProxyConfig`) while mocking the async metadata fetch.
|
||||
vi.mock('../../src/utils/live-provider.js', async (importOriginal) => ({
|
||||
...(await importOriginal<typeof LiveProviderUtils>()),
|
||||
liveProviderSupports2WayAudio: vi.fn(),
|
||||
}));
|
||||
|
||||
describe('Camera', () => {
|
||||
it('should get config', async () => {
|
||||
@@ -1068,6 +1074,31 @@ describe('Camera', () => {
|
||||
ssl_ciphers: 'default' as const,
|
||||
},
|
||||
],
|
||||
[
|
||||
'when go2rtc-experimental has no url',
|
||||
{ live_provider: 'go2rtc-experimental' },
|
||||
{
|
||||
dynamic: true,
|
||||
live: false,
|
||||
media: false,
|
||||
ssl_verification: true,
|
||||
ssl_ciphers: 'default' as const,
|
||||
},
|
||||
],
|
||||
[
|
||||
'when go2rtc-experimental has a url',
|
||||
{
|
||||
live_provider: 'go2rtc-experimental',
|
||||
go2rtc: { url: 'http://localhost:1984' },
|
||||
},
|
||||
{
|
||||
dynamic: true,
|
||||
live: true,
|
||||
media: false,
|
||||
ssl_verification: true,
|
||||
ssl_ciphers: 'default' as const,
|
||||
},
|
||||
],
|
||||
])(
|
||||
'%s',
|
||||
(_name: string, cameraConfig: unknown, expectedResult: CameraProxyConfig) => {
|
||||
|
||||
@@ -2,12 +2,14 @@ import { expect, it, vi } from 'vitest';
|
||||
import { mock } from 'vitest-mock-extended';
|
||||
|
||||
import { PauseAction } from '../../../../src/card-controller/actions/actions/pause';
|
||||
import type { MediaPlayerController } from '../../../../src/types';
|
||||
import type { MediaPlayerController, PlaybackControl } from '../../../../src/types';
|
||||
import { createCardAPI, createMediaLoadedInfo } from '../../../test-utils';
|
||||
|
||||
it('should handle pause action', async () => {
|
||||
const api = createCardAPI();
|
||||
const mediaPlayerController = mock<MediaPlayerController>();
|
||||
const mediaPlayerController = mock<MediaPlayerController>({
|
||||
playback: mock<PlaybackControl>(),
|
||||
});
|
||||
vi.mocked(api.getMediaLoadedInfoManager().get).mockReturnValue(
|
||||
createMediaLoadedInfo({
|
||||
mediaPlayerController,
|
||||
@@ -23,5 +25,5 @@ it('should handle pause action', async () => {
|
||||
|
||||
await action.execute(api);
|
||||
|
||||
expect(mediaPlayerController.pause).toBeCalled();
|
||||
expect(mediaPlayerController.playback?.pause).toBeCalled();
|
||||
});
|
||||
|
||||
@@ -2,12 +2,14 @@ import { expect, it, vi } from 'vitest';
|
||||
import { mock } from 'vitest-mock-extended';
|
||||
|
||||
import { PlayAction } from '../../../../src/card-controller/actions/actions/play';
|
||||
import type { MediaPlayerController } from '../../../../src/types';
|
||||
import type { MediaPlayerController, PlaybackControl } from '../../../../src/types';
|
||||
import { createCardAPI, createMediaLoadedInfo } from '../../../test-utils';
|
||||
|
||||
it('should handle play action', async () => {
|
||||
const api = createCardAPI();
|
||||
const mediaPlayerController = mock<MediaPlayerController>();
|
||||
const mediaPlayerController = mock<MediaPlayerController>({
|
||||
playback: mock<PlaybackControl>(),
|
||||
});
|
||||
vi.mocked(api.getMediaLoadedInfoManager().get).mockReturnValue(
|
||||
createMediaLoadedInfo({
|
||||
mediaPlayerController,
|
||||
@@ -23,5 +25,5 @@ it('should handle play action', async () => {
|
||||
|
||||
await action.execute(api);
|
||||
|
||||
expect(mediaPlayerController.play).toBeCalled();
|
||||
expect(mediaPlayerController.playback?.play).toBeCalled();
|
||||
});
|
||||
|
||||
@@ -35,6 +35,22 @@ describe('ProviderErrorDetector', () => {
|
||||
expect(onChange).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should adopt the specific reason carried on the event', () => {
|
||||
const host = createHostInDocument();
|
||||
const detector = new ProviderErrorDetector(host, vi.fn());
|
||||
detector.subscribe();
|
||||
|
||||
host.dispatchEvent(
|
||||
new CustomEvent(LIVE_ERROR_EVENT, { bubbles: true, detail: 'unsupported' }),
|
||||
);
|
||||
|
||||
expect(detector.getVerdict()).toEqual({
|
||||
state: 'not_live',
|
||||
authority: 'hard',
|
||||
reason: 'unsupported',
|
||||
});
|
||||
});
|
||||
|
||||
it('should notify only on the transition to not live', () => {
|
||||
const host = createHostInDocument();
|
||||
const onChange = vi.fn();
|
||||
|
||||
@@ -89,7 +89,7 @@ describe('StreamLivenessController', () => {
|
||||
const { controller } = setup();
|
||||
|
||||
expect(controller.isLive()).toBe(true);
|
||||
expect(controller.getPlaceholder()).toBeNull();
|
||||
expect(controller.getFailure()).toBeNull();
|
||||
});
|
||||
|
||||
it('should aggregate a detector losing liveness only after host connect', () => {
|
||||
@@ -123,9 +123,13 @@ describe('StreamLivenessController', () => {
|
||||
|
||||
failViaProviderError();
|
||||
|
||||
// Provider-error is not-live but does not want a placeholder.
|
||||
// Provider-error is not-live but does not want a placeholder; its cause is
|
||||
// still available for a wrapper that fills the frame itself.
|
||||
expect(controller.isLive()).toBe(false);
|
||||
expect(controller.getPlaceholder()).toBeNull();
|
||||
expect(controller.getFailure()).toEqual({
|
||||
reason: 'playback_error',
|
||||
renderPlaceholder: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should not fire the issue without a target', () => {
|
||||
@@ -154,7 +158,10 @@ describe('StreamLivenessController', () => {
|
||||
fireMediaPlayerLiveness(false);
|
||||
|
||||
expect(controller.isLive()).toBe(false);
|
||||
expect(controller.getPlaceholder()).toEqual({ reason: 'stalled' });
|
||||
expect(controller.getFailure()).toEqual({
|
||||
reason: 'stalled',
|
||||
renderPlaceholder: true,
|
||||
});
|
||||
expect(issueTriggers).toEqual([
|
||||
{ key: 'media_unavailable', targetID: 'camera.office', reason: 'stalled' },
|
||||
]);
|
||||
@@ -232,7 +239,10 @@ describe('StreamLivenessController', () => {
|
||||
controller.hostConnected();
|
||||
|
||||
expect(controller.isLive()).toBe(false);
|
||||
expect(controller.getPlaceholder()).toEqual({ reason: 'entity_unavailable' });
|
||||
expect(controller.getFailure()).toEqual({
|
||||
reason: 'entity_unavailable',
|
||||
renderPlaceholder: true,
|
||||
});
|
||||
expect(issueTriggers).toEqual([
|
||||
{
|
||||
key: 'media_unavailable',
|
||||
@@ -298,7 +308,7 @@ describe('StreamLivenessController', () => {
|
||||
|
||||
// Direct frame evidence outranks the entity proxy: no teardown, no issue.
|
||||
expect(controller.isLive()).toBe(true);
|
||||
expect(controller.getPlaceholder()).toBeNull();
|
||||
expect(controller.getFailure()).toBeNull();
|
||||
expect(issueTriggers).toEqual([]);
|
||||
|
||||
vi.useRealTimers();
|
||||
@@ -343,6 +353,9 @@ describe('StreamLivenessController', () => {
|
||||
// The always_error opt-in is authoritative: an unavailable entity overrides
|
||||
// even confirmed frames.
|
||||
expect(controller.isLive()).toBe(false);
|
||||
expect(controller.getPlaceholder()).toEqual({ reason: 'entity_unavailable' });
|
||||
expect(controller.getFailure()).toEqual({
|
||||
reason: 'entity_unavailable',
|
||||
renderPlaceholder: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
+226
@@ -0,0 +1,226 @@
|
||||
import { afterEach, assert, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { createBrowserMediaSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/adapters/media-source';
|
||||
|
||||
class FakeMediaSource extends EventTarget {
|
||||
public static instances: FakeMediaSource[] = [];
|
||||
|
||||
public addSourceBuffer = vi.fn();
|
||||
public setLiveSeekableRange = vi.fn();
|
||||
public static isTypeSupported = vi.fn<[string], boolean>(() => true);
|
||||
|
||||
constructor() {
|
||||
super();
|
||||
FakeMediaSource.instances.push(this);
|
||||
}
|
||||
}
|
||||
|
||||
class FakeManagedMediaSource extends EventTarget {
|
||||
public addSourceBuffer = vi.fn();
|
||||
public setLiveSeekableRange = vi.fn();
|
||||
public readyState: 'closed' | 'open' | 'ended' = 'closed';
|
||||
public static isTypeSupported = vi.fn<[string], boolean>(() => true);
|
||||
}
|
||||
|
||||
const createObjectURL = vi.fn(() => 'blob:fake-url');
|
||||
const revokeObjectURL = vi.fn();
|
||||
|
||||
const stubManagedMediaSource = (): void => {
|
||||
vi.stubGlobal('ManagedMediaSource', FakeManagedMediaSource);
|
||||
};
|
||||
|
||||
const stubClassicMediaSource = (): void => {
|
||||
vi.stubGlobal('MediaSource', FakeMediaSource);
|
||||
vi.stubGlobal('URL', { createObjectURL, revokeObjectURL });
|
||||
};
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('media-source', () => {
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
vi.clearAllMocks();
|
||||
FakeMediaSource.instances = [];
|
||||
});
|
||||
|
||||
it('should return null without any MediaSource support', () => {
|
||||
expect(createBrowserMediaSource()).toBeNull();
|
||||
});
|
||||
|
||||
describe('with ManagedMediaSource support', () => {
|
||||
it('should prefer ManagedMediaSource over MediaSource', () => {
|
||||
stubManagedMediaSource();
|
||||
stubClassicMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
|
||||
expect(video.srcObject).toBeInstanceOf(FakeManagedMediaSource);
|
||||
});
|
||||
|
||||
it('should attach via srcObject with remote playback disabled', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
|
||||
expect(video.disableRemotePlayback).toBe(true);
|
||||
expect(video.srcObject).toBeInstanceOf(FakeManagedMediaSource);
|
||||
});
|
||||
|
||||
it('should detach by clearing srcObject', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
instance?.detach(video);
|
||||
|
||||
expect(video.srcObject).toBeNull();
|
||||
});
|
||||
|
||||
it('should delegate isTypeSupported to ManagedMediaSource', () => {
|
||||
stubManagedMediaSource();
|
||||
FakeManagedMediaSource.isTypeSupported.mockReturnValue(false);
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
|
||||
expect(instance?.isTypeSupported('video/mp4')).toBe(false);
|
||||
expect(FakeManagedMediaSource.isTypeSupported).toBeCalledWith('video/mp4');
|
||||
});
|
||||
});
|
||||
|
||||
describe('with classic MediaSource support', () => {
|
||||
it('should attach via an object URL', () => {
|
||||
stubClassicMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
|
||||
expect(createObjectURL).toBeCalledTimes(1);
|
||||
expect(video.src).toContain('blob:fake-url');
|
||||
expect(video.srcObject).toBeNull();
|
||||
});
|
||||
|
||||
it('should revoke the object URL once the source opens', () => {
|
||||
stubClassicMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
|
||||
expect(revokeObjectURL).not.toBeCalled();
|
||||
|
||||
FakeMediaSource.instances[0].dispatchEvent(new Event('sourceopen'));
|
||||
|
||||
expect(revokeObjectURL).toBeCalledWith('blob:fake-url');
|
||||
|
||||
instance?.detach(video);
|
||||
|
||||
expect(revokeObjectURL).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should delegate isTypeSupported to MediaSource', () => {
|
||||
stubClassicMediaSource();
|
||||
FakeMediaSource.isTypeSupported.mockReturnValue(false);
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
|
||||
expect(instance?.isTypeSupported('video/mp4')).toBe(false);
|
||||
expect(FakeMediaSource.isTypeSupported).toBeCalledWith('video/mp4');
|
||||
});
|
||||
|
||||
it('should detach by clearing src and revoking the object URL', () => {
|
||||
stubClassicMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
instance?.detach(video);
|
||||
|
||||
expect(revokeObjectURL).toBeCalledWith('blob:fake-url');
|
||||
expect(video.getAttribute('src')).toBe('');
|
||||
});
|
||||
|
||||
it('should not revoke the object URL twice', () => {
|
||||
stubClassicMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
instance?.detach(video);
|
||||
instance?.detach(video);
|
||||
|
||||
expect(revokeObjectURL).toBeCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('shared behavior', () => {
|
||||
it('should subscribe and unsubscribe from sourceopen', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
const callback = vi.fn();
|
||||
const unsubscribe = instance?.subscribeToSourceOpen(callback);
|
||||
|
||||
const mediaSource = video.srcObject;
|
||||
assert(mediaSource instanceof FakeManagedMediaSource);
|
||||
mediaSource.dispatchEvent(new Event('sourceopen'));
|
||||
expect(callback).toBeCalledTimes(1);
|
||||
|
||||
unsubscribe?.();
|
||||
|
||||
mediaSource.dispatchEvent(new Event('sourceopen'));
|
||||
expect(callback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should delegate addSourceBuffer', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
instance?.addSourceBuffer('video/mp4; codecs="avc1.640029"');
|
||||
|
||||
const mediaSource = video.srcObject;
|
||||
assert(mediaSource instanceof FakeManagedMediaSource);
|
||||
expect(mediaSource.addSourceBuffer).toBeCalledWith(
|
||||
'video/mp4; codecs="avc1.640029"',
|
||||
);
|
||||
});
|
||||
|
||||
it('should delegate setLiveSeekableRange', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
instance?.setLiveSeekableRange(10, 20);
|
||||
|
||||
const mediaSource = video.srcObject;
|
||||
assert(mediaSource instanceof FakeManagedMediaSource);
|
||||
expect(mediaSource.setLiveSeekableRange).toBeCalledWith(10, 20);
|
||||
});
|
||||
|
||||
it('should report open only while the media source readyState is open', () => {
|
||||
stubManagedMediaSource();
|
||||
|
||||
const instance = createBrowserMediaSource();
|
||||
const video = document.createElement('video');
|
||||
instance?.attach(video);
|
||||
|
||||
const mediaSource = video.srcObject;
|
||||
assert(mediaSource instanceof FakeManagedMediaSource);
|
||||
|
||||
mediaSource.readyState = 'open';
|
||||
expect(instance?.isOpen()).toBe(true);
|
||||
|
||||
mediaSource.readyState = 'closed';
|
||||
expect(instance?.isOpen()).toBe(false);
|
||||
});
|
||||
});
|
||||
});
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import {
|
||||
createBrowserPeerConnection,
|
||||
GO2RTC_PEER_CONNECTION_CONFIG,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/adapters/peer-connection';
|
||||
|
||||
describe('peer-connection', () => {
|
||||
it('should configure two STUN servers with max-bundle', () => {
|
||||
expect(GO2RTC_PEER_CONNECTION_CONFIG).toEqual({
|
||||
bundlePolicy: 'max-bundle',
|
||||
iceServers: [
|
||||
{
|
||||
urls: ['stun:stun.l.google.com:19302', 'stun:stun.cloudflare.com:3478'],
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('should construct a real peer connection', () => {
|
||||
const RTCPeerConnectionMock = vi.fn();
|
||||
vi.stubGlobal('RTCPeerConnection', RTCPeerConnectionMock);
|
||||
|
||||
createBrowserPeerConnection(GO2RTC_PEER_CONNECTION_CONFIG);
|
||||
|
||||
expect(RTCPeerConnectionMock).toHaveBeenCalledWith(GO2RTC_PEER_CONNECTION_CONFIG);
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
});
|
||||
+232
@@ -0,0 +1,232 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { ImageSurfaceController } from '../../../../../src/components-lib/live/providers/go2rtc-experimental/image-surface-controller';
|
||||
import { createLitElement, flushPromises } from '../../../../test-utils';
|
||||
|
||||
const createFrame = (): Blob => new Blob(['frame'], { type: 'image/jpeg' });
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('ImageSurfaceController', () => {
|
||||
let nextId: number;
|
||||
|
||||
beforeEach(() => {
|
||||
nextId = 0;
|
||||
URL.createObjectURL = vi.fn(() => `blob:mock-${nextId++}`);
|
||||
URL.revokeObjectURL = vi.fn();
|
||||
document.body.replaceChildren();
|
||||
});
|
||||
|
||||
const createdURLs = (): string[] =>
|
||||
vi.mocked(URL.createObjectURL).mock.results.map((result) => result.value);
|
||||
|
||||
// A connected visible <img> plus a detached loader whose decode() is stubbed
|
||||
// (jsdom has none). The loader is injected so a test can hold a decode in
|
||||
// flight to exercise coalescing.
|
||||
const createSurfaceController = (options?: {
|
||||
connected?: boolean;
|
||||
decode?: () => Promise<void>;
|
||||
}) => {
|
||||
const image = document.createElement('img');
|
||||
if (options?.connected !== false) {
|
||||
document.body.appendChild(image);
|
||||
}
|
||||
const decoder = document.createElement('img');
|
||||
decoder.decode = vi.fn(options?.decode ?? (() => Promise.resolve()));
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => image, {
|
||||
createImage: () => decoder,
|
||||
});
|
||||
return { image, decoder, controller };
|
||||
};
|
||||
|
||||
describe('controller', () => {
|
||||
it('should expose an image media player controller', () => {
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => null);
|
||||
|
||||
expect(controller.getMediaPlayer().getPIPElement()).toBeNull();
|
||||
});
|
||||
|
||||
it('should have liveness when given liveness options', () => {
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => null, {
|
||||
livenessOptions: { isFrameExpected: () => true, stallWindowSeconds: 10 },
|
||||
});
|
||||
|
||||
expect(controller.getMediaPlayer().subscribeLiveness).toBeDefined();
|
||||
});
|
||||
|
||||
it('should have no liveness without liveness options', () => {
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => null);
|
||||
|
||||
expect(controller.getMediaPlayer().subscribeLiveness).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should expose the image element', () => {
|
||||
const { image, controller } = createSurfaceController();
|
||||
|
||||
expect(controller.getElement()).toBe(image);
|
||||
});
|
||||
});
|
||||
|
||||
describe('showFrame', () => {
|
||||
it('should decode a frame off-DOM and show it via an object URL', async () => {
|
||||
const { image, decoder, controller } = createSurfaceController();
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
expect(decoder.decode).toBeCalledTimes(1);
|
||||
expect(URL.createObjectURL).toBeCalledTimes(1);
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[0]);
|
||||
expect(URL.revokeObjectURL).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should revoke the previous frame when showing the next', async () => {
|
||||
const { image, controller } = createSurfaceController();
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[1]);
|
||||
expect(URL.revokeObjectURL).toBeCalledTimes(1);
|
||||
expect(URL.revokeObjectURL).toBeCalledWith(createdURLs()[0]);
|
||||
});
|
||||
|
||||
it('should keep showing the previous frame until the next has decoded', async () => {
|
||||
let releaseSecond: () => void = () => {};
|
||||
let calls = 0;
|
||||
const { image, controller } = createSurfaceController({
|
||||
decode: () => {
|
||||
calls++;
|
||||
return calls >= 2
|
||||
? new Promise<void>((resolve) => (releaseSecond = resolve))
|
||||
: Promise.resolve();
|
||||
},
|
||||
});
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[0]);
|
||||
|
||||
controller.showFrame(createFrame());
|
||||
await flushPromises();
|
||||
|
||||
// The next frame is still decoding off-DOM, so the visible <img> is
|
||||
// untouched and the previous URL stays valid (revoking it would blank it).
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[0]);
|
||||
expect(URL.revokeObjectURL).not.toBeCalled();
|
||||
|
||||
releaseSecond();
|
||||
await flushPromises();
|
||||
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[1]);
|
||||
expect(URL.revokeObjectURL).toBeCalledWith(createdURLs()[0]);
|
||||
});
|
||||
|
||||
it('should present only the newest frame while a decode is in flight', async () => {
|
||||
let releaseDecode: () => void = () => {};
|
||||
const decoding = new Promise<void>((resolve) => (releaseDecode = resolve));
|
||||
const { image, controller } = createSurfaceController({ decode: () => decoding });
|
||||
|
||||
controller.showFrame(createFrame());
|
||||
controller.showFrame(createFrame());
|
||||
controller.showFrame(createFrame());
|
||||
await flushPromises();
|
||||
|
||||
// Only the first frame's decode has started; the rest wait behind it.
|
||||
expect(URL.createObjectURL).toBeCalledTimes(1);
|
||||
|
||||
releaseDecode();
|
||||
await flushPromises();
|
||||
|
||||
// The middle frame was superseded, so only the newest is presented next.
|
||||
expect(URL.createObjectURL).toBeCalledTimes(2);
|
||||
expect(image.getAttribute('src')).toBe(createdURLs()[1]);
|
||||
});
|
||||
|
||||
it('should drop an undecodable frame and keep the current one', async () => {
|
||||
const { image, controller } = createSurfaceController({
|
||||
decode: () => Promise.reject(new Error('bad frame')),
|
||||
});
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
expect(URL.revokeObjectURL).toBeCalledWith(createdURLs()[0]);
|
||||
});
|
||||
|
||||
it('should not paint a frame that decoded after the surface detached', async () => {
|
||||
let releaseDecode: () => void = () => {};
|
||||
const { image, controller } = createSurfaceController({
|
||||
decode: () => new Promise<void>((resolve) => (releaseDecode = resolve)),
|
||||
});
|
||||
|
||||
controller.showFrame(createFrame());
|
||||
await flushPromises();
|
||||
|
||||
image.remove();
|
||||
releaseDecode();
|
||||
await flushPromises();
|
||||
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
expect(URL.revokeObjectURL).toBeCalledWith(createdURLs()[0]);
|
||||
});
|
||||
|
||||
it('should do nothing without an image element', async () => {
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => null);
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
expect(URL.createObjectURL).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should not paint onto a detached element', async () => {
|
||||
const { controller } = createSurfaceController({ connected: false });
|
||||
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
expect(URL.createObjectURL).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('reset', () => {
|
||||
it('should revoke the current object URL and clear the image', async () => {
|
||||
const { image, controller } = createSurfaceController();
|
||||
await controller.showFrame(createFrame());
|
||||
|
||||
controller.reset();
|
||||
|
||||
expect(URL.revokeObjectURL).toBeCalledWith(createdURLs()[0]);
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
});
|
||||
|
||||
it('should clear the image with no frame shown', () => {
|
||||
const { image, controller } = createSurfaceController();
|
||||
|
||||
controller.reset();
|
||||
|
||||
expect(URL.revokeObjectURL).not.toBeCalled();
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
});
|
||||
|
||||
it('should drop a frame still decoding so it never paints', async () => {
|
||||
let releaseDecode: () => void = () => {};
|
||||
const { image, controller } = createSurfaceController({
|
||||
decode: () => new Promise<void>((resolve) => (releaseDecode = resolve)),
|
||||
});
|
||||
|
||||
controller.showFrame(createFrame());
|
||||
await flushPromises();
|
||||
|
||||
controller.reset();
|
||||
releaseDecode();
|
||||
await flushPromises();
|
||||
|
||||
// The in-flight frame was invalidated by reset, so it never painted.
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
});
|
||||
|
||||
it('should do nothing without an image element', () => {
|
||||
const controller = new ImageSurfaceController(createLitElement(), () => null);
|
||||
|
||||
expect(() => controller.reset()).not.toThrow();
|
||||
expect(URL.revokeObjectURL).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,54 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { OffscreenImage } from '../../../../../src/components-lib/live/providers/go2rtc-experimental/offscreen-image';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('OffscreenImage', () => {
|
||||
it('should create the image from the factory on get', () => {
|
||||
const image = document.createElement('img');
|
||||
const offscreen = new OffscreenImage(() => image);
|
||||
|
||||
expect(offscreen.get()).toBe(image);
|
||||
});
|
||||
|
||||
it('should reuse the same image across repeated get', () => {
|
||||
const create = vi.fn(() => document.createElement('img'));
|
||||
const offscreen = new OffscreenImage(create);
|
||||
|
||||
expect(offscreen.get()).toBe(offscreen.get());
|
||||
expect(create).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should create an image with the default factory when none is injected', () => {
|
||||
const offscreen = new OffscreenImage();
|
||||
|
||||
expect(offscreen.get()).toBeInstanceOf(HTMLImageElement);
|
||||
});
|
||||
|
||||
it('should detach src on clear', () => {
|
||||
const image = document.createElement('img');
|
||||
const offscreen = new OffscreenImage(() => image);
|
||||
offscreen.get();
|
||||
image.src = 'data:image/jpeg;base64,AAAA';
|
||||
|
||||
offscreen.clear();
|
||||
|
||||
expect(image.hasAttribute('src')).toBe(false);
|
||||
});
|
||||
|
||||
it('should create a fresh image after clear', () => {
|
||||
const create = vi.fn(() => document.createElement('img'));
|
||||
const offscreen = new OffscreenImage(create);
|
||||
offscreen.get();
|
||||
offscreen.clear();
|
||||
offscreen.get();
|
||||
|
||||
expect(create).toBeCalledTimes(2);
|
||||
});
|
||||
|
||||
it('should tolerate clear when no image is held', () => {
|
||||
const offscreen = new OffscreenImage(() => document.createElement('img'));
|
||||
|
||||
expect(() => offscreen.clear()).not.toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,59 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { OffscreenVideo } from '../../../../../src/components-lib/live/providers/go2rtc-experimental/offscreen-video';
|
||||
import { FakeMediaStream, FakeMediaStreamTrack } from './test-utils';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('OffscreenVideo', () => {
|
||||
it('should create the video from the factory on get', () => {
|
||||
const video = document.createElement('video');
|
||||
const offscreen = new OffscreenVideo(() => video);
|
||||
|
||||
expect(offscreen.get()).toBe(video);
|
||||
});
|
||||
|
||||
it('should reuse the same video across repeated get', () => {
|
||||
const create = vi.fn(() => document.createElement('video'));
|
||||
const offscreen = new OffscreenVideo(create);
|
||||
|
||||
expect(offscreen.get()).toBe(offscreen.get());
|
||||
expect(create).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should create a video with the default factory when none is injected', () => {
|
||||
const offscreen = new OffscreenVideo();
|
||||
|
||||
expect(offscreen.get()).toBeInstanceOf(HTMLVideoElement);
|
||||
});
|
||||
|
||||
it('should detach src and srcObject on clear', () => {
|
||||
const video = document.createElement('video');
|
||||
const offscreen = new OffscreenVideo(() => video);
|
||||
offscreen.get();
|
||||
video.src = 'data:video/mp4;base64,AAAA';
|
||||
video.srcObject = new FakeMediaStream([
|
||||
new FakeMediaStreamTrack('video'),
|
||||
]).asMediaStream();
|
||||
|
||||
offscreen.clear();
|
||||
|
||||
expect(video.hasAttribute('src')).toBe(false);
|
||||
expect(video.srcObject).toBeNull();
|
||||
});
|
||||
|
||||
it('should create a fresh video after clear', () => {
|
||||
const create = vi.fn(() => document.createElement('video'));
|
||||
const offscreen = new OffscreenVideo(create);
|
||||
offscreen.get();
|
||||
offscreen.clear();
|
||||
offscreen.get();
|
||||
|
||||
expect(create).toBeCalledTimes(2);
|
||||
});
|
||||
|
||||
it('should tolerate clear when no video is held', () => {
|
||||
const offscreen = new OffscreenVideo(() => document.createElement('video'));
|
||||
|
||||
expect(() => offscreen.clear()).not.toThrow();
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,251 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { SignalingChannel } from '../../../../../src/components-lib/live/providers/go2rtc-experimental/signaling';
|
||||
import { FakeWebSocket } from './test-utils';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('SignalingChannel', () => {
|
||||
const setup = (options?: {
|
||||
openCallback?: () => void;
|
||||
disconnectCallback?: () => void;
|
||||
}) => {
|
||||
const websockets: FakeWebSocket[] = [];
|
||||
const createWebSocket = vi.fn<[string], WebSocket>(() => {
|
||||
const websocket = new FakeWebSocket();
|
||||
websockets.push(websocket);
|
||||
return websocket.asWebSocket();
|
||||
});
|
||||
const channel = new SignalingChannel(
|
||||
'ws://host/api/ws?src=camera',
|
||||
{
|
||||
openCallback: options?.openCallback,
|
||||
disconnectCallback: options?.disconnectCallback,
|
||||
},
|
||||
{ createWebSocket },
|
||||
);
|
||||
return { channel, createWebSocket, websockets };
|
||||
};
|
||||
|
||||
it('should connect with an arraybuffer binary type', () => {
|
||||
const { channel, createWebSocket, websockets } = setup();
|
||||
channel.connect();
|
||||
|
||||
expect(createWebSocket).toBeCalledWith('ws://host/api/ws?src=camera');
|
||||
expect(websockets[0].binaryType).toBe('arraybuffer');
|
||||
});
|
||||
|
||||
it('should not connect twice', () => {
|
||||
const { channel, createWebSocket } = setup();
|
||||
channel.connect();
|
||||
channel.connect();
|
||||
|
||||
expect(createWebSocket).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should report open state and call the open callback', () => {
|
||||
const openCallback = vi.fn();
|
||||
const { channel, websockets } = setup({ openCallback });
|
||||
channel.connect();
|
||||
|
||||
expect(channel.isOpen()).toBe(false);
|
||||
|
||||
websockets[0].fireOpen();
|
||||
|
||||
expect(channel.isOpen()).toBe(true);
|
||||
expect(openCallback).toBeCalled();
|
||||
});
|
||||
|
||||
it('should tolerate an absent open callback', () => {
|
||||
const { channel, websockets } = setup();
|
||||
channel.connect();
|
||||
|
||||
expect(() => websockets[0].fireOpen()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should not send before the connection is open', () => {
|
||||
const { channel, websockets } = setup();
|
||||
channel.connect();
|
||||
channel.send({ type: 'mse', value: 'codecs' });
|
||||
|
||||
expect(websockets[0].send).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should send JSON once open', () => {
|
||||
const { channel, websockets } = setup();
|
||||
channel.connect();
|
||||
websockets[0].fireOpen();
|
||||
const message = { type: 'mse', value: 'codecs' };
|
||||
channel.send(message);
|
||||
|
||||
expect(websockets[0].sent).toEqual([JSON.stringify(message)]);
|
||||
});
|
||||
|
||||
it('should dispatch parsed messages to subscribers', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const callback = vi.fn();
|
||||
channel.subscribeToMessages(callback);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage('{"type":"mse","value":"video/mp4"}');
|
||||
|
||||
expect(callback).toBeCalledWith({ type: 'mse', value: 'video/mp4' });
|
||||
});
|
||||
|
||||
it('should stop dispatching after unsubscribe', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const callback = vi.fn();
|
||||
const unsubscribe = channel.subscribeToMessages(callback);
|
||||
channel.connect();
|
||||
unsubscribe();
|
||||
websockets[0].fireMessage('{"type":"mse"}');
|
||||
|
||||
expect(callback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should dispatch to remaining subscribers when one unsubscribes during dispatch', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const secondCallback = vi.fn();
|
||||
const unsubscribeDuringDispatch = vi.fn((): void => {
|
||||
unsubscribe();
|
||||
});
|
||||
const unsubscribe = channel.subscribeToMessages(unsubscribeDuringDispatch);
|
||||
channel.subscribeToMessages(secondCallback);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage('{"type":"mse"}');
|
||||
|
||||
expect(unsubscribeDuringDispatch).toBeCalledTimes(1);
|
||||
expect(secondCallback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should ignore invalid JSON', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const callback = vi.fn();
|
||||
channel.subscribeToMessages(callback);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage('NOT JSON');
|
||||
|
||||
expect(callback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should ignore malformed messages', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const callback = vi.fn();
|
||||
channel.subscribeToMessages(callback);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage('{"type":6}');
|
||||
|
||||
expect(callback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should ignore unexpected data types', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const callback = vi.fn();
|
||||
channel.subscribeToMessages(callback);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage(42);
|
||||
|
||||
expect(callback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should route binary data to the binary callback', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const binaryCallback = vi.fn();
|
||||
channel.setBinaryCallback(binaryCallback);
|
||||
channel.connect();
|
||||
const data = new ArrayBuffer(8);
|
||||
websockets[0].fireMessage(data);
|
||||
|
||||
expect(binaryCallback).toBeCalledWith(data);
|
||||
});
|
||||
|
||||
it('should drop binary data without a binary callback', () => {
|
||||
const { channel, websockets } = setup();
|
||||
channel.connect();
|
||||
|
||||
expect(() => websockets[0].fireMessage(new ArrayBuffer(8))).not.toThrow();
|
||||
});
|
||||
|
||||
it('should drop binary data after the binary callback is cleared', () => {
|
||||
const { channel, websockets } = setup();
|
||||
const binaryCallback = vi.fn();
|
||||
channel.setBinaryCallback(binaryCallback);
|
||||
channel.setBinaryCallback(null);
|
||||
channel.connect();
|
||||
websockets[0].fireMessage(new ArrayBuffer(8));
|
||||
|
||||
expect(binaryCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should close the underlying websocket without firing the disconnect callback', () => {
|
||||
const disconnectCallback = vi.fn();
|
||||
const { channel, websockets } = setup({ disconnectCallback });
|
||||
channel.connect();
|
||||
websockets[0].fireOpen();
|
||||
channel.close();
|
||||
|
||||
expect(websockets[0].close).toBeCalled();
|
||||
expect(channel.isOpen()).toBe(false);
|
||||
expect(disconnectCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should tolerate closing when never connected', () => {
|
||||
const { channel } = setup();
|
||||
|
||||
expect(() => channel.close()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should ignore websocket events delivered after close', () => {
|
||||
const openCallback = vi.fn();
|
||||
const disconnectCallback = vi.fn();
|
||||
const messageCallback = vi.fn();
|
||||
const { channel, websockets } = setup({ openCallback, disconnectCallback });
|
||||
channel.subscribeToMessages(messageCallback);
|
||||
channel.connect();
|
||||
channel.close();
|
||||
|
||||
websockets[0].fireOpen();
|
||||
websockets[0].fireMessage('{"type":"mse"}');
|
||||
websockets[0].fireClose();
|
||||
|
||||
expect(openCallback).not.toBeCalled();
|
||||
expect(messageCallback).not.toBeCalled();
|
||||
expect(disconnectCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should fire the disconnect callback on unexpected closure', () => {
|
||||
const disconnectCallback = vi.fn();
|
||||
const { channel, websockets } = setup({ disconnectCallback });
|
||||
channel.connect();
|
||||
websockets[0].fireOpen();
|
||||
websockets[0].fireClose();
|
||||
|
||||
expect(disconnectCallback).toBeCalledTimes(1);
|
||||
expect(channel.isOpen()).toBe(false);
|
||||
});
|
||||
|
||||
it('should tolerate an absent disconnect callback on unexpected closure', () => {
|
||||
const { channel, websockets } = setup();
|
||||
channel.connect();
|
||||
|
||||
expect(() => websockets[0].fireClose()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should allow reconnecting after unexpected closure', () => {
|
||||
const { channel, createWebSocket, websockets } = setup();
|
||||
channel.connect();
|
||||
websockets[0].fireClose();
|
||||
channel.connect();
|
||||
|
||||
expect(createWebSocket).toBeCalledTimes(2);
|
||||
});
|
||||
|
||||
it('should construct a real websocket by default', () => {
|
||||
const webSocketConstructor = vi.fn(() => new FakeWebSocket().asWebSocket());
|
||||
vi.stubGlobal('WebSocket', webSocketConstructor);
|
||||
const channel = new SignalingChannel('ws://host/api/ws', {});
|
||||
channel.connect();
|
||||
channel.close();
|
||||
|
||||
expect(webSocketConstructor).toHaveBeenCalledWith('ws://host/api/ws');
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,90 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import {
|
||||
createBinarySource,
|
||||
createWebRTCSource,
|
||||
type BinaryStreamTargets,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/factory';
|
||||
import { MJPEGStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mjpeg';
|
||||
import { MP4StreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mp4';
|
||||
import { MSEStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mse';
|
||||
import { WebRTCStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/webrtc';
|
||||
import type {
|
||||
StreamSourceCallbacks,
|
||||
StreamSourceContext,
|
||||
VideoStreamTarget,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import { FakeStreamSourceChannel } from '../test-utils';
|
||||
|
||||
const createTargets = (): BinaryStreamTargets => ({
|
||||
video: { kind: 'video', video: document.createElement('video') },
|
||||
image: { kind: 'image', showFrame: vi.fn() },
|
||||
});
|
||||
|
||||
const createCallbacks = (): StreamSourceCallbacks => ({
|
||||
loadedCallback: vi.fn(),
|
||||
failedCallback: vi.fn(),
|
||||
});
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('createBinarySource', () => {
|
||||
it('should create an MSE source on the video surface', () => {
|
||||
const result = createBinarySource(
|
||||
'mse',
|
||||
createTargets(),
|
||||
new FakeStreamSourceChannel(),
|
||||
createCallbacks(),
|
||||
);
|
||||
|
||||
expect(result?.source).toBeInstanceOf(MSEStreamSource);
|
||||
expect(result?.surface).toBe('video');
|
||||
});
|
||||
|
||||
it('should create an MP4 source on the image surface', () => {
|
||||
const result = createBinarySource(
|
||||
'mp4',
|
||||
createTargets(),
|
||||
new FakeStreamSourceChannel(),
|
||||
createCallbacks(),
|
||||
);
|
||||
|
||||
expect(result?.source).toBeInstanceOf(MP4StreamSource);
|
||||
expect(result?.surface).toBe('image');
|
||||
});
|
||||
|
||||
it('should create an MJPEG source on the image surface', () => {
|
||||
const result = createBinarySource(
|
||||
'mjpeg',
|
||||
createTargets(),
|
||||
new FakeStreamSourceChannel(),
|
||||
createCallbacks(),
|
||||
);
|
||||
|
||||
expect(result?.source).toBeInstanceOf(MJPEGStreamSource);
|
||||
expect(result?.surface).toBe('image');
|
||||
});
|
||||
|
||||
it('should return null for the webrtc mode', () => {
|
||||
// WebRTC is not a binary source; it is created via createWebRTCSource.
|
||||
expect(
|
||||
createBinarySource(
|
||||
'webrtc',
|
||||
createTargets(),
|
||||
new FakeStreamSourceChannel(),
|
||||
createCallbacks(),
|
||||
),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('createWebRTCSource', () => {
|
||||
it('should create a WebRTC source', () => {
|
||||
const context: StreamSourceContext<VideoStreamTarget> = {
|
||||
target: { kind: 'video', video: document.createElement('video') },
|
||||
channel: new FakeStreamSourceChannel(),
|
||||
callbacks: createCallbacks(),
|
||||
};
|
||||
|
||||
expect(createWebRTCSource(context)).toBeInstanceOf(WebRTCStreamSource);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,170 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { MJPEGStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mjpeg';
|
||||
import type {
|
||||
ImageStreamTarget,
|
||||
StreamSourceContext,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import { flushPromises } from '../../../../../test-utils';
|
||||
import { FakeStreamSourceChannel } from '../test-utils';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('MJPEGStreamSource', () => {
|
||||
const setup = () => {
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const loadedCallback = vi.fn();
|
||||
const failedCallback = vi.fn();
|
||||
const showFrame = vi.fn<[Blob], Promise<void>>(() => Promise.resolve());
|
||||
|
||||
const context: StreamSourceContext<ImageStreamTarget> = {
|
||||
target: { kind: 'image', showFrame },
|
||||
channel,
|
||||
callbacks: { loadedCallback, failedCallback },
|
||||
};
|
||||
const source = new MJPEGStreamSource(context);
|
||||
|
||||
return { channel, failedCallback, loadedCallback, showFrame, source };
|
||||
};
|
||||
|
||||
const frame = (): ArrayBuffer => new TextEncoder().encode('Hi').buffer;
|
||||
|
||||
it('should request the mjpeg stream on start', () => {
|
||||
const { source, channel } = setup();
|
||||
source.start();
|
||||
|
||||
expect(channel.sent).toEqual([{ type: 'mjpeg' }]);
|
||||
expect(channel.binaryCallback).not.toBeNull();
|
||||
});
|
||||
|
||||
it('should show each frame as a JPEG image', () => {
|
||||
const { source, channel, showFrame } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
|
||||
expect(showFrame).toBeCalledTimes(1);
|
||||
const shown = showFrame.mock.calls[0][0] as Blob;
|
||||
expect(shown).toBeInstanceOf(Blob);
|
||||
expect(shown.type).toBe('image/jpeg');
|
||||
expect(shown.size).toBe(2);
|
||||
});
|
||||
|
||||
it('should report loaded only on the first frame', async () => {
|
||||
const { source, channel, loadedCallback } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
channel.binaryCallback?.(frame());
|
||||
await flushPromises();
|
||||
|
||||
expect(loadedCallback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should not report loaded when stopped before the first frame decodes', async () => {
|
||||
const { source, channel, loadedCallback, showFrame } = setup();
|
||||
let resolveDecode: () => void = () => {};
|
||||
showFrame.mockReturnValue(
|
||||
new Promise<void>((resolve) => {
|
||||
resolveDecode = resolve;
|
||||
}),
|
||||
);
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
|
||||
// Stop before the frame's decode resolves; the deferred loaded report must
|
||||
// then be dropped.
|
||||
source.stop();
|
||||
resolveDecode();
|
||||
await flushPromises();
|
||||
|
||||
expect(loadedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should fail on a server error for mjpeg', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'mjpeg: stream not found' });
|
||||
|
||||
expect(failedCallback).toBeCalledWith('server_error');
|
||||
});
|
||||
|
||||
it('should ignore a server error for another mode', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'mse: stream not found' });
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should stop cleanly', () => {
|
||||
const { source, channel } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
|
||||
expect(channel.binaryCallback).toBeNull();
|
||||
expect(channel.getMessageCallbackCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('should tolerate stopping before starting', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(() => source.stop()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should report no-pause capabilities', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getCapabilities()).toEqual({ supportsPause: false });
|
||||
});
|
||||
|
||||
it('should report mjpeg technology', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getTechnology()).toEqual(['mjpeg']);
|
||||
});
|
||||
|
||||
it('should report a video-only stream profile', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getStreamProfile()).toEqual({
|
||||
hasVideo: true,
|
||||
hasH265Video: false,
|
||||
hasAudio: false,
|
||||
hasAACAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
describe('first-frame timeout', () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('should fail when no frame arrives within the timeout', () => {
|
||||
const { source, failedCallback } = setup();
|
||||
source.start();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).toBeCalledWith('connect_timeout');
|
||||
});
|
||||
|
||||
it('should not fail once a frame has arrived', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should not fail after stop', () => {
|
||||
const { source, failedCallback } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,232 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { MP4StreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mp4';
|
||||
import type {
|
||||
ImageStreamTarget,
|
||||
StreamSourceContext,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import { flushPromises } from '../../../../../test-utils';
|
||||
import { FakeStreamSourceChannel } from '../test-utils';
|
||||
|
||||
class FakeCanvasContext {
|
||||
public drawImage = vi.fn();
|
||||
}
|
||||
|
||||
class FakeCanvas {
|
||||
public width = 0;
|
||||
public height = 0;
|
||||
public context: FakeCanvasContext | null = new FakeCanvasContext();
|
||||
public getContext = vi.fn(() => this.context);
|
||||
public toBlob = vi.fn((callback: (blob: Blob | null) => void) =>
|
||||
callback(new Blob(['frame'], { type: 'image/jpeg' })),
|
||||
);
|
||||
|
||||
public asCanvas(): HTMLCanvasElement {
|
||||
return this as unknown as HTMLCanvasElement;
|
||||
}
|
||||
}
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('MP4StreamSource', () => {
|
||||
const setup = () => {
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const loadedCallback = vi.fn();
|
||||
const failedCallback = vi.fn();
|
||||
const showFrame = vi.fn<[Blob], Promise<void>>(() => Promise.resolve());
|
||||
const context: StreamSourceContext<ImageStreamTarget> = {
|
||||
target: { kind: 'image', showFrame },
|
||||
channel,
|
||||
callbacks: { loadedCallback, failedCallback },
|
||||
};
|
||||
const decoderVideo = document.createElement('video');
|
||||
const canvas = new FakeCanvas();
|
||||
const createVideoElement = vi.fn(() => decoderVideo);
|
||||
const source = new MP4StreamSource(context, {
|
||||
createVideoElement,
|
||||
createCanvasElement: () => canvas.asCanvas(),
|
||||
});
|
||||
return {
|
||||
canvas,
|
||||
channel,
|
||||
createVideoElement,
|
||||
decoderVideo,
|
||||
failedCallback,
|
||||
loadedCallback,
|
||||
showFrame,
|
||||
source,
|
||||
};
|
||||
};
|
||||
|
||||
const frame = (): ArrayBuffer => new TextEncoder().encode('Hi').buffer;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('should request the mp4 stream on start', () => {
|
||||
const { source, channel } = setup();
|
||||
source.start();
|
||||
|
||||
expect(channel.sent[0].type).toBe('mp4');
|
||||
expect(channel.binaryCallback).not.toBeNull();
|
||||
});
|
||||
|
||||
it('should feed each frame to a muted autoplay decoder video', () => {
|
||||
const { source, channel, decoderVideo } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
|
||||
expect(decoderVideo.autoplay).toBe(true);
|
||||
expect(decoderVideo.muted).toBe(true);
|
||||
expect(decoderVideo.getAttribute('src')).toBe('data:video/mp4;base64,SGk=');
|
||||
});
|
||||
|
||||
it('should reuse the decoder across frames', () => {
|
||||
const { source, channel, createVideoElement } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
channel.binaryCallback?.(frame());
|
||||
|
||||
// Second frame reuses the same decoder rather than creating another.
|
||||
expect(createVideoElement).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should draw a decoded frame and show it as an image', async () => {
|
||||
const { source, channel, decoderVideo, canvas, showFrame, loadedCallback } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
await flushPromises();
|
||||
|
||||
expect(canvas.context?.drawImage).toBeCalled();
|
||||
expect(showFrame).toBeCalledTimes(1);
|
||||
const shown = showFrame.mock.calls[0][0] as Blob;
|
||||
expect(shown).toBeInstanceOf(Blob);
|
||||
expect(shown.type).toBe('image/jpeg');
|
||||
expect(loadedCallback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should report loaded only on the first drawn frame', async () => {
|
||||
const { source, channel, decoderVideo, loadedCallback } = setup();
|
||||
source.start();
|
||||
|
||||
channel.binaryCallback?.(frame());
|
||||
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
await flushPromises();
|
||||
|
||||
expect(loadedCallback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should not show a frame when the canvas produces no blob', () => {
|
||||
const { source, channel, decoderVideo, canvas, showFrame } = setup();
|
||||
canvas.toBlob = vi.fn((callback: (blob: Blob | null) => void) => callback(null));
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
|
||||
expect(showFrame).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should do nothing when the canvas has no 2d context', () => {
|
||||
const { source, channel, decoderVideo, canvas, showFrame } = setup();
|
||||
canvas.context = null;
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
|
||||
expect(showFrame).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should fail on a server error for mp4', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'mp4: stream not found' });
|
||||
|
||||
expect(failedCallback).toBeCalledWith('server_error');
|
||||
});
|
||||
|
||||
it('should clear the decoder on stop', () => {
|
||||
const { source, channel, decoderVideo } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
source.stop();
|
||||
|
||||
expect(decoderVideo.hasAttribute('src')).toBe(false);
|
||||
expect(channel.binaryCallback).toBeNull();
|
||||
});
|
||||
|
||||
it('should not show a frame decoded after stop', () => {
|
||||
const { source, channel, decoderVideo, showFrame } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
source.stop();
|
||||
|
||||
// A frame whose decode completes after stop() must not reach the image
|
||||
// surface.
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
|
||||
expect(showFrame).not.toBeCalled();
|
||||
});
|
||||
|
||||
describe('first-frame timeout', () => {
|
||||
it('should fail when no frame arrives within the timeout', () => {
|
||||
const { source, failedCallback } = setup();
|
||||
source.start();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).toBeCalledWith('connect_timeout');
|
||||
});
|
||||
|
||||
it('should not fail once a frame has been drawn', () => {
|
||||
const { source, channel, decoderVideo, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
decoderVideo.dispatchEvent(new Event('loadeddata'));
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should not fail after stop', () => {
|
||||
const { source, failedCallback } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
|
||||
it('should report mp4 technology', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getTechnology()).toEqual(['mp4']);
|
||||
});
|
||||
|
||||
it('should default to document element factories when none are injected', () => {
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const source = new MP4StreamSource({
|
||||
target: { kind: 'image', showFrame: vi.fn() },
|
||||
channel,
|
||||
callbacks: { loadedCallback: vi.fn(), failedCallback: vi.fn() },
|
||||
});
|
||||
|
||||
const createElement = vi.spyOn(document, 'createElement');
|
||||
source.start();
|
||||
channel.binaryCallback?.(frame());
|
||||
const decoder = createElement.mock.results[
|
||||
createElement.mock.calls.findIndex((call) => call[0] === 'video')
|
||||
].value as HTMLVideoElement;
|
||||
decoder.dispatchEvent(new Event('loadeddata'));
|
||||
|
||||
expect(createElement).toHaveBeenCalledWith('video');
|
||||
expect(createElement).toHaveBeenCalledWith('canvas');
|
||||
createElement.mockRestore();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,646 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { MSEStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/mse';
|
||||
import type {
|
||||
StreamSourceContext,
|
||||
VideoStreamTarget,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import {
|
||||
CHROME_USER_AGENT,
|
||||
createFakeMediaSourceFactory,
|
||||
createTimeRanges,
|
||||
FakeMediaSourceInstance,
|
||||
FakeStreamSourceChannel,
|
||||
SAFARI_17_USER_AGENT,
|
||||
} from '../test-utils';
|
||||
|
||||
const ALL_CHROME_CODECS =
|
||||
'avc1.640029,avc1.64002A,avc1.640033,hvc1.1.6.L153.B0,mp4a.40.2,mp4a.40.5,flac,opus';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('MSEStreamSource', () => {
|
||||
const setup = (options?: { userAgent?: string; unsupported?: boolean }) => {
|
||||
const video = document.createElement('video');
|
||||
// jsdom reports a fresh <video> as paused; live playback is the default
|
||||
// state under test, and the live-edge logic only runs while playing.
|
||||
Object.defineProperty(video, 'paused', {
|
||||
configurable: true,
|
||||
writable: true,
|
||||
value: false,
|
||||
});
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const loadedCallback = vi.fn();
|
||||
const failedCallback = vi.fn();
|
||||
const context: StreamSourceContext<VideoStreamTarget> = {
|
||||
target: { kind: 'video', video },
|
||||
channel,
|
||||
callbacks: { loadedCallback, failedCallback },
|
||||
};
|
||||
const instance = new FakeMediaSourceInstance();
|
||||
const source = new MSEStreamSource(context, {
|
||||
createMediaSource: createFakeMediaSourceFactory(
|
||||
options?.unsupported ? null : instance,
|
||||
),
|
||||
userAgent: options?.userAgent ?? CHROME_USER_AGENT,
|
||||
});
|
||||
return { channel, failedCallback, instance, loadedCallback, source, video };
|
||||
};
|
||||
|
||||
type SetupResult = ReturnType<typeof setup>;
|
||||
|
||||
const negotiate = (setupResult: SetupResult): void => {
|
||||
setupResult.source.start();
|
||||
setupResult.instance.fireSourceOpen();
|
||||
setupResult.channel.receiveMessage({
|
||||
type: 'mse',
|
||||
value: 'video/mp4; codecs="avc1.640029,mp4a.40.2"',
|
||||
});
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
describe('startup', () => {
|
||||
it('should fail without MediaSource support', () => {
|
||||
const { source, failedCallback } = setup({ unsupported: true });
|
||||
source.start();
|
||||
|
||||
expect(failedCallback).toBeCalledWith('unsupported');
|
||||
});
|
||||
|
||||
it('should attach the media source to the video on start', () => {
|
||||
const { source, instance, video } = setup();
|
||||
source.start();
|
||||
|
||||
expect(instance.attach).toBeCalledWith(video);
|
||||
});
|
||||
});
|
||||
|
||||
describe('negotiation', () => {
|
||||
it('should offer supported codecs when the source opens', () => {
|
||||
const { source, instance, channel } = setup();
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
|
||||
expect(channel.sent).toEqual([{ type: 'mse', value: ALL_CHROME_CODECS }]);
|
||||
});
|
||||
|
||||
it('should offer Safari-reliable codecs on Safari', () => {
|
||||
const { source, instance, channel } = setup({ userAgent: SAFARI_17_USER_AGENT });
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
|
||||
expect(channel.sent).toEqual([
|
||||
{
|
||||
type: 'mse',
|
||||
value:
|
||||
'avc1.640029,avc1.64002A,avc1.640033,hvc1.1.6.L153.B0,mp4a.40.2,mp4a.40.5,flac',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('should exclude codecs the media source does not support', () => {
|
||||
const { source, instance, channel } = setup();
|
||||
instance.isTypeSupported.mockImplementation(
|
||||
(mimeType: string) => mimeType === 'video/mp4; codecs="avc1.640029"',
|
||||
);
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
|
||||
expect(channel.sent).toEqual([{ type: 'mse', value: 'avc1.640029' }]);
|
||||
});
|
||||
|
||||
it('should fail when negotiation times out', () => {
|
||||
const { source, instance, failedCallback } = setup();
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).toBeCalledWith('negotiation_timeout');
|
||||
});
|
||||
|
||||
it('should not time out after a successful negotiation', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should create a source buffer in segments mode on negotiation', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
|
||||
expect(setupResult.instance.addSourceBuffer).toBeCalledWith(
|
||||
'video/mp4; codecs="avc1.640029,mp4a.40.2"',
|
||||
);
|
||||
expect(setupResult.instance.sourceBuffer.mode).toBe('segments');
|
||||
expect(setupResult.channel.binaryCallback).not.toBeNull();
|
||||
});
|
||||
|
||||
it('should ignore a repeated negotiation response', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.channel.receiveMessage({ type: 'mse', value: 'video/mp4' });
|
||||
|
||||
expect(setupResult.instance.addSourceBuffer).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should ignore negotiation responses without a string value', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.channel.receiveMessage({ type: 'mse', value: 42 });
|
||||
|
||||
expect(setupResult.instance.addSourceBuffer).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should ignore unrelated messages', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.channel.receiveMessage({ type: 'webrtc/answer', value: 'sdp' });
|
||||
|
||||
expect(setupResult.instance.addSourceBuffer).not.toBeCalled();
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('server errors', () => {
|
||||
it('should fail on a server error for mse', () => {
|
||||
const { source, instance, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
channel.receiveMessage({ type: 'error', value: 'mse: stream not found' });
|
||||
|
||||
expect(failedCallback).toBeCalledWith('server_error');
|
||||
|
||||
// The negotiation timer must have stopped.
|
||||
failedCallback.mockClear();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should ignore server errors for other modes', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'webrtc/offer: failed' });
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should ignore server errors without a string value', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error' });
|
||||
|
||||
expect(failedCallback).not.toBeCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('source buffer', () => {
|
||||
it('should fail when the source buffer cannot be created', () => {
|
||||
const { source, instance, channel, failedCallback } = setup();
|
||||
instance.addSourceBuffer.mockImplementation(() => {
|
||||
throw new Error('InvalidStateError');
|
||||
});
|
||||
source.start();
|
||||
instance.fireSourceOpen();
|
||||
channel.receiveMessage({ type: 'mse', value: 'video/mp4' });
|
||||
|
||||
expect(failedCallback).toBeCalledWith('media_error');
|
||||
});
|
||||
|
||||
it('should append binary data directly when idle', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const data = new ArrayBuffer(8);
|
||||
setupResult.channel.binaryCallback?.(data);
|
||||
|
||||
expect(setupResult.instance.sourceBuffer.appendBuffer).toBeCalledWith(data);
|
||||
});
|
||||
|
||||
it('should swallow direct append failures', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.instance.sourceBuffer.appendBuffer.mockImplementation(() => {
|
||||
throw new Error('QuotaExceededError');
|
||||
});
|
||||
|
||||
expect(() =>
|
||||
setupResult.channel.binaryCallback?.(new ArrayBuffer(8)),
|
||||
).not.toThrow();
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should stage binary data while the source buffer updates', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.instance.sourceBuffer.updating = true;
|
||||
const staged = new ArrayBuffer(8);
|
||||
setupResult.channel.binaryCallback?.(staged);
|
||||
|
||||
expect(setupResult.instance.sourceBuffer.appendBuffer).not.toBeCalled();
|
||||
|
||||
setupResult.instance.sourceBuffer.updating = false;
|
||||
setupResult.instance.sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(setupResult.instance.sourceBuffer.appendBuffer).toBeCalledWith(staged);
|
||||
});
|
||||
|
||||
it('should stage binary data behind earlier staged data', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
|
||||
sourceBuffer.updating = true;
|
||||
const first = new ArrayBuffer(8);
|
||||
const second = new ArrayBuffer(4);
|
||||
setupResult.channel.binaryCallback?.(first);
|
||||
sourceBuffer.updating = false;
|
||||
setupResult.channel.binaryCallback?.(second);
|
||||
|
||||
expect(sourceBuffer.appendBuffer).not.toBeCalled();
|
||||
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(sourceBuffer.appendBuffer).toHaveBeenNthCalledWith(1, first);
|
||||
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(sourceBuffer.appendBuffer).toHaveBeenNthCalledWith(2, second);
|
||||
});
|
||||
|
||||
it('should fail when staged data exceeds the pending limit', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
|
||||
sourceBuffer.updating = true;
|
||||
setupResult.channel.binaryCallback?.(new ArrayBuffer(2 * 1024 * 1024));
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
|
||||
setupResult.channel.binaryCallback?.(new ArrayBuffer(1));
|
||||
expect(setupResult.failedCallback).toBeCalledWith('buffer_overflow');
|
||||
});
|
||||
});
|
||||
|
||||
describe('live edge', () => {
|
||||
it('should do nothing on updateend while still updating', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
sourceBuffer.updating = true;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(sourceBuffer.remove).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should do nothing on updateend without buffered content', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.instance.sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(setupResult.instance.sourceBuffer.remove).not.toBeCalled();
|
||||
expect(setupResult.instance.setLiveSeekableRange).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should not trim after the media source has closed', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
setupResult.video.currentTime = 19;
|
||||
|
||||
// A queued updateend fires after the source detaches on reconnect: its
|
||||
// duration is NaN, so remove() would throw. The trim must be skipped.
|
||||
setupResult.instance.isOpen.mockReturnValue(false);
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(sourceBuffer.remove).not.toBeCalled();
|
||||
expect(setupResult.instance.setLiveSeekableRange).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should trim media behind the retained window', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
setupResult.video.currentTime = 19;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
// Retains the last 15s (end 20 -> retainedStart 5).
|
||||
expect(sourceBuffer.remove).toBeCalledWith(0, 5);
|
||||
expect(setupResult.instance.setLiveSeekableRange).toBeCalledWith(5, 20);
|
||||
});
|
||||
|
||||
it('should not trim when all media is within the retained window', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[16, 20]]);
|
||||
setupResult.video.currentTime = 19;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(sourceBuffer.remove).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should not move the playhead when it falls behind the window', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
setupResult.video.currentTime = 2;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
// The trim no longer snaps the playhead forward; a playhead behind the
|
||||
// window is caught up by rate (or, on resume, a jump), not dragged.
|
||||
expect(setupResult.video.currentTime).toBe(2);
|
||||
});
|
||||
|
||||
it('should raise the playback rate when lag rises above the stream norm', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[16, 20]]);
|
||||
|
||||
// Establish a healthy 1s lag norm.
|
||||
setupResult.video.currentTime = 19;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(setupResult.video.playbackRate).toBe(1);
|
||||
|
||||
// A 5s lag now exceeds the adaptive threshold and triggers a gentle
|
||||
// catch-up above realtime.
|
||||
setupResult.video.currentTime = 15;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(setupResult.video.playbackRate).toBeGreaterThan(1);
|
||||
expect(setupResult.video.playbackRate).toBeLessThan(1.1);
|
||||
});
|
||||
|
||||
it('should reset the playback rate when caught up', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[16, 20]]);
|
||||
setupResult.video.currentTime = 19;
|
||||
setupResult.video.playbackRate = 2;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(setupResult.video.playbackRate).toBe(1);
|
||||
});
|
||||
|
||||
it('should leave an unchanged playback rate alone', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[16, 20]]);
|
||||
setupResult.video.currentTime = 19;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(setupResult.video.playbackRate).toBe(1);
|
||||
});
|
||||
|
||||
it('should seek forward to the hold-back on WebKit instead of changing the rate', () => {
|
||||
const setupResult = setup({ userAgent: SAFARI_17_USER_AGENT });
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[10, 20]]);
|
||||
setupResult.video.currentTime = 13;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
// Default GOP 1s -> hold-back 3s; a 7s lag is far behind, so seek to
|
||||
// bufferedEnd - 3 rather than changing the rate.
|
||||
expect(setupResult.video.currentTime).toBe(17);
|
||||
expect(setupResult.video.playbackRate).toBe(1);
|
||||
});
|
||||
|
||||
it('should seek back to the hold-back when starving at the live edge on WebKit', () => {
|
||||
const setupResult = setup({ userAgent: SAFARI_17_USER_AGENT });
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[10, 20]]);
|
||||
setupResult.video.currentTime = 19.5;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
// Within a GOP of the edge -> seek back to bufferedEnd - hold-back.
|
||||
expect(setupResult.video.currentTime).toBe(17);
|
||||
});
|
||||
|
||||
it('should respect the forward-seek cooldown on WebKit', () => {
|
||||
const setupResult = setup({ userAgent: SAFARI_17_USER_AGENT });
|
||||
negotiate(setupResult);
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[10, 20]]);
|
||||
setupResult.video.currentTime = 13;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
expect(setupResult.video.currentTime).toBe(17);
|
||||
|
||||
// Fall behind again immediately: within the cooldown there is no second
|
||||
// seek.
|
||||
setupResult.video.currentTime = 13;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(setupResult.video.currentTime).toBe(13);
|
||||
|
||||
// After the cooldown the forward seek resumes.
|
||||
vi.advanceTimersByTime(6 * 1000);
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
expect(setupResult.video.currentTime).toBe(17);
|
||||
});
|
||||
|
||||
it('should trim but not chase the live edge while paused', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
Object.defineProperty(setupResult.video, 'paused', {
|
||||
configurable: true,
|
||||
value: true,
|
||||
});
|
||||
const sourceBuffer = setupResult.instance.sourceBuffer;
|
||||
sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
setupResult.video.currentTime = 2;
|
||||
setupResult.video.playbackRate = 1;
|
||||
sourceBuffer.fireUpdateEnd();
|
||||
|
||||
// Trim still bounds memory, but the playhead and rate are left untouched.
|
||||
expect(sourceBuffer.remove).toBeCalled();
|
||||
expect(setupResult.video.currentTime).toBe(2);
|
||||
expect(setupResult.video.playbackRate).toBe(1);
|
||||
});
|
||||
|
||||
it('should jump to the live edge on resume', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.instance.sourceBuffer.buffered = createTimeRanges([[10, 20]]);
|
||||
setupResult.video.currentTime = 12;
|
||||
setupResult.video.dispatchEvent(new Event('play'));
|
||||
|
||||
// Jumps to bufferedEnd - 0.75.
|
||||
expect(setupResult.video.currentTime).toBe(19.25);
|
||||
});
|
||||
|
||||
it('should not jump on resume when already near the live edge', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.instance.sourceBuffer.buffered = createTimeRanges([[10, 20]]);
|
||||
setupResult.video.currentTime = 19.5;
|
||||
setupResult.video.dispatchEvent(new Event('play'));
|
||||
|
||||
// Lag (0.5s) is under the resume threshold, so the playhead is left alone.
|
||||
expect(setupResult.video.currentTime).toBe(19.5);
|
||||
});
|
||||
|
||||
it('should ignore resume before any media is buffered', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.video.currentTime = 5;
|
||||
setupResult.video.dispatchEvent(new Event('play'));
|
||||
|
||||
expect(setupResult.video.currentTime).toBe(5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('media events', () => {
|
||||
it('should report loaded media', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.video.dispatchEvent(new Event('loadeddata'));
|
||||
|
||||
expect(setupResult.loadedCallback).toBeCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should fail on video element errors', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.video.dispatchEvent(new Event('error'));
|
||||
|
||||
expect(setupResult.failedCallback).toBeCalledWith('media_error');
|
||||
});
|
||||
});
|
||||
|
||||
describe('lifecycle', () => {
|
||||
it('should stop cleanly', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
setupResult.source.stop();
|
||||
|
||||
expect(setupResult.instance.detach).toBeCalledWith(setupResult.video);
|
||||
expect(setupResult.channel.binaryCallback).toBeNull();
|
||||
expect(setupResult.channel.getMessageCallbackCount()).toBe(0);
|
||||
expect(setupResult.instance.getSourceOpenCallbackCount()).toBe(0);
|
||||
|
||||
setupResult.video.dispatchEvent(new Event('loadeddata'));
|
||||
setupResult.video.dispatchEvent(new Event('error'));
|
||||
expect(setupResult.loadedCallback).not.toBeCalled();
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
|
||||
setupResult.instance.sourceBuffer.buffered = createTimeRanges([[0, 20]]);
|
||||
setupResult.instance.sourceBuffer.fireUpdateEnd();
|
||||
expect(setupResult.instance.sourceBuffer.remove).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should stop the negotiation timer on stop', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.instance.fireSourceOpen();
|
||||
setupResult.source.stop();
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(setupResult.failedCallback).not.toBeCalled();
|
||||
});
|
||||
|
||||
it('should tolerate stopping before starting', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(() => source.stop()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('reporting', () => {
|
||||
it('should report capabilities before negotiation', () => {
|
||||
const { source } = setup();
|
||||
|
||||
// Without negotiated codecs, audio detection falls back to the video
|
||||
// element, whose jsdom audio track list is empty.
|
||||
expect(source.getCapabilities()).toEqual({
|
||||
supportsPause: true,
|
||||
hasAudio: false,
|
||||
has2WayAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report audio from negotiated codecs', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
|
||||
expect(setupResult.source.getCapabilities()).toEqual({
|
||||
supportsPause: true,
|
||||
hasAudio: true,
|
||||
has2WayAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report no audio for video-only codecs', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.instance.fireSourceOpen();
|
||||
setupResult.channel.receiveMessage({
|
||||
type: 'mse',
|
||||
value: 'video/mp4; codecs="avc1.640029"',
|
||||
});
|
||||
|
||||
expect(setupResult.source.getCapabilities()).toEqual({
|
||||
supportsPause: true,
|
||||
hasAudio: false,
|
||||
has2WayAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report technology', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getTechnology()).toEqual(['mse']);
|
||||
});
|
||||
|
||||
it('should report an empty stream profile before negotiation', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getStreamProfile()).toEqual({
|
||||
hasVideo: false,
|
||||
hasH265Video: false,
|
||||
hasAudio: false,
|
||||
hasAACAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report an H.264 and AAC stream profile from negotiated codecs', () => {
|
||||
const setupResult = setup();
|
||||
negotiate(setupResult);
|
||||
|
||||
expect(setupResult.source.getStreamProfile()).toEqual({
|
||||
hasVideo: true,
|
||||
hasH265Video: false,
|
||||
hasAudio: true,
|
||||
hasAACAudio: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report an H.265 stream profile', () => {
|
||||
const setupResult = setup();
|
||||
setupResult.source.start();
|
||||
setupResult.instance.fireSourceOpen();
|
||||
setupResult.channel.receiveMessage({
|
||||
type: 'mse',
|
||||
value: 'video/mp4; codecs="hvc1.1.6.L153.B0,opus"',
|
||||
});
|
||||
|
||||
expect(setupResult.source.getStreamProfile()).toEqual({
|
||||
hasVideo: true,
|
||||
hasH265Video: true,
|
||||
hasAudio: true,
|
||||
hasAACAudio: false,
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,549 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { WebRTCStreamSource } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/sources/webrtc';
|
||||
import type {
|
||||
StreamSourceContext,
|
||||
VideoStreamTarget,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import { flushPromises } from '../../../../../test-utils';
|
||||
import {
|
||||
FakeMediaStream,
|
||||
FakeMediaStreamTrack,
|
||||
FakeRTCPeerConnection,
|
||||
FakeStreamSourceChannel,
|
||||
} from '../test-utils';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('WebRTCStreamSource', () => {
|
||||
const setup = (options?: { microphoneStream?: FakeMediaStream | null }) => {
|
||||
const video = document.createElement('video');
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const loadedCallback = vi.fn();
|
||||
const failedCallback = vi.fn();
|
||||
|
||||
const context: StreamSourceContext<VideoStreamTarget> = {
|
||||
target: { kind: 'video', video },
|
||||
channel,
|
||||
callbacks: { loadedCallback, failedCallback },
|
||||
};
|
||||
|
||||
const pc = new FakeRTCPeerConnection();
|
||||
const createPeerConnection = vi.fn(() => pc.asPeerConnection());
|
||||
const source = new WebRTCStreamSource(context, {
|
||||
createPeerConnection,
|
||||
createMediaStream: (tracks) =>
|
||||
new FakeMediaStream(tracks as unknown as FakeMediaStreamTrack[]).asMediaStream(),
|
||||
microphoneStream: options?.microphoneStream?.asMediaStream() ?? null,
|
||||
});
|
||||
|
||||
return {
|
||||
channel,
|
||||
context,
|
||||
createPeerConnection,
|
||||
failedCallback,
|
||||
loadedCallback,
|
||||
pc,
|
||||
source,
|
||||
video,
|
||||
};
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
describe('transceivers', () => {
|
||||
it('should pre-arm a sendonly audio transceiver and recvonly video and audio', () => {
|
||||
const { source, pc } = setup();
|
||||
source.start();
|
||||
|
||||
expect(pc.transceivers).toHaveLength(3);
|
||||
expect(pc.transceivers[0].direction).toBe('sendonly');
|
||||
expect(pc.transceivers[1].direction).toBe('recvonly');
|
||||
expect(pc.transceivers[2].direction).toBe('recvonly');
|
||||
|
||||
// Kind-only pre-arm: no track, so no getUserMedia and no permission prompt.
|
||||
expect(pc.transceivers[0].sender.track).toBeNull();
|
||||
});
|
||||
|
||||
it('should pre-arm with the current microphone track', () => {
|
||||
const micTrack = new FakeMediaStreamTrack('audio');
|
||||
const { source, pc } = setup({
|
||||
microphoneStream: new FakeMediaStream([micTrack]),
|
||||
});
|
||||
source.start();
|
||||
|
||||
expect(pc.transceivers[0].sender.track).toBe(micTrack);
|
||||
});
|
||||
});
|
||||
|
||||
describe('negotiation', () => {
|
||||
it('should send an offer after setting the local description', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
await flushPromises();
|
||||
|
||||
expect(pc.setLocalDescription).toHaveBeenCalled();
|
||||
expect(channel.sent).toContainEqual({
|
||||
type: 'webrtc/offer',
|
||||
value: 'v=0\r\noffer',
|
||||
});
|
||||
});
|
||||
|
||||
it('should send an empty offer sdp when the browser omits it', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
pc.createOffer.mockResolvedValue({ type: 'offer', sdp: undefined });
|
||||
source.start();
|
||||
await flushPromises();
|
||||
|
||||
expect(channel.sent).toContainEqual({ type: 'webrtc/offer', value: '' });
|
||||
});
|
||||
|
||||
it('should not send an offer if the connection was replaced mid-negotiation', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
let resolveOffer: (value: { type: string; sdp: string }) => void = () => {};
|
||||
pc.createOffer.mockReturnValue(
|
||||
new Promise((resolve) => {
|
||||
resolveOffer = resolve;
|
||||
}),
|
||||
);
|
||||
source.start();
|
||||
source.stop();
|
||||
resolveOffer({ type: 'offer', sdp: 'v=0\r\noffer' });
|
||||
await flushPromises();
|
||||
|
||||
expect(channel.sent).not.toContainEqual(
|
||||
expect.objectContaining({ type: 'webrtc/offer' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('should not send an offer if replaced between offer and local description', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
let resolveLocal: () => void = () => {};
|
||||
pc.setLocalDescription.mockReturnValue(
|
||||
new Promise((resolve) => {
|
||||
resolveLocal = resolve;
|
||||
}),
|
||||
);
|
||||
source.start();
|
||||
await flushPromises();
|
||||
source.stop();
|
||||
resolveLocal();
|
||||
await flushPromises();
|
||||
|
||||
expect(channel.sent).not.toContainEqual(
|
||||
expect.objectContaining({ type: 'webrtc/offer' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('should fail on a negotiation error for the current connection', async () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
pc.createOffer.mockRejectedValue(new Error('offer failed'));
|
||||
source.start();
|
||||
await flushPromises();
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('media_error');
|
||||
});
|
||||
|
||||
it('should ignore a negotiation error after the connection was replaced', async () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
let rejectOffer: (reason: Error) => void = () => {};
|
||||
pc.createOffer.mockReturnValue(
|
||||
new Promise((_resolve, reject) => {
|
||||
rejectOffer = reject;
|
||||
}),
|
||||
);
|
||||
source.start();
|
||||
source.stop();
|
||||
rejectOffer(new Error('offer failed'));
|
||||
await flushPromises();
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('signaling', () => {
|
||||
it('should send ICE candidates to the server', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
pc.fireIceCandidate('candidate:1 1 udp 2 1.2.3.4 5 typ host');
|
||||
|
||||
expect(channel.sent).toContainEqual({
|
||||
type: 'webrtc/candidate',
|
||||
value: 'candidate:1 1 udp 2 1.2.3.4 5 typ host',
|
||||
});
|
||||
});
|
||||
|
||||
it('should send an empty candidate at end-of-candidates', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
pc.fireIceCandidate(null);
|
||||
|
||||
expect(channel.sent).toContainEqual({ type: 'webrtc/candidate', value: '' });
|
||||
});
|
||||
|
||||
it('should not send a candidate after stop', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
|
||||
const sentBefore = channel.sent.length;
|
||||
pc.fireIceCandidate('candidate:1 1 udp 2 1.2.3.4 5 typ host');
|
||||
|
||||
expect(channel.sent).toHaveLength(sentBefore);
|
||||
});
|
||||
|
||||
it('should apply the server answer', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'webrtc/answer', value: 'v=0\r\nanswer' });
|
||||
|
||||
expect(pc.setRemoteDescription).toHaveBeenCalledWith({
|
||||
type: 'answer',
|
||||
sdp: 'v=0\r\nanswer',
|
||||
});
|
||||
});
|
||||
|
||||
it('should swallow a rejected setRemoteDescription', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
pc.setRemoteDescription.mockRejectedValue(new Error('bad answer'));
|
||||
source.start();
|
||||
|
||||
expect(() =>
|
||||
channel.receiveMessage({ type: 'webrtc/answer', value: 'v=0\r\nanswer' }),
|
||||
).not.toThrow();
|
||||
await flushPromises();
|
||||
});
|
||||
|
||||
it('should swallow a rejected addIceCandidate', async () => {
|
||||
const { source, channel, pc } = setup();
|
||||
pc.addIceCandidate.mockRejectedValue(new Error('bad candidate'));
|
||||
source.start();
|
||||
|
||||
expect(() =>
|
||||
channel.receiveMessage({
|
||||
type: 'webrtc/candidate',
|
||||
value: 'candidate:1 1 udp 2 1.2.3.4 5 typ host',
|
||||
}),
|
||||
).not.toThrow();
|
||||
await flushPromises();
|
||||
});
|
||||
|
||||
it('should add server ICE candidates with a fixed sdpMid', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({
|
||||
type: 'webrtc/candidate',
|
||||
value: 'candidate:2 1 udp 1 5.6.7.8 9 typ host',
|
||||
});
|
||||
|
||||
expect(pc.addIceCandidate).toHaveBeenCalledWith({
|
||||
candidate: 'candidate:2 1 udp 1 5.6.7.8 9 typ host',
|
||||
sdpMid: '0',
|
||||
});
|
||||
});
|
||||
|
||||
it('should ignore an empty server ICE candidate', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'webrtc/candidate', value: '' });
|
||||
|
||||
expect(pc.addIceCandidate).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should ignore messages with a non-string value', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'webrtc/answer', value: 42 });
|
||||
|
||||
expect(pc.setRemoteDescription).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should ignore messages after the connection was replaced', () => {
|
||||
const { source, channel, pc } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
channel.receiveMessage({ type: 'webrtc/answer', value: 'v=0\r\nanswer' });
|
||||
|
||||
expect(pc.setRemoteDescription).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('server errors', () => {
|
||||
it('should fail on a webrtc server error', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'webrtc/offer: stream not found' });
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('server_error');
|
||||
});
|
||||
|
||||
it('should ignore server errors for other modes', () => {
|
||||
const { source, channel, failedCallback } = setup();
|
||||
source.start();
|
||||
channel.receiveMessage({ type: 'error', value: 'mse: stream not found' });
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('connection', () => {
|
||||
it('should attach the received stream and report loaded', () => {
|
||||
const { source, pc, video, loadedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(video.srcObject).not.toBeNull();
|
||||
|
||||
video.dispatchEvent(new Event('loadeddata'));
|
||||
expect(loadedCallback).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should only build the received stream once', () => {
|
||||
const { source, pc, video } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
const firstStream = video.srcObject;
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(video.srcObject).toBe(firstStream);
|
||||
});
|
||||
|
||||
it('should fail when the connection fails', () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('failed');
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('media_error');
|
||||
});
|
||||
|
||||
it('should not fail on a recoverable disconnect', () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('disconnected');
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should fail when a disconnect escalates to failed', () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('disconnected');
|
||||
pc.fireConnectionStateChange('failed');
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('media_error');
|
||||
});
|
||||
|
||||
it('should ignore intermediate connection states', () => {
|
||||
const { source, pc, video, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connecting');
|
||||
|
||||
expect(video.srcObject).toBeFalsy();
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should ignore connection state changes after stop', () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
source.stop();
|
||||
pc.fireConnectionStateChange('failed');
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should build the media stream with the global MediaStream by default', () => {
|
||||
vi.stubGlobal('MediaStream', FakeMediaStream);
|
||||
const video = document.createElement('video');
|
||||
const channel = new FakeStreamSourceChannel();
|
||||
const pc = new FakeRTCPeerConnection();
|
||||
const source = new WebRTCStreamSource(
|
||||
{
|
||||
target: { kind: 'video', video },
|
||||
channel,
|
||||
callbacks: { loadedCallback: vi.fn(), failedCallback: vi.fn() },
|
||||
},
|
||||
{ createPeerConnection: () => pc.asPeerConnection() },
|
||||
);
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(source.getMediaStream()).toBeInstanceOf(FakeMediaStream);
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
});
|
||||
|
||||
describe('connect timeout', () => {
|
||||
it('should fail if no frame decodes within the connect timeout', () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
// Connected, but the real video never fires loadeddata.
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('connect_timeout');
|
||||
});
|
||||
|
||||
it('should cancel the connect timeout once loaded', () => {
|
||||
const { source, pc, video, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
video.dispatchEvent(new Event('loadeddata'));
|
||||
vi.advanceTimersByTime(5 * 1000);
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalledWith('connect_timeout');
|
||||
});
|
||||
});
|
||||
|
||||
describe('lifecycle', () => {
|
||||
it('should close the connection on stop and clear the video', () => {
|
||||
const { source, pc, video } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
source.stop();
|
||||
|
||||
expect(pc.close).toHaveBeenCalled();
|
||||
expect(video.srcObject).toBeNull();
|
||||
expect(source.getMediaStream()).toBeNull();
|
||||
});
|
||||
|
||||
it('should tolerate stopping before starting', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(() => source.stop()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('reporting', () => {
|
||||
it('should expose the media stream and peer connection', () => {
|
||||
const { source, pc } = setup();
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(source.getMediaStream()).not.toBeNull();
|
||||
expect(source.getPeerConnection()).toBe(pc.asPeerConnection());
|
||||
});
|
||||
|
||||
it('should report webrtc technology', () => {
|
||||
const { source } = setup();
|
||||
|
||||
expect(source.getTechnology()).toEqual(['webrtc']);
|
||||
});
|
||||
|
||||
it('should report a stream profile from the tracks and SDP', () => {
|
||||
const { source, pc } = setup();
|
||||
source.start();
|
||||
pc.setRemoteDescription({ sdp: 'a=rtpmap:98 H265/90000\r\n' });
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(source.getStreamProfile()).toEqual({
|
||||
hasVideo: true,
|
||||
hasH265Video: true,
|
||||
hasAudio: true,
|
||||
hasAACAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report an empty profile before connection', () => {
|
||||
const { source } = setup();
|
||||
source.start();
|
||||
|
||||
expect(source.getStreamProfile()).toEqual({
|
||||
hasVideo: false,
|
||||
hasH265Video: false,
|
||||
hasAudio: false,
|
||||
hasAACAudio: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('should report 2-way audio capability once a mic track is armed', () => {
|
||||
const micTrack = new FakeMediaStreamTrack('audio');
|
||||
const { source, pc } = setup({
|
||||
microphoneStream: new FakeMediaStream([micTrack]),
|
||||
});
|
||||
source.start();
|
||||
pc.fireConnectionStateChange('connected');
|
||||
|
||||
expect(source.getCapabilities().has2WayAudio).toBe(true);
|
||||
expect(source.getCapabilities().supportsPause).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('setMicrophoneStream', () => {
|
||||
it('should do nothing for an unchanged stream', async () => {
|
||||
const stream = new FakeMediaStream([new FakeMediaStreamTrack('audio')]);
|
||||
const { source, pc } = setup({ microphoneStream: stream });
|
||||
source.start();
|
||||
await source.setMicrophoneStream(stream.asMediaStream());
|
||||
|
||||
expect(pc.getMicrophoneTransceiver().sender.replaceTrack).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should replace the outbound track without renegotiating', async () => {
|
||||
const { source, pc } = setup();
|
||||
source.start();
|
||||
const newTrack = new FakeMediaStreamTrack('audio');
|
||||
await source.setMicrophoneStream(new FakeMediaStream([newTrack]).asMediaStream());
|
||||
|
||||
expect(pc.getMicrophoneTransceiver().sender.replaceTrack).toHaveBeenCalledWith(
|
||||
newTrack,
|
||||
);
|
||||
});
|
||||
|
||||
it('should clear the outbound track for a null stream', async () => {
|
||||
const stream = new FakeMediaStream([new FakeMediaStreamTrack('audio')]);
|
||||
const { source, pc } = setup({ microphoneStream: stream });
|
||||
source.start();
|
||||
await source.setMicrophoneStream(null);
|
||||
|
||||
expect(pc.getMicrophoneTransceiver().sender.replaceTrack).toHaveBeenCalledWith(
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
it('should do nothing before there is a peer connection', async () => {
|
||||
const { source, failedCallback } = setup();
|
||||
await source.setMicrophoneStream(
|
||||
new FakeMediaStream([new FakeMediaStreamTrack('audio')]).asMediaStream(),
|
||||
);
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should fail when a current replaceTrack rejects', async () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
pc.getMicrophoneTransceiver().sender.replaceTrack.mockRejectedValue(
|
||||
new Error('replace failed'),
|
||||
);
|
||||
await source.setMicrophoneStream(
|
||||
new FakeMediaStream([new FakeMediaStreamTrack('audio')]).asMediaStream(),
|
||||
);
|
||||
|
||||
expect(failedCallback).toHaveBeenCalledWith('two_way_audio_error');
|
||||
});
|
||||
|
||||
it('should ignore a stale replaceTrack rejection after stop', async () => {
|
||||
const { source, pc, failedCallback } = setup();
|
||||
source.start();
|
||||
let rejectReplace: (reason: Error) => void = () => {};
|
||||
pc.getMicrophoneTransceiver().sender.replaceTrack.mockReturnValue(
|
||||
new Promise((_resolve, reject) => {
|
||||
rejectReplace = reject;
|
||||
}),
|
||||
);
|
||||
const promise = source.setMicrophoneStream(
|
||||
new FakeMediaStream([new FakeMediaStreamTrack('audio')]).asMediaStream(),
|
||||
);
|
||||
source.stop();
|
||||
rejectReplace(new Error('replace failed'));
|
||||
await promise;
|
||||
|
||||
expect(failedCallback).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,270 @@
|
||||
import { vi, type Mock } from 'vitest';
|
||||
|
||||
import type {
|
||||
MediaSourceFactory,
|
||||
MediaSourceInterface,
|
||||
} from '../../../../../src/components-lib/live/providers/go2rtc-experimental/adapters/media-source';
|
||||
import type {
|
||||
BinaryCallback,
|
||||
Go2RTCMessage,
|
||||
MessageCallback,
|
||||
StreamSourceChannel,
|
||||
} from '../../../../../src/components-lib/live/providers/go2rtc-experimental/types';
|
||||
import type { UnsubscribeCallback } from '../../../../../src/types';
|
||||
|
||||
// ===========================================================================
|
||||
// User agents.
|
||||
// ===========================================================================
|
||||
|
||||
export const CHROME_USER_AGENT =
|
||||
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' +
|
||||
'(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36';
|
||||
|
||||
export const SAFARI_17_USER_AGENT =
|
||||
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 ' +
|
||||
'(KHTML, like Gecko) Version/17.4 Safari/605.1.15';
|
||||
|
||||
// ===========================================================================
|
||||
// Fakes for browser APIs jsdom does not provide.
|
||||
// ===========================================================================
|
||||
|
||||
export class FakeWebSocket extends EventTarget {
|
||||
public binaryType = '';
|
||||
public sent: string[] = [];
|
||||
|
||||
public close = vi.fn();
|
||||
public send = vi.fn((data: string): void => {
|
||||
this.sent.push(data);
|
||||
});
|
||||
|
||||
public asWebSocket(): WebSocket {
|
||||
return this as unknown as WebSocket;
|
||||
}
|
||||
|
||||
public fireOpen(): void {
|
||||
this.dispatchEvent(new Event('open'));
|
||||
}
|
||||
|
||||
public fireClose(): void {
|
||||
this.dispatchEvent(new Event('close'));
|
||||
}
|
||||
|
||||
public fireMessage(data: unknown): void {
|
||||
this.dispatchEvent(new MessageEvent('message', { data }));
|
||||
}
|
||||
}
|
||||
|
||||
export const createTimeRanges = (ranges: [number, number][]): TimeRanges => ({
|
||||
length: ranges.length,
|
||||
start: (index: number) => ranges[index][0],
|
||||
end: (index: number) => ranges[index][1],
|
||||
});
|
||||
|
||||
class FakeSourceBuffer extends EventTarget {
|
||||
public mode = '';
|
||||
public updating = false;
|
||||
public buffered: TimeRanges = createTimeRanges([]);
|
||||
|
||||
public appendBuffer = vi.fn();
|
||||
public remove = vi.fn();
|
||||
|
||||
public asSourceBuffer(): SourceBuffer {
|
||||
return this as unknown as SourceBuffer;
|
||||
}
|
||||
|
||||
public fireUpdateEnd(): void {
|
||||
this.dispatchEvent(new Event('updateend'));
|
||||
}
|
||||
}
|
||||
|
||||
export class FakeMediaStreamTrack extends EventTarget {
|
||||
public muted = false;
|
||||
public kind: string;
|
||||
|
||||
constructor(kind: string) {
|
||||
super();
|
||||
this.kind = kind;
|
||||
}
|
||||
|
||||
public asTrack(): MediaStreamTrack {
|
||||
return this as unknown as MediaStreamTrack;
|
||||
}
|
||||
|
||||
public setMuted(muted: boolean): void {
|
||||
this.muted = muted;
|
||||
this.dispatchEvent(new Event(muted ? 'mute' : 'unmute'));
|
||||
}
|
||||
}
|
||||
|
||||
export class FakeMediaStream {
|
||||
private _tracks: FakeMediaStreamTrack[];
|
||||
|
||||
constructor(tracks: FakeMediaStreamTrack[] = []) {
|
||||
this._tracks = tracks;
|
||||
}
|
||||
|
||||
public getTracks(): FakeMediaStreamTrack[] {
|
||||
return this._tracks;
|
||||
}
|
||||
|
||||
public getVideoTracks(): FakeMediaStreamTrack[] {
|
||||
return this._tracks.filter((track) => track.kind === 'video');
|
||||
}
|
||||
|
||||
public getAudioTracks(): FakeMediaStreamTrack[] {
|
||||
return this._tracks.filter((track) => track.kind === 'audio');
|
||||
}
|
||||
|
||||
public asMediaStream(): MediaStream {
|
||||
return this as unknown as MediaStream;
|
||||
}
|
||||
}
|
||||
|
||||
class FakeRTCTransceiver {
|
||||
public direction: string;
|
||||
public currentDirection: string;
|
||||
public sender: {
|
||||
track: FakeMediaStreamTrack | null;
|
||||
replaceTrack: Mock<[MediaStreamTrack | null], Promise<void>>;
|
||||
};
|
||||
public receiver: { track: FakeMediaStreamTrack };
|
||||
|
||||
constructor(direction: string, kind: string, track: FakeMediaStreamTrack | null) {
|
||||
this.direction = direction;
|
||||
this.currentDirection = direction;
|
||||
this.sender = {
|
||||
track,
|
||||
replaceTrack: vi.fn<[MediaStreamTrack | null], Promise<void>>(() =>
|
||||
Promise.resolve(),
|
||||
),
|
||||
};
|
||||
this.receiver = { track: new FakeMediaStreamTrack(kind) };
|
||||
}
|
||||
}
|
||||
|
||||
export class FakeRTCPeerConnection extends EventTarget {
|
||||
public connectionState: RTCPeerConnectionState = 'new';
|
||||
public remoteDescription: { sdp: string } | null = null;
|
||||
public transceivers: FakeRTCTransceiver[] = [];
|
||||
|
||||
public createOffer = vi.fn(
|
||||
(): Promise<{ type: string; sdp?: string }> =>
|
||||
Promise.resolve({ type: 'offer', sdp: 'v=0\r\noffer' }),
|
||||
);
|
||||
public setLocalDescription = vi.fn(() => Promise.resolve());
|
||||
public setRemoteDescription = vi.fn((description: { sdp: string }) => {
|
||||
this.remoteDescription = description;
|
||||
return Promise.resolve();
|
||||
});
|
||||
public addIceCandidate = vi.fn(() => Promise.resolve());
|
||||
public close = vi.fn();
|
||||
|
||||
public addTransceiver(
|
||||
trackOrKind: FakeMediaStreamTrack | string,
|
||||
init: { direction: string },
|
||||
): FakeRTCTransceiver {
|
||||
const kind = typeof trackOrKind === 'string' ? trackOrKind : trackOrKind.kind;
|
||||
const track = typeof trackOrKind === 'string' ? null : trackOrKind;
|
||||
const transceiver = new FakeRTCTransceiver(init.direction, kind, track);
|
||||
this.transceivers.push(transceiver);
|
||||
return transceiver;
|
||||
}
|
||||
|
||||
public getTransceivers(): FakeRTCTransceiver[] {
|
||||
return this.transceivers;
|
||||
}
|
||||
|
||||
public getReceivers(): { track: FakeMediaStreamTrack }[] {
|
||||
return this.transceivers.map((transceiver) => transceiver.receiver);
|
||||
}
|
||||
|
||||
public asPeerConnection(): RTCPeerConnection {
|
||||
return this as unknown as RTCPeerConnection;
|
||||
}
|
||||
|
||||
public getMicrophoneTransceiver(): FakeRTCTransceiver {
|
||||
return this.transceivers[0];
|
||||
}
|
||||
|
||||
public fireConnectionStateChange(state: RTCPeerConnectionState): void {
|
||||
this.connectionState = state;
|
||||
this.dispatchEvent(new Event('connectionstatechange'));
|
||||
}
|
||||
|
||||
public fireIceCandidate(candidate: string | null): void {
|
||||
const event = new Event('icecandidate');
|
||||
Object.assign(event, {
|
||||
candidate: candidate === null ? null : { candidate },
|
||||
});
|
||||
this.dispatchEvent(event);
|
||||
}
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Fakes for custom interfaces.
|
||||
// ===========================================================================
|
||||
|
||||
export class FakeStreamSourceChannel implements StreamSourceChannel {
|
||||
public sent: Go2RTCMessage[] = [];
|
||||
public binaryCallback: BinaryCallback | null = null;
|
||||
|
||||
private _messageCallbacks = new Set<MessageCallback>();
|
||||
|
||||
public send(message: Go2RTCMessage): void {
|
||||
this.sent.push(message);
|
||||
}
|
||||
|
||||
public subscribeToMessages(callback: MessageCallback): UnsubscribeCallback {
|
||||
this._messageCallbacks.add(callback);
|
||||
|
||||
return () => {
|
||||
this._messageCallbacks.delete(callback);
|
||||
};
|
||||
}
|
||||
|
||||
public setBinaryCallback(callback: BinaryCallback | null): void {
|
||||
this.binaryCallback = callback;
|
||||
}
|
||||
|
||||
public receiveMessage(message: Go2RTCMessage): void {
|
||||
[...this._messageCallbacks].forEach((callback) => callback(message));
|
||||
}
|
||||
|
||||
public getMessageCallbackCount(): number {
|
||||
return this._messageCallbacks.size;
|
||||
}
|
||||
}
|
||||
|
||||
export class FakeMediaSourceInstance implements MediaSourceInterface {
|
||||
public sourceBuffer = new FakeSourceBuffer();
|
||||
|
||||
public attach = vi.fn();
|
||||
public detach = vi.fn();
|
||||
public setLiveSeekableRange = vi.fn();
|
||||
public isOpen = vi.fn<[], boolean>(() => true);
|
||||
public isTypeSupported = vi.fn<[string], boolean>(() => true);
|
||||
public addSourceBuffer = vi.fn<[string], SourceBuffer>(() =>
|
||||
this.sourceBuffer.asSourceBuffer(),
|
||||
);
|
||||
|
||||
private _sourceOpenCallbacks = new Set<() => void>();
|
||||
|
||||
public subscribeToSourceOpen(callback: () => void): UnsubscribeCallback {
|
||||
this._sourceOpenCallbacks.add(callback);
|
||||
return () => {
|
||||
this._sourceOpenCallbacks.delete(callback);
|
||||
};
|
||||
}
|
||||
|
||||
public fireSourceOpen(): void {
|
||||
[...this._sourceOpenCallbacks].forEach((callback) => callback());
|
||||
}
|
||||
|
||||
public getSourceOpenCallbackCount(): number {
|
||||
return this._sourceOpenCallbacks.size;
|
||||
}
|
||||
}
|
||||
|
||||
export const createFakeMediaSourceFactory = (
|
||||
instance: FakeMediaSourceInstance | null,
|
||||
): MediaSourceFactory => vi.fn(() => instance);
|
||||
@@ -0,0 +1,15 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { arrayBufferToBase64 } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/utils/base64';
|
||||
|
||||
// @vitest-environment jsdom
|
||||
describe('arrayBufferToBase64', () => {
|
||||
it('should base64-encode the bytes', () => {
|
||||
const bytes = new TextEncoder().encode('Hi');
|
||||
expect(arrayBufferToBase64(bytes.buffer)).toBe('SGk=');
|
||||
});
|
||||
|
||||
it('should encode an empty buffer as an empty string', () => {
|
||||
expect(arrayBufferToBase64(new ArrayBuffer(0))).toBe('');
|
||||
});
|
||||
});
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { BoundedBufferQueue } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/utils/bounded-buffer-queue';
|
||||
|
||||
describe('BoundedBufferQueue', () => {
|
||||
it('should start empty', () => {
|
||||
expect(new BoundedBufferQueue(10).isEmpty).toBe(true);
|
||||
});
|
||||
|
||||
it('should accept a chunk within the byte cap', () => {
|
||||
const queue = new BoundedBufferQueue(10);
|
||||
expect(queue.push(new ArrayBuffer(4))).toBe(true);
|
||||
expect(queue.isEmpty).toBe(false);
|
||||
});
|
||||
|
||||
it('should accept a chunk that fills the cap exactly', () => {
|
||||
expect(new BoundedBufferQueue(10).push(new ArrayBuffer(10))).toBe(true);
|
||||
});
|
||||
|
||||
it('should reject a chunk that would exceed the byte cap and stage nothing', () => {
|
||||
const queue = new BoundedBufferQueue(10);
|
||||
expect(queue.push(new ArrayBuffer(8))).toBe(true);
|
||||
expect(queue.push(new ArrayBuffer(3))).toBe(false);
|
||||
|
||||
// The rejected chunk left the byte total unchanged, so a smaller one fits.
|
||||
expect(queue.push(new ArrayBuffer(2))).toBe(true);
|
||||
});
|
||||
|
||||
it('should return staged chunks oldest first', () => {
|
||||
const queue = new BoundedBufferQueue(10);
|
||||
const first = new ArrayBuffer(2);
|
||||
const second = new ArrayBuffer(3);
|
||||
queue.push(first);
|
||||
queue.push(second);
|
||||
|
||||
expect(queue.shift()).toBe(first);
|
||||
expect(queue.shift()).toBe(second);
|
||||
expect(queue.isEmpty).toBe(true);
|
||||
});
|
||||
|
||||
it('should return null when shifting an empty queue', () => {
|
||||
expect(new BoundedBufferQueue(10).shift()).toBeNull();
|
||||
});
|
||||
|
||||
it('should free the shifted chunk bytes back toward the cap', () => {
|
||||
const queue = new BoundedBufferQueue(10);
|
||||
queue.push(new ArrayBuffer(8));
|
||||
expect(queue.push(new ArrayBuffer(4))).toBe(false);
|
||||
|
||||
queue.shift();
|
||||
expect(queue.push(new ArrayBuffer(4))).toBe(true);
|
||||
});
|
||||
|
||||
it('should reset the chunks and the byte total on clear', () => {
|
||||
const queue = new BoundedBufferQueue(10);
|
||||
queue.push(new ArrayBuffer(8));
|
||||
queue.clear();
|
||||
|
||||
expect(queue.isEmpty).toBe(true);
|
||||
|
||||
// The byte total reset too, so the full cap is available again.
|
||||
expect(queue.push(new ArrayBuffer(10))).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,102 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import {
|
||||
convertToCodecString,
|
||||
getCodecsForUserAgent,
|
||||
GO2RTC_CODECS,
|
||||
selectSupportedCodecs,
|
||||
} from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/utils/codecs';
|
||||
import { CHROME_USER_AGENT, SAFARI_17_USER_AGENT } from '../test-utils';
|
||||
|
||||
const safariUserAgent = (version: number): string =>
|
||||
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 ' +
|
||||
`(KHTML, like Gecko) Version/${version}.0 Safari/605.1.15`;
|
||||
|
||||
describe('getCodecsForUserAgent', () => {
|
||||
it('should return all codecs for non-Safari browsers', () => {
|
||||
expect(getCodecsForUserAgent(CHROME_USER_AGENT)).toEqual([...GO2RTC_CODECS]);
|
||||
});
|
||||
|
||||
it('should exclude AAC and later for Safari before version 13', () => {
|
||||
expect(getCodecsForUserAgent(safariUserAgent(12))).toEqual([
|
||||
'avc1.640029',
|
||||
'avc1.64002A',
|
||||
'avc1.640033',
|
||||
'hvc1.1.6.L153.B0',
|
||||
]);
|
||||
});
|
||||
|
||||
it('should exclude FLAC and later for Safari before version 14', () => {
|
||||
expect(getCodecsForUserAgent(safariUserAgent(13))).toEqual([
|
||||
'avc1.640029',
|
||||
'avc1.64002A',
|
||||
'avc1.640033',
|
||||
'hvc1.1.6.L153.B0',
|
||||
'mp4a.40.2',
|
||||
'mp4a.40.5',
|
||||
]);
|
||||
});
|
||||
|
||||
it('should exclude OPUS for modern Safari', () => {
|
||||
expect(getCodecsForUserAgent(SAFARI_17_USER_AGENT)).toEqual([
|
||||
'avc1.640029',
|
||||
'avc1.64002A',
|
||||
'avc1.640033',
|
||||
'hvc1.1.6.L153.B0',
|
||||
'mp4a.40.2',
|
||||
'mp4a.40.5',
|
||||
'flac',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('selectSupportedCodecs', () => {
|
||||
it('should include all supported codecs for video and audio', () => {
|
||||
expect(
|
||||
selectSupportedCodecs(GO2RTC_CODECS, { audio: true, video: true }, () => true),
|
||||
).toEqual([
|
||||
'avc1.640029',
|
||||
'avc1.64002A',
|
||||
'avc1.640033',
|
||||
'hvc1.1.6.L153.B0',
|
||||
'mp4a.40.2',
|
||||
'mp4a.40.5',
|
||||
'flac',
|
||||
'opus',
|
||||
]);
|
||||
});
|
||||
|
||||
it('should exclude audio codecs when audio is not requested', () => {
|
||||
expect(
|
||||
selectSupportedCodecs(GO2RTC_CODECS, { audio: false, video: true }, () => true),
|
||||
).toEqual(['avc1.640029', 'avc1.64002A', 'avc1.640033', 'hvc1.1.6.L153.B0']);
|
||||
});
|
||||
|
||||
it('should exclude video codecs when video is not requested', () => {
|
||||
expect(
|
||||
selectSupportedCodecs(GO2RTC_CODECS, { audio: true, video: false }, () => true),
|
||||
).toEqual(['mp4a.40.2', 'mp4a.40.5', 'flac', 'opus']);
|
||||
});
|
||||
|
||||
it('should exclude codecs the support callback rejects', () => {
|
||||
expect(
|
||||
selectSupportedCodecs(
|
||||
GO2RTC_CODECS,
|
||||
{ audio: true, video: true },
|
||||
(mimeType) => mimeType === 'video/mp4; codecs="avc1.640029"',
|
||||
),
|
||||
).toEqual(['avc1.640029']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('convertToCodecString', () => {
|
||||
it('should join codecs with commas', () => {
|
||||
expect(convertToCodecString(['avc1.640029', 'mp4a.40.2'])).toBe(
|
||||
'avc1.640029,mp4a.40.2',
|
||||
);
|
||||
});
|
||||
|
||||
it('should return an empty string for no codecs', () => {
|
||||
expect(convertToCodecString([])).toBe('');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,21 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { mapFailureReasonToIssueReason } from '../../../../../../src/components-lib/live/providers/go2rtc-experimental/utils/failure-reason';
|
||||
|
||||
describe('mapFailureReasonToIssueReason', () => {
|
||||
it.each([
|
||||
['connect_timeout', 'not_loading'],
|
||||
['negotiation_timeout', 'not_loading'],
|
||||
['media_error', 'playback_error'],
|
||||
['buffer_overflow', 'playback_error'],
|
||||
['two_way_audio_error', 'two_way_audio_error'],
|
||||
['server_error', 'server_error'],
|
||||
['unsupported', 'unsupported'],
|
||||
] as const)('should map %s to the %s cause', (reason, expected) => {
|
||||
expect(mapFailureReasonToIssueReason(reason)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should map a null reason to a generic playback error', () => {
|
||||
expect(mapFailureReasonToIssueReason(null)).toBe('playback_error');
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user