2.29.0-preview.1

2.29.0-preview.1 (August 13, 2024)

Changes

• AddProcessorOptions.inputFrameBufferType now has a new value videoframe. On browsers that support VideoFrame, the inputFrameBuffer argument of VideoProcessor.processFrame() will be a VideoFrame. On other supported browsers, it will be an HTMLVideoElement.
• AddProcessorOptions.outputFrameBufferContextType now has a new value bitmaprenderer. Currently, this is only supported for Chromium-based browsers. On other supported browsers, it falls back to 2d.
• Patched the build script to work around the issue: markdown-it/linkify-it#111.

2.28.1

2.28.1 (October 3, 2023)

Bug Fixes

• Previously, a Chrome iOS 17 Participant's local audio (Krisp noise cancellation enabled) did not recover after foregrounding the browser following the playing of a YouTube video (or some other application which requires microphone permissions). We work around this by permanently disabling the Krisp noise cancellation upon foregrounding the browser. (VIDEO-13006)

2.28.0

2.28.0 (September 14, 2023)

Bug Fixes

• Fixed a bug where a Chrome iOS 17 Participant's local and remote media (Krisp Noise Cancellation enabled) failed to recover after interacting with a PSTN call in full-screen mode. (VIDEO-13011)
• Fixed a bug where a Chrome iOS 17 Participant's audio with Krisp Noise Cancellation did not recover after interacting with a system popup. (VIDEO-13012)
• Fixed a bug where a Chrome iOS 17 Participant's audio with Krisp Noise Cancellation did not recover after invoking Siri. (VIDEO-13013)

2.27.0

2.27.0 (March 21, 2023)

Changes

VideoTrack.addProcessor now works on browsers that support OffscreenCanvas as well as HTMLCanvasElement. When used with
@twilio/video-processors v2.0.0, the Virtual Background feature will work on browsers that supports WebGL2. See VideoTrack.addProcessor and @twilio/video-processors v2.0.0 for details.

Example

import { createLocalVideoTrack } from 'twilio-video';
import { Pipeline, VirtualBackgroundProcessor } from '@twilio/video-processors';

const virtualBackgroundProcessor = new VirtualBackgroundProcessor({
  pipeline: Pipeline.WebGL2,
  // ...otherOptions
});

await virtualBackgroundProcessor.loadModel();

const videoTrack = await createLocalVideoTrack({
  width: 640,
  height: 480,
  frameRate: 24
});

videoTrack.addProcessor(processor, {
  inputFrameBufferType: 'video',
  outputFrameBufferContextType: 'webgl2',
});

2.26.2

2.26.2 (February 21, 2023)

Changes

• Starting from version 110, Chrome will no longer support the iSAC audio codec. The SDK will now log a warning to the console whenever an audio or a video codec that is specified in ConnectOptions.preferredAudioCodecs and ConnectOptions.preferredVideoCodecs is not supported by the browser. (VIDEO-12494)

Bug Fixes

• Fixed a bug on Chrome versions 112+ where Room.getStats() did not reject the returned Promise when an exception was raised while accessing WebRTC stats that due to a TypeError caused by trying to read from the now-removed RTCMediaStreamTrackStats. (VIDEO-12534)

2.26.1

2.26.1 (January 31, 2023)

Bug Fixes

• Fixed a bug that manifests on Chrome versions 112+ where Room.getStats() rejects the returned Promise due to a TypeError caused by trying to read from the now-removed RTCMediaStreamTrackStats. Instead, the SDK now reads from the RTCMediaSourceStats. (VIDEO-12411)
• Fixed an error in the type definition for the attach() method of AudioTrack and VideoTrack. (VIDEO-12242)
• Fixed an error in the type definition for createLocalAudioTrack(). (VIDEO-12383)

2.26.0

2.26.0 (December 14, 2022)

New Features

• The LocalAudioTrack and
  LocalVideoTrack classes now provide a
  new boolean property called isMuted, which lets you know if the audio or video source is currently providing raw media
  samples. The classes also emit muted and unmuted events if the value of isMuted toggles. The application can use
  these APIs to detect temporary loss of microphone or camera to other applications (ex: an incoming phone call on an iOS device),
  and update the user interface accordingly. (VIDEO-11360)

• The Room class provides a new method called refreshInactiveMedia,
  which restarts any muted local media Tracks, and plays any inadvertently paused HTMLMediaElements that are attached to
  local and remote media Tracks. This is useful especially on iOS devices, where sometimes your application's media may
  not recover after an incoming phone call. You can use this method in conjunction with the local media Track's isMuted
  property described previously to recover local and remote media after an incoming phone call as shown below. (VIDEO-11360)

  Vanilla JS

  html

  <button id="refresh-inactive-media" disabled>Refresh Inactive Media</button>

  js

  const { connect } = require('twilio-video');
  
  const room = await connect('token', { name: 'my-cool-room' });
  
  const $refreshInactiveMedia = document.getElementById('refresh-inactive-media');
  $refreshInactiveMedia.onclick = () => room.refreshInactiveMedia();
  
  const [{ track: localAudioTrack }] = [...room.localParticipant.audioTracks.values()];
  const [{ track: localVideoTrack }] = [...room.localParticipant.videoTracks.values()];
  
  const isLocalAudioOrVideoMuted = () => {
    return localAudioTrack.isMuted || localVideoTrack.isMuted;
  }
  
  const onLocalMediaMutedChanged = () => {
    $refreshInactiveMedia.disabled = !isLocalAudioOrVideoMuted();
  };
  
  [localAudioTrack, localVideoTrack].forEach(localMediaTrack => {
    ['muted', 'unmuted'].forEach(event => {
      localMediaTrack.on(event, onLocalMediaMutedChanged);
    });
  });

  React

  src/hooks/useLocalMediaMuted.js

  import { useEffect, useState } from 'react';
  
  export default function useLocalMediaMuted(localMediaTrack) {
    const [isMuted, setIsMuted] = useState(localMediaTrack?.isMuted ?? false);
  
    useEffect(() => {
      const updateMuted = () => setIsMuted(localMediaTrack?.isMuted ?? false);
      updateMuted();
  
      localMediaTrack?.on('muted', updateMuted);
      localMediaTrack?.on('unmuted', updateMuted);
  
      return () => {
        localMediaTrack?.off('muted', updateMuted);
        localMediaTrack?.off('unmuted', updateMuted);
      };
    }, [localMediaTrack]);
  
    return isMuted;
  }

  src/components/room.js

  import useLocalMediaMuted from '../hooks/useLocalMediaMuted';
  
  export default function Room({ room }) {
    const [{ track: localAudioTrack }] = [...room.localParticipant.audioTracks.values()];
    const [{ track: localVideoTrack }] = [...room.localParticipant.videoTracks.values()];
  
    const isLocalAudioMuted = useLocalMediaMuted(localAudioTrack);
    const isLocalVideoMuted = useLocalMediaMuted(localVideoTrack);
    const isLocalMediaMuted = isLocalAudioMuted || isLocalVideoMuted;
  
    const refreshInactiveMedia = () => {
      room.refreshInactiveMedia();
    };
  
    return (
      <>
        ...
        {isLocalMediaMuted && <Button onClick={refreshInactiveMedia}>
          Refresh Inactive Media
        </Button>}
        ...
      </>
    );
  }

2.25.0

2.25.0 (November 14, 2022)

New Features

Auto-switch default audio input devices

This release adds a new feature that preserves audio continuity in situations where end-users change the default audio input device.
A LocalAudioTrack is said to be capturing audio from the default audio input device if:

• it was created using the MediaTrackConstraints { audio: true }, or
• it was created using the MediaTrackConstraints { audio: { deviceId: 'foo' } }, and "foo" is not available, or
• it was created using the MediaTrackConstraints { audio: { deviceId: { ideal: 'foo' } } } and "foo" is not available

In previous versions of the SDK, if the default device changed (ex: a bluetooth headset is connected to a mac or windows laptop),
the LocalAudioTrack continued to capture audio from the old default device (ex: the laptop microphone). Now, a LocalAudioTrack
will switch automatically from the old default audio input device to the new default audio input device (ex: from the laptop microphone to the headset microphone).
This feature is controlled by a new CreateLocalAudioTrackOptions
property defaultDeviceCaptureMode, which defaults to auto (new behavior) or can be set to manual (old behavior).

The application can decide to capture audio from a specific audio input device by creating a LocalAudioTrack:

• using the MediaTrackConstraints { audio: { deviceId: 'foo' } }, and "foo" is available, or
• using the MediaTrackConstraints { audio: { deviceId: { ideal: 'foo' } } } and "foo" is available, or
• using the MediaTrackConstraints { audio: { deviceId: { exact: 'foo' } } } and "foo" is available

In this case, the LocalAudioTrack DOES NOT switch to another audio input device if the current audio input device is no
longer available. See below for the behavior of this property based on how the LocalAudioTrack is created. (VIDEO-11701)

const { connect, createLocalAudioTrack, createLocalTracks } = require('twilio-video');

// Auto-switch default audio input devices: option 1
const audioTrack = await createLocalAudioTrack();

// Auto-switch default audio input devices: option 2
const audioTrack1 = await createLocalAudioTrack({ defaultDeviceCaptureMode: 'auto' });

// Auto-switch default audio input devices: option 3
const [audioTrack3] = await createLocalTracks({ audio: true });

// Auto-switch default audio input devices: option 4
const [audioTrack4] = await createLocalTracks({ audio: { defaultDeviceCaptureMode: 'auto' } });

// Auto-switch default audio input devices: option 5
const room1 = await connect({ audio: true });

// Auto-switch default audio input devices: option 6
const room2 = await connect({ audio: { defaultDeviceCaptureMode: 'auto' } });

// Disable auto-switch default audio input devices
const room = await createLocalAudioTrack({ defaultDeviceCaptureMode: 'manual' });

Limitations

• This feature is not enabled on iOS as it is natively supported.
• Due to this WebKit bug, MacOS Safari Participants may lose their local audio after switching between default audio input devices two-three times.
• This feature is not supported on Android Chrome, as it does not support the MediaDevices.ondevicechange event.

2.24.3

2.24.3 (October 10, 2022)

Bug Fixes

• Fixed a bug where iOS Safari Participant could not hear or see others after switching back from YouTube picture-in-picture mode. (VIDEO-11352)
• Fixed a bug where iOS Safari Participant could not hear others after switching from recording an audio message in a messenger app. (VIDEO-11354)
• Fixed a bug where iOS Safari Participant could not hear or see others after watching a video in another browser tab. (VIDEO-11356)
• Fixed a bug where iOS Safari Participant sometimes could not hear or see others after finishing an incoming call in full screen mode. (VIDEO-11359)

2.24.2

2.24.2 (September 29, 2022)

Bug Fixes

• Fixed a bug where sometimes, a MediaClientRemoteDescFailedError was raised when a Chrome Participant who had enabled Adaptive Simulcast (ConnectOptions.preferredVideoCodecs = 'auto') tried to publish a camera Track after publishing a <canvas> Track. (VIDEO-11516)
• Fixed an issue where the Krisp Noise Cancellation fails to load in an application where the content security policy directives default-src self unsafe-eval are used. (VIDEO-11537)