Skip to content

Add docstrings explaining the role of capture/render processing delegates more clearly #548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions Sources/LiveKit/Protocols/AudioCustomProcessingDelegate.swift
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,24 @@ internal import LiveKitWebRTC

public let kLiveKitKrispAudioProcessorName = "livekit_krisp_noise_cancellation"

/// Used to modify audio buffers before they are sent to the network or played to the user
@objc
public protocol AudioCustomProcessingDelegate {
/// An optional identifier for the audio processor implementation.
/// This can be used to identify different types of audio processing (e.g. noise cancellation).
/// Generally you can leave this as the default value.
@objc optional
var audioProcessingName: String { get }

/// Provides the sample rate and number of channels to configure your delegate for processing
@objc
func audioProcessingInitialize(sampleRate sampleRateHz: Int, channels: Int)

/// Provides a chunk of audio data that can be modified in place
@objc
func audioProcessingProcess(audioBuffer: LKAudioBuffer)

/// Called when the audio processing is no longer needed so it may clean up any resources
@objc
func audioProcessingRelease()
}
Expand Down
3 changes: 2 additions & 1 deletion Sources/LiveKit/Protocols/AudioRenderer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ internal import LiveKitWebRTC
@_implementationOnly import LiveKitWebRTC
#endif

/// Used to observe audio buffers before playback, e.g. for visualization, recording, etc
/// - Note: AudioRenderer is not suitable for buffer modification. If you need to modify the buffer, use `AudioCustomProcessingDelegate` instead.
@objc
public protocol AudioRenderer {
@objc
Expand All @@ -38,7 +40,6 @@ class AudioRendererAdapter: MulticastDelegate<AudioRenderer>, LKRTCAudioRenderer
}

// MARK: - LKRTCAudioRenderer

func render(pcmBuffer: AVAudioPCMBuffer) {
notify { $0.render(pcmBuffer: pcmBuffer) }
}
Expand Down
7 changes: 7 additions & 0 deletions Sources/LiveKit/Track/AudioManager.swift
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,9 @@ public class AudioManager: Loggable {

let capturePostProcessingDelegateSubject = CurrentValueSubject<AudioCustomProcessingDelegate?, Never>(nil)

/// Add a delegate to modify the local audio buffer before it is sent to the network
/// - Note: Only one delegate can be set at a time, but you can create one to wrap others if needed
/// - Note: If you only need to observe the buffer (rather than modify it), use ``add(localAudioRenderer:)`` instead
public var capturePostProcessingDelegate: AudioCustomProcessingDelegate? {
get { capturePostProcessingDelegateAdapter.target }
set {
Expand All @@ -177,6 +180,10 @@ public class AudioManager: Loggable {
}
}

/// Add a delegate to modify the combined remote audio buffer (all tracks) before it is played to the user
/// - Note: Only one delegate can be set at a time, but you can create one to wrap others if needed
/// - Note: If you only need to observe the buffer (rather than modify it), use ``add(remoteAudioRenderer:)`` instead
/// - Note: If you need to observe the buffer for individual tracks, use ``RemoteAudioTrack/add(audioRenderer:)`` instead
public var renderPreProcessingDelegate: AudioCustomProcessingDelegate? {
get { renderPreProcessingDelegateAdapter.target }
set { renderPreProcessingDelegateAdapter.set(target: newValue) }
Expand Down