Metal renderMode in VideoView looks very pixelated on smaller VideoViews

[patryk-sredzinski]

Describe the bug
If you draw a camera feed into a small VideoView, it looks really bad when using metal renderMode.
sampleBuffer looks way better.

SDK Version
2.3.1

iOS/macOS Version
iOS 18.4

Steps to Reproduce

import UIKit
import LiveKit

class ViewController: UIViewController {

    let metalView: VideoView = {
        let view = VideoView()
        view.backgroundColor = .red
        view.translatesAutoresizingMaskIntoConstraints = false
        view.renderMode = .metal
        return view
    }()
    
    let saBufView: VideoView = {
        let view = VideoView()
        view.backgroundColor = .blue
        view.translatesAutoresizingMaskIntoConstraints = false
        view.renderMode = .sampleBuffer
        return view
    }()
    
    lazy var track = LocalVideoTrack.createCameraTrack()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        
        view.addSubview(metalView)
        view.addSubview(saBufView)
        
        NSLayoutConstraint.activate([
            metalView.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 16),
            metalView.trailingAnchor.constraint(equalTo: view.centerXAnchor, constant: -16),
            saBufView.leadingAnchor.constraint(equalTo: view.centerXAnchor, constant: 16),
            saBufView.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -16),
            metalView.centerYAnchor.constraint(equalTo: view.centerYAnchor),
            saBufView.centerYAnchor.constraint(equalTo: view.centerYAnchor),
            metalView.heightAnchor.constraint(equalToConstant: 100),
            saBufView.heightAnchor.constraint(equalToConstant: 100)
        ])
        
        Task {
            do {
                try await track.start()
                metalView.track = track
                saBufView.track = track
            } catch {
                print(error)
            }
            
        }
    }
}

Screenshots
Above code runs and results like that:
Metal on the left, CMSampleBuffer on the right

Screenshot 1

Screenshot 2

Solution
Seems like there is a problem with sampling pixels onto a small layer.

---

Edit
Solved the issue by doing:

Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [self] _ in
    let lkrtCMTLVideoView = metalView.subviews[0]
    let mtkView = lkrtCMTLVideoView.subviews[0]
    mtkView.contentScaleFactor = UIScreen.main.scale
}

Seems like changing scale factor for MTKView solves the issue.
Would be nice to have it exposed in the SDK.

It must be in a loop, as somehow MTKView changes back the scale to 1

[hiroshihorie]

Thank you @patryk-sredzinski, I patched WebRTC in this branch : #630
Can you try it out ?

[patryk-sredzinski]

@hiroshihorie #630 works perfectly, thanks