frame rate - How can I fix the choppy video output in AVKIT? - Stack Overflow

Intro

The capture video device supports 1920x1080, { 60.00 30.00 25.00 20.00 10.00 fps }. So I first set the device to the proper active format (1920x1080) and min/max frame duration (60fps).

• Video is capped at 25 FPS, stream is smooth.
• If I set the video output videoMaxFrameDuration to be the same as the device (60fps). Video is capped at 30 FPS, stream is smooth.
• If i set the video output videoMinFrameDuration to be the same as the device (60fps). Video is capped at the desired 60 FPS, stream is choppy.

I don't get why if the device is outputting at 60 FPS and the video output does the same it does not deliver a smooth stream and why is the actual measured FPS 30 when I don't set videoMinFrameDuration? It feels like I actually get 30 FPS and try to force it to 60 FPS giving it the choppy stream. So why does it do this and how can this be fixed?

Weirdly enough when I choose 1920x1080 @ 10 fps, func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) will also fire at 30 FPS. I would suggest it would do so at 10 FPS now. So something feels off.

Code

videoOutput.videoSettings = [ kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA ]
videoOutput.alwaysDiscardsLateVideoFrames = true
---

guard let device = self.selectedDevice else { return }
guard let input = try? AVCaptureDeviceInput(device: device) else { return }

do {
    try device.lockForConfiguration()

    device.activeFormat = format
    device.activeVideoMinFrameDuration = frameRateRange.minFrameDuration
    device.activeVideoMaxFrameDuration = frameRateRange.maxFrameDuration
    
    device.unlockForConfiguration()
} catch {
    print("Failed to lock device for configuration: \(error)")
}

captureSession.beginConfiguration()

captureSession.addInput(input)
captureSession.addOutput(videoOutput)

captureSessionmitConfiguration()

if let connection = videoOutput.connection(with: .video) {
    connection.videoMinFrameDuration = device.activeVideoMinFrameDuration
    connection.videoMaxFrameDuration = device.activeVideoMaxFrameDuration
}

captureSession.startRunning()

if let device = selectedDevice {
    print("Active format: \(device.activeFormat)")
    print("Current FPS range: \(device.activeVideoMinFrameDuration) to \(device.activeVideoMaxFrameDuration)")
    
    if let connection = videoOutput.connection(with: .video) {
        print("Connection minFrameDuration: \(connection.videoMinFrameDuration)")
        print("Connection maxFrameDuration: \(connection.videoMaxFrameDuration)")
    }
}

Output

Active format: <AVCaptureDeviceFormat: 0x600003000780> 'vide'/'420v' 1920x1080, { 60.00 30.00 25.00 20.00 10.00 fps }, supports Portrait Effect, supports Portrait Effect Aperture, supports Studio Lighting, supports Studio Lighting Intensity, supports Reactions, supports Background Replacement
Current FPS range: CMTime(value: 1000000, timescale: 60000240, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0) to CMTime(value: 1000000, timescale: 60000240, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)
Connection minFrameDuration: CMTime(value: 1000000, timescale: 60000240, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)
Connection maxFrameDuration: CMTime(value: 1000000, timescale: 60000240, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)