如何使用 AVCaptureVideoDataOutput 在 AVCaptureSession 中避免丢帧答案

【问题标题】：How to avoid dropped frames in AVCaptureSession with AVCaptureVideoDataOutput如何使用 AVCaptureVideoDataOutput 在 AVCaptureSession 中避免丢帧
【发布时间】：2018-07-12 21:26:25
【问题描述】：

我在 Xamarin.iOS 项目中设置 AVCaptureSession，无论我在设备上设置的帧速率如何，我都会收到很高的丢帧率。当我剥离会话以使用后置摄像头和仅一个输出（AVCaptureVideoDataOutput）时，我的委托的 DidOutputSampleBuffer 方法仅在 2/3 的时间内被调用。即使我将最大和最小帧持续时间设置为 1/3 秒并且基本上将委托方法除递增计数器之外什么都不做，这也是正确的。很抱歉后面的代码墙，但我不知道哪里出了问题。

捕获会话设置：

AVCaptureDeviceType[] deviceTypes = new AVCaptureDeviceType[] {
AVCaptureDeviceType.BuiltInTrueDepthCamera,
AVCaptureDeviceType.BuiltInDualCamera,
AVCaptureDeviceType.BuiltInWideAngleCamera};
AVCaptureDeviceDiscoverySession discoverySession =
    AVCaptureDeviceDiscoverySession.Create(deviceTypes,
        AVMediaType.Video, AVCaptureDevicePosition.Back);
AVCaptureDevice[] devices = discoverySession.Devices;
AVCaptureDevice device = devices.FirstOrDefault();
if (device == null)
{
    return false;
}

previewView.VideoPreviewLayer.Session = null;
m_CaptureSession?.Dispose();
m_CaptureSession = new AVCaptureSession();

NSError error;
AVCaptureDeviceInput input = new AVCaptureDeviceInput(device, out error);
if (error == null)
{
    if (m_CaptureSession.CanAddInput(input))
    {
        m_CaptureSession.AddInput(input);
    }

    // note: will try for 120fps on my iPhone 5S, but I've
    // also overridden the settings with a hard-coded 3fps
    // and get the same results (dropping a large percentage
    // of frames)
    ConfigureCameraForHighestFrameRate(device);

    AVCaptureVideoDataOutput output = new AVCaptureVideoDataOutput();
    AVVideoSettingsUncompressed settings =
        new AVVideoSettingsUncompressed();
    settings.PixelFormatType = CVPixelFormatType.CV32BGRA;
    output.UncompressedVideoSetting = settings;

    m_OutputQueue = new DispatchQueue("outputQueue", false);
    m_OutputRecorder = new OutputRecorder(device);
    output.SetSampleBufferDelegateQueue(m_OutputRecorder, m_OutputQueue);
    if (m_CaptureSession.CanAddOutput(output))
    {
        m_CaptureSession.AddOutput(output);
    }

    // don't hook up preview view, so we can test the video data
    // output on its own...
    // previewView.VideoPreviewLayer.Session = m_CaptureSession;

    m_CaptureSession.StartRunning();
}

委托方法：

public override void DidOutputSampleBuffer(AVCaptureOutput captureOutput,
    CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
{
    lock (m_Mutex)
    {
        if (!m_Recording)
        {
            return;
        }
        m_FrameCount++;
        return;
    }
}

public override void DidDropSampleBuffer(AVCaptureOutput captureOutput,
    CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
{
    lock (m_Mutex)
    {
        if (!m_Recording)
        {
            return;
        }
        m_DroppedFrameCount++;
    }
}

有什么明显的我做错了吗？

【问题讨论】：

标签： c# ios xamarin avcapturesession

【解决方案1】：

所以，问题显然在于，处理 sampleBuffer 是代表的工作，而当我对录制时生成的帧执行此操作时，我不是在录制开始之前捕获的帧。解决方案是确保 sampleBuffer 得到处理，无论应用程序是否正在录制。这可能是 Xamarin 特有的问题和解决方案；我不知道它是否对 Swift 开发者有用。

【讨论】：