你如何只将某些 CMSampleBuffer 帧写入 AVFoundation 中的新电影文件？答案

【问题标题】：How do you write only certain CMSampleBuffer frames to a new movie file in AVFoundation?你如何只将某些 CMSampleBuffer 帧写入 AVFoundation 中的新电影文件？
【发布时间】：2021-06-13 13:16:39
【问题描述】：

我想对用户导入的视频执行姿势分析，自动生成AVFoundation 视频输出，其中只有带有a detected pose 的帧是结果的一部分。在Building a Feature-Rich App for Sports Analysis 示例代码中，通过实现func cameraViewController(_ controller: CameraViewController, didReceiveBuffer buffer: CMSampleBuffer, orientation: CGImagePropertyOrientation) 委托回调进行分析，例如GameViewController.swift 的第326 行。

我遇到的困难是使用此分析仅保留检测到姿势的特定帧。假设我已经分析了所有CMSampleBuffer 帧并分类了哪些帧具有我想要的姿势。我如何只为新的视频输出那些特定的帧？

Apple 的文档存档中的“AVFoundation Programming Guide”链接已损坏，因此很难找到概念性解释。首选是 Swift 中的答案，尽管 Objective-C 仍然会有所帮助。

【问题讨论】：

标签： ios swift avfoundation coreml

【解决方案1】：

如果要记录CMSampleBuffer，可以使用这段代码：

import Foundation
import AVFoundation

class MovieRecorder {
    
    private var assetWriter: AVAssetWriter?
    
    private var assetWriterVideoInput: AVAssetWriterInput?
    
    private var assetWriterAudioInput: AVAssetWriterInput?
    
    private var videoTransform: CGAffineTransform
    
    private var videoSettings: [String: Any]

    private var audioSettings: [String: Any]

    private(set) var isRecording = false
    
    init(audioSettings: [String: Any], videoSettings: [String: Any], videoTransform: CGAffineTransform) {
        self.audioSettings = audioSettings
        self.videoSettings = videoSettings
        self.videoTransform = videoTransform
    }
    
    func startRecording() {
        // Create an asset writer that records to a temporary file
        let outputFileName = NSUUID().uuidString
        let outputFileURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(outputFileName).appendingPathExtension("MOV")
        guard let assetWriter = try? AVAssetWriter(url: outputFileURL, fileType: .mov) else {
            return
        }
        
        // Add an audio input
        let assetWriterAudioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioSettings)
        assetWriterAudioInput.expectsMediaDataInRealTime = true
        assetWriter.add(assetWriterAudioInput)
        
        // Add a video input
        let assetWriterVideoInput = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings)
        assetWriterVideoInput.expectsMediaDataInRealTime = true
        assetWriterVideoInput.transform = videoTransform
        assetWriter.add(assetWriterVideoInput)
        
        self.assetWriter = assetWriter
        self.assetWriterAudioInput = assetWriterAudioInput
        self.assetWriterVideoInput = assetWriterVideoInput
        
        isRecording = true
    }
    
    func stopRecording(completion: @escaping (URL) -> Void) {
        guard let assetWriter = assetWriter else {
            return
        }
        
        self.isRecording = false
        self.assetWriter = nil
        
        assetWriter.finishWriting {
            completion(assetWriter.outputURL)
        }
    }
    
    func recordVideo(sampleBuffer: CMSampleBuffer) {
        guard isRecording,
            let assetWriter = assetWriter else {
                return
        }
        
        if assetWriter.status == .unknown {
            assetWriter.startWriting()
            assetWriter.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(sampleBuffer))
        } else if assetWriter.status == .writing {
            if let input = assetWriterVideoInput,
                input.isReadyForMoreMediaData {
                input.append(sampleBuffer)
            }
        }
    }
    
    func recordAudio(sampleBuffer: CMSampleBuffer) {
        guard isRecording,
            let assetWriter = assetWriter,
            assetWriter.status == .writing,
            let input = assetWriterAudioInput,
            input.isReadyForMoreMediaData else {
                return
        }
        
        input.append(sampleBuffer)
    }
}

【讨论】：