使用 WebRTC 通过 ReplayKit 发送 iOS 设备的屏幕截图

Question

我们想通过 WebRTC 使用 ReplayKit 发送 iOS 设备的屏幕截图。 ReplayKit 有一个 processSampleBuffer 回调，它提供 CMSampleBuffer。

但这就是我们卡住的地方，我们似乎无法将 CMSampleBuffer 发送到连接的对等方。 我们尝试从 sampleBuffer 中创建 pixelBuffer，然后创建 RTCVideoFrame。

我们还从 RTCPeerConnectionFactory 中提取了 RTCVideoSource，然后使用 RTCVideoCapturer 并将其流式传输到 localVideoSource。

知道我们做错了什么吗？

var peerConnectionFactory: RTCPeerConnectionFactory?

override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
 switch sampleBufferType {
           case RPSampleBufferType.video:

        // create the CVPixelBuffer
        let pixelBuffer:CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!;

        // create the RTCVideoFrame
        var videoFrame:RTCVideoFrame?;
        let timestamp = NSDate().timeIntervalSince1970 * 1000
        videoFrame = RTCVideoFrame(pixelBuffer: pixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timestamp))

        // connect the video frames to the WebRTC
        let localVideoSource = self.peerConnectionFactory!.videoSource()
        let videoCapturer = RTCVideoCapturer()
        localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

        let videoTrack : RTCVideoTrack =   self.peerConnectionFactory!.videoTrack(with: localVideoSource, trackId: "100”)

        let mediaStream: RTCMediaStream = (self.peerConnectionFactory?.mediaStream(withStreamId: “1"))!
        mediaStream.addVideoTrack(videoTrack)
        self.newPeerConnection!.add(mediaStream)

        break
    }
}

Answer 1

这是一个实现的好主意，您只需在 sn-p 中使用的方法中渲染 RTCVideoFrame，所有其他对象将初始化超大方法，这是最好的方法.为了更好地理解，我给你一个sn-p。

    var peerConnectionFactory: RTCPeerConnectionFactory?
    var localVideoSource: RTCVideoSource?
    var videoCapturer: RTCVideoCapturer?
    func setupVideoCapturer(){
          // localVideoSource and videoCapturer will use 
            localVideoSource = self.peerConnectionFactory!.videoSource() 
            videoCapturer = RTCVideoCapturer()
    //      localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

            let videoTrack : RTCVideoTrack =   self.peerConnectionFactory!.videoTrack(with: localVideoSource, trackId: "100”)

            let mediaStream: RTCMediaStream = (self.peerConnectionFactory?.mediaStream(withStreamId: “1"))!
            mediaStream.addVideoTrack(videoTrack)
            self.newPeerConnection!.add(mediaStream)
        }


 override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
     switch sampleBufferType {
               case RPSampleBufferType.video:

            // create the CVPixelBuffer
            let pixelBuffer:CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!;

            // create the RTCVideoFrame
            var videoFrame:RTCVideoFrame?;
            let timestamp = NSDate().timeIntervalSince1970 * 1000
            videoFrame = RTCVideoFrame(pixelBuffer: pixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timestamp))
            // connect the video frames to the WebRTC
            localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

            break
        }
    }

希望这会对你有所帮助。