如何从相机中抓取 YUV 格式的视频，显示并处理它

Question

我正在编写一个 iphone (IOS 4) 程序，该程序从摄像头捕获实时视频并实时处理。

我更喜欢以 kCVPixelFormatType_420YpCbCr8BiPlanarFullRange 格式进行捕获，以便于处理（我需要处理 Y 通道）。如何以这种格式显示数据？我想我需要以某种方式将其转换为 UIImage，然后将其放入某个 ImageView 中？

目前我有显示 kCVPixelFormatType_32BGRA 数据的代码，但它自然不适用于 kCVPixelFormatType_420YpCbCr8BiPlanarFullRange。

这是我现在用于转换的代码，任何有关如何对 kCVPixelFormatType_420YpCbCr8BiPlanarFullRange 执行相同操作的帮助/示例将不胜感激。 （也批评我目前的方法）。

// Create a UIImage from sample buffer data
- (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer 
{
    // Get a CMSampleBuffer's Core Video image buffer for the media data
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 
    // Lock the base address of the pixel buffer
    CVPixelBufferLockBaseAddress(imageBuffer, 0); 

    // Get the number of bytes per row for the pixel buffer
    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); 

    // Get the number of bytes per row for the pixel buffer
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
    // Get the pixel buffer width and height
    size_t width = CVPixelBufferGetWidth(imageBuffer); 
    size_t height = CVPixelBufferGetHeight(imageBuffer); 

    // Create a device-dependent RGB color space
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 

    // Create a bitmap graphics context with the sample buffer data
    CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, 
                                                 bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
    // Create a Quartz image from the pixel data in the bitmap graphics context
    CGImageRef quartzImage = CGBitmapContextCreateImage(context);

    // Unlock the pixel buffer
    CVPixelBufferUnlockBaseAddress(imageBuffer,0);

    // Free up the context and color space
    CGContextRelease(context); 
    CGColorSpaceRelease(colorSpace);

    // Create an image object from the Quartz image
    UIImage *image = [UIImage imageWithCGImage:quartzImage];

    // Release the Quartz image
    CGImageRelease(quartzImage);

    return (image);
}

Answer 1

回答我自己的问题。 这解决了我遇到的问题（即获取 yuv 输出，显示并处理它），尽管它不完全是问题的答案：

从相机获取 YUV 输出：

AVCaptureVideoDataOutput *videoOut = [[AVCaptureVideoDataOutput alloc] init];
[videoOut setAlwaysDiscardsLateVideoFrames:YES];
[videoOut setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];

要按原样显示，请使用 AVCaptureVideoPreviewLayer，它不需要太多代码。 （例如，您可以在 WWDC 示例包中查看 FindMyiCon 示例）。

要处理 YUV y 通道（在这种情况下是双平面，所以它都在一个块中，您也可以使用 memcpy 而不是循环）：

- (void)processPixelBuffer: (CVImageBufferRef)pixelBuffer {

    CVPixelBufferLockBaseAddress( pixelBuffer, 0 );

    int bufferHeight = CVPixelBufferGetHeight(pixelBuffer);
    int bufferWidth = CVPixelBufferGetWidth(pixelBuffer);

    // allocate space for ychannel, reallocating as needed.
    if (bufferWidth != y_channel.width || bufferHeight != y_channel.height)
    {
        if (y_channel.data) free(y_channel.data);
        y_channel.width = bufferWidth;
        y_channel.height = bufferHeight;
        y_channel.data = malloc(y_channel.width * y_channel.height);        
    }

    uint8_t *yc = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);
    int total = bufferWidth * bufferHeight;
    for(int k=0;k<total;k++)
    {
        y_channel.data[k] = yc[k++]; // copy y channel
    }

    CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );
}