像 CIColorMatrix 一样，使用 vImageMatrixMultiply_ARGB8888 将颜色矩阵过滤器应用于 UIImage

Question

我正在尝试使用 Accelerate 框架的 vImageMatrixMultiply_ARGB8888 构建一个快速函数以将颜色矩阵过滤器应用于图像。 我以前能够使用 CIFilter CIColorMatrix 将颜色矩阵应用于图像。

     let ciimage = CIImage(image: image)!
     let colorMatrix = CIFilter(name: "CIColorMatrix")!
     colorMatrix.setDefaults()
     colorMatrix.setValue(ciimage, forKey: "inputImage")
     colorMatrix.setValue(CIVector(x: 1.9692307692307693, y: 0, z: 0, w: 0), forKey: "inputRVector")
     colorMatrix.setValue(CIVector(x: 0, y: 2.226086956521739, z: 0, w: 0), forKey: "inputGVector")
     colorMatrix.setValue(CIVector(x: 0, y: 0, z: 2.585858585858586, w: 0), forKey: "inputBVector")
     colorMatrix.setValue(CIVector(x: 0, y: 0, z: 0, w: 1), forKey: "inputAVector")
     return UIImage(ciImage: colorMatrix.outputImage!)

vImageMatrixMultiply_ARGB8888 中使用的

RotatioMatrix 使用 CIFilter 的输入向量的相同值。

rotationMatrix = [1.9692307692307693, 0, 0, 0
                  0, 2.226086956521739, 0, 0,
                  0, 0, 2.585858585858586, 0,
                  0, 0, 0, 1].map {
                     return Int16(Float($0) * Float(divisor)) // divisor = 256
                  }

以下代码不会产生与上面 CIFilter 相同的结果。

private func rgbAdjustmentWithMatrix(image sourceImage: UIImage, rotationMatrix: [Int16]) -> UIImage? {

    guard
        let cgImage = sourceImage.cgImage,
        let sourceImageFormat = vImage_CGImageFormat(cgImage: cgImage),
        let rgbDestinationImageFormat = vImage_CGImageFormat(
            bitsPerComponent: 8,
            bitsPerPixel: 32,
            colorSpace: CGColorSpaceCreateDeviceRGB(),
            bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.first.rawValue),
            renderingIntent: .defaultIntent) else {
                print("Unable to initialize cgImage or colorSpace.")
                return nil
    }

    guard
        let sourceBuffer = try? vImage_Buffer(cgImage: cgImage),
        var rgbDestinationBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width),
            height: Int(sourceBuffer.height),
            bitsPerPixel: rgbDestinationImageFormat.bitsPerPixel) else {
                print("Unable to initialize source buffer or destination buffer.")
                return nil
    }

    defer {
        sourceBuffer.free()
        rgbDestinationBuffer.free()
    }

    do {
        let toRgbConverter = try vImageConverter.make(sourceFormat: sourceImageFormat, destinationFormat: rgbDestinationImageFormat)
            try toRgbConverter.convert(source: sourceBuffer, destination: &rgbDestinationBuffer)
    } catch {
        print("Unable to initialize converter or unable to convert.")
        return nil
    }

    guard var resultBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width),
            height: Int(sourceBuffer.height),
            bitsPerPixel: rgbDestinationImageFormat.bitsPerPixel) else {
                print("Unable to initialize result buffer.")
                return nil
    }

    defer {
        resultBuffer.free()
    }
    var error: vImage_Error

    let divisor: Int32 = 256

    let preBias = [Int16](repeating: -256, count: 4)
    let postBias = [Int32](repeating: 256 * divisor, count: 4)

    error = vImageMatrixMultiply_ARGB8888(&rgbDestinationBuffer,
                                  &resultBuffer,
                                  rotationMatrix,
                                  divisor,
                                  preBias,
                                  postBias,
                                  0)
    guard error == kvImageNoError else { return nil }

    if let cgImage = try? resultBuffer.createCGImage(format: rgbDestinationImageFormat) {
        return UIImage(cgImage: cgImage)
    } else {
        return nil
    } 

}

可能是色彩空间问题？还是 vImageMatrixMultiply_ARGB8888 输入矩阵与 CIFilter 输入矩阵不同？

Answer 1

更新

因此，看起来 vImageMatrixMultiply_ARGB8888 期望值按以下顺序：

let rotationMatrix = [
    1, 0,                  0,                 0,                 // A
    0, 1.9692307692307693, 0,                 0,                 // R
    0, 0,                  2.226086956521739, 0,                 // G
    0, 0,                  0,                 2.585858585858586  // B
    ].map { return Int16($0 * Float(divisor)) } // divisor = 0x1000

这与暗示 BGRA 的 Apple 示例代码不一致。可能该矩阵最初打算用于不同的 pixelBuffer 格式 - 或其他一些问题。

我相信您遇到的另一个问题是 CIFilter 使用的是默认 CIContext。当我运行它时，我得到了一个线性 SRGB 工作颜色空间。

let context = CIContext()
print(context.workingColorSpace ?? "")

输出：

<CGColorSpace 0x7fe4abd06d30> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB Linear)

因此，您可以将 CGColorSpaceCreateDeviceRGB() 指定为 CIFilter 的 CIContext workingColorSpace，或者在 vImage 使用中将 rgbDestinationImageFormat colorSpace 设置为 .linearSRGB。在这个和修正后的矩阵顺序之间，我相信你的输出图像现在​​会匹配。

// error handling removed for brevity
let ciImage = CIImage(cgImage: cgImage)
let colorMatrix = CIFilter(name: "CIColorMatrix")!
colorMatrix.setDefaults()
colorMatrix.setValue(ciImage, forKey: "inputImage")
colorMatrix.setValue(CIVector(x: 1.9692307692307693, y: 0, z: 0, w: 0), forKey: "inputRVector")
colorMatrix.setValue(CIVector(x: 0, y: 2.226086956521739, z: 0, w: 0), forKey: "inputGVector")
colorMatrix.setValue(CIVector(x: 0, y: 0, z: 2.585858585858586, w: 0), forKey: "inputBVector")
colorMatrix.setValue(CIVector(x: 0, y: 0, z: 0, w: 1), forKey: "inputAVector")
let context = CIContext(options: [.workingColorSpace : CGColorSpaceCreateDeviceRGB()]) // without this, colorSpace defaults to linearSRGB
return context.createCGImage(colorMatrix.outputImage!, from: rect)! 

在这种情况下，除非您有其他原因，否则可能不需要前后偏差。

最后，您可能会受益于更大的除数。每个通道的像素数据为 8 位，累加器为 32 位，矩阵值为 16 位。大部分参考代码使用0x1000（4096）的除数。

func rgbAdjustmentWithMatrix(image sourceImage: NSImage, rotationMatrix: [Int16]) -> CGImage?  {

    var rect = CGRect(x: 0, y: 0, width: sourceImage.size.width, height: sourceImage.size.height)

    guard
        let cgImage = sourceImage.cgImage(forProposedRect: &rect, context: nil, hints: nil),
        let sourceImageFormat = vImage_CGImageFormat(cgImage: cgImage),
        let rgbDestinationImageFormat = vImage_CGImageFormat(
            bitsPerComponent: 8,
            bitsPerPixel: 32,
            colorSpace: CGColorSpaceCreateDeviceRGB(),
            // can match CIFilter default with this:
            // colorSpace: CGColorSpace(name: CGColorSpace.linearSRGB)!,
            bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.first.rawValue),
            renderingIntent: .defaultIntent)
        else {
                print("Unable to initialize cgImage or colorSpace.")
                return nil
    }

    guard
        var sourceBuffer = try? vImage_Buffer(cgImage: cgImage),
        var rgbDestinationBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width),
                                                      height: Int(sourceBuffer.height),
                                                    bitsPerPixel: rgbDestinationImageFormat.bitsPerPixel)
        else {
            fatalError("Error initializing source and destination buffers.")
    }

    defer {
        sourceBuffer.free()
        rgbDestinationBuffer.free()
    }

    do {
        let toRgbConverter = try vImageConverter.make(sourceFormat: sourceImageFormat, destinationFormat: rgbDestinationImageFormat)
        try toRgbConverter.convert(source: sourceBuffer, destination: &rgbDestinationBuffer)
    } catch {
        fatalError(error.localizedDescription)
    }

    let divisor: Int32 = 0x1000 // matrix values are 16 bit and accumulator is 32 bit
                                // 4096 gives us decent overhead for the matrix operation
                                // 12-bit color

    let preBias: [Int16] = [0, 0, 0, 0]  // or simply pass nil
    let postBias: [Int32] = [0, 0, 0, 0] // or simply pass nil

    let error = vImageMatrixMultiply_ARGB8888(&rgbDestinationBuffer,
                                              &rgbDestinationBuffer,
                                              rotationMatrix,
                                              divisor,
                                              preBias,
                                              postBias,
                                              vImage_Flags(kvImageNoFlags))

    if error != kvImageNoError { print("Error: \(error)") }

    let result = try? rgbDestinationBuffer.createCGImage(format: rgbDestinationImageFormat)
    return result

}

上一个答案

查看您的代码和输出图像，我相信您的红色和蓝色系数颠倒了。如果是这样，您将像这样填充 vImage 矩阵：

rotationMatrix = [2.585858585858586, 0, 0, 0   // b
                  0, 2.226086956521739, 0, 0,  // g
                  0, 0, 1.9692307692307693, 0, // r
                  0, 0, 0, 1].map {            // a
                     return Int16(Float($0) * Float(divisor)) // divisor = 256
                  }

这与 Apple 的 desaturating a selected portion of an image 的 vImage 示例代码一致。

    let desaturationMatrix = [
        0.0722, 0.0722, 0.0722, 0,
        0.7152, 0.7152, 0.7152, 0,
        0.2126, 0.2126, 0.2126, 0,
        0,      0,      0,      1
        ].map {
            return Int16($0 * Float(divisor))
    }

请注意，coefficients for Rec709 到 Luma 的转换为：

Y = R * 0.2126 + G * 0.7152 + B * 0.0722