OpenCV Java：从图像中提取卡片

Question

我正在尝试使用 OpenCV 和 Java 实现一些图像处理，以从图像中提取卡片。

以下是我的方法：

1. 转换为 BGR 图像
2. 转换为灰度图像
3. 应用高斯模糊
4. 应用 Canny 边缘检测
5. 扩张
6. 寻找轮廓
7. 找到最大的轮廓
8. 使用 approxPolyDP 查找最大轮廓的角
9. 沿最大轮廓获取裁剪图像的俯视图

在第 8 步，我遇到了一些问题，因为我没有得到合适的角/顶点。以下示例图片显示了该场景：

原始图像

在边缘检测和膨胀之后。 （如何获得合适的边缘？？这里我有破损的边缘。无法让霍夫变换工作）

找到顶点之后。 （以绿色显示）

以下是代码：

System.loadLibrary( Core.NATIVE_LIBRARY_NAME );

         //load Image
         File input = new File("card4.png");
         BufferedImage image = ImageIO.read(input); 
         byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();

         //put read image to Mat
         mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3); //original Mat
         mat.put(0, 0, data);
         mat_f = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3); //for storing manipulated Mat

         //conversion to grayscale, blurring and edge detection
         Imgproc.cvtColor(mat, mat_f, Imgproc.COLOR_RGB2BGR);
         Imgproc.cvtColor(mat_f, mat_f, Imgproc.COLOR_RGB2GRAY);
         Imgproc.GaussianBlur(mat_f, mat_f, new Size(13,13), 0);             
         Imgproc.Canny(mat_f, mat_f, 300, 600, 5, true);
         Imgproc.dilate(mat_f, mat_f, new Mat(), new Point(-1, -1), 2);
         Imgcodecs.imwrite("D:\\JAVA\\Image_Proc\\CVTest1.jpg",mat_f);

         //finding contours
         List<MatOfPoint> contours = new ArrayList<MatOfPoint>();    
         Mat hierarchy = new Mat();
         Imgproc.findContours(mat_f, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
         double maxArea=0;
         int maxAreaIdx=0;

         //finding largest contour
         for (int idx = 0; idx != contours.size(); ++idx)
         {
               Mat contour = contours.get(idx);
               double contourarea = Imgproc.contourArea(contour);
               if (contourarea > maxArea)
               {
                   maxArea = contourarea;
                   maxAreaIdx = idx;
               }

          }

            //Rect rect = Imgproc.boundingRect(contours.get(maxAreaIdx));
            //Imgproc.rectangle(mat, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height),new Scalar(0,0,255),7);
           // mat = mat.submat(rect.y, rect.y + rect.height, rect.x, rect.x + rect.width);


          //Polygon approximation
          MatOfPoint2f approxCurve = new MatOfPoint2f();
          MatOfPoint2f oriCurve = new MatOfPoint2f(contours.get(maxAreaIdx).toArray());
          Imgproc.approxPolyDP(oriCurve, approxCurve, 6.0, true);

          //drawing red markers at vertices
          Point [] array = approxCurve.toArray();
          for(int i=0; i < array.length;i++) {
             Imgproc.circle(mat, array[i], 2, new Scalar(0, 255, 0), 5);
          }
          Imgcodecs.imwrite("D:\\JAVA\\Image_Proc\\CVTest.jpg",mat);

寻求帮助以获取适当的角顶点... 提前谢谢..

Answer 1

为了使用您的方法存档好的结果，您的卡片必须包含 4 个角。但我更喜欢使用 HoughLine 方法来完成这项任务。

第 1 步：调整图片大小以获得更高性能

第 2 步：边缘检测

• 将图像转换为灰度
• 模糊图像以消除噪音
• 使用 Canny 过滤器进行边缘检测

您可以使用膨胀使白色更大以进行下一步

第 3 步：找到卡片的角

• 查找图像轮廓
• 从轮廓列表中获取最大轮廓
• 获取它的凸包
• 使用 approxPolyDP 简化凸包（这应该给出一个四边形）
• 从现在开始，您可以在恢复比例后绘制轮廓以获得矩形
• 从四边形可以得到 4 个角。
• 查找单应性
• 使用计算出的单应矩阵扭曲输入图像

这里是Java的示例代码

    // STEP 1: Resize input image to img_proc to reduce computation
    double ratio = DOWNSCALE_IMAGE_SIZE / Math.max(frame.width(), frame.height());
    Size downscaledSize = new Size(frame.width() * ratio, frame.height() * ratio);
    Mat dst = new Mat(downscaledSize, frame.type());
    Imgproc.resize(frame, dst, downscaledSize);
    Mat grayImage = new Mat();
    Mat detectedEdges = new Mat();
    // STEP 2: convert to grayscale
    Imgproc.cvtColor(dst, grayImage, Imgproc.COLOR_BGR2GRAY);
    // STEP 3: try to filter text inside document
    Imgproc.medianBlur(grayImage, detectedEdges, 9);
    // STEP 4: Edge detection
    Mat edges = new Mat();
    // Imgproc.erode(edges, edges, new Mat());
    // Imgproc.dilate(edges, edges, new Mat(), new Point(-1, -1), 1); // 1
    // canny detector, with ratio of lower:upper threshold of 3:1
    Imgproc.Canny(detectedEdges, edges, this.threshold.getValue(), this.threshold.getValue() * 3, 3, true);
    // STEP 5: makes the object in white bigger to join nearby lines
    Imgproc.dilate(edges, edges, new Mat(), new Point(-1, -1), 1); // 1
    Image imageToShow = Utils.mat2Image(edges);
    updateImageView(cannyFrame, imageToShow);
    // STEP 6: Compute the contours
    List<MatOfPoint> contours = new ArrayList<>();
    Imgproc.findContours(edges, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
    // STEP 7: Sort the contours by length and only keep the largest one
    MatOfPoint largestContour = getMaxContour(contours);
    // STEP 8: Generate the convex hull of this contour
    Mat convexHullMask = Mat.zeros(frame.rows(), frame.cols(), frame.type());
    MatOfInt hullInt = new MatOfInt();
    Imgproc.convexHull(largestContour, hullInt);
    MatOfPoint hullPoint = OpenCVUtil.getNewContourFromIndices(largestContour, hullInt);
    // STEP 9: Use approxPolyDP to simplify the convex hull (this should give a quadrilateral)
    MatOfPoint2f polygon = new MatOfPoint2f();
    Imgproc.approxPolyDP(OpenCVUtil.convert(hullPoint), polygon, 20, true);
    List<MatOfPoint> tmp = new ArrayList<>();
    tmp.add(OpenCVUtil.convert(polygon));
    restoreScaleMatOfPoint(tmp, ratio);
    Imgproc.drawContours(convexHullMask, tmp, 0, new Scalar(25, 25, 255), 2);
    // Image extractImageToShow = Utils.mat2Image(convexHullMask);
    // updateImageView(extractFrame, extractImageToShow);
    MatOfPoint2f finalCorners = new MatOfPoint2f();
    Point[] tmpPoints = polygon.toArray();
    for (Point point : tmpPoints) {
        point.x = point.x / ratio;
        point.y = point.y / ratio;
    }
    finalCorners.fromArray(tmpPoints);
    boolean clockwise = true;
    double currentThreshold = this.threshold.getValue();
    if (finalCorners.toArray().length == 4) {
        Size size = getRectangleSize(finalCorners);
        Mat result = Mat.zeros(size, frame.type());
        // STEP 10: Homography: Use findHomography to find the affine transformation of your paper sheet
        Mat homography = new Mat();
        MatOfPoint2f dstPoints = new MatOfPoint2f();
        Point[] arrDstPoints = { new Point(result.cols(), result.rows()), new Point(0, result.rows()), new Point(0, 0), new Point(result.cols(), 0) };
        dstPoints.fromArray(arrDstPoints);
        homography = Calib3d.findHomography(finalCorners, dstPoints);

        // STEP 11: Warp the input image using the computed homography matrix
        Imgproc.warpPerspective(frame, result, homography, size);
    }