从给定的掩码逼近一个四边形

Question

目标：

我想估计给定蒙版对象的 4 个坐标 四边形 （不仅是矩形），如图所示 + 不会丢失蒙版的任何像素对象。

试验：

我尝试使用 CV2，但最终无法找到解决方案。

1. cv2.boundingRect：返回边界矩形的坐标（而四边形估计不一定是完美矩形）
2. cv2.findContours + cv2.approxPolyDP：不是那么准确并返回对象的估计极值点（需要更多工作来估计四边形 4 坐标，并且可能有更简单和更快的解决方案）。

代码片段：

尝试 cv2.boundinRect:

#mask = grayed image with only a specific object being masked
#image = the original rgb image
x,y,x_width,y_height = cv2.boundingRect(mask)
image=np.array(im[0])
cv2.rectangle(image,(x,y),(x+x_width,y+y_height),(0,255,0),2)
plt.imshow(image)

尝试 cv2.findContours + cv2.approxPolyDP：

#mask = grayed image with only a specific object being masked
#image = the original rgb image
contours, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
selected_contour = max(contours, key=lambda x: cv2.contourArea(x))
approx = cv2.approxPolyDP(selected_contour, 0.0035 * cv2.arcLength(selected_contour, True), True)
cv2.drawContours(image, [approx], 0, (0, 0, 255), 5)
plt.imshow(image)

Answer 1

我不确定是否有更好的或内置的版本；但我有一个基于随机数的简单想法：

我只对顶部进行了此操作，但您可以对其他侧执行相同操作。这个想法是首先找到对象的边界框；然后将对象分成相等的部分，以便我们可以找到最高峰。

在每个范围内，你可以随机找到点；但为了获得最佳结果，最好检查形状的所有顶点以正确找到最高峰。

找到最高峰后，我们必须针对这两个点计算一个线方程，以便我们可以根据该线方程绘制一条全局线.

import sys
import cv2
import random
import numpy as np
from tqdm import tqdm


def rndPt(l, t, r, b):
    # Generate a random point in given ROI
    return (random.randint(int(l), int(r)), random.randint(int(t), int(b)))


def intArr(arr):
    # Cast each item of 1D array to integer
    return [int(x) for x in arr]


# Load our image
pth = sys.path[0]
org = cv2.imread(pth+'/bound.png')
im = org.copy()
H, W = im.shape[:2]

# Make mask and copy from that image
im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
bw = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY)[1]
im = bw.copy()

# Find the ROI of object
cnts, _ = cv2.findContours(bw, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
cnts.sort(key=lambda x: cv2.boundingRect(x)[0])
ROI = None
for cnt in cnts:
    x, y, w, h = cv2.boundingRect(cnt)
    if w < W-1 and h < H-1:
        cv2.rectangle(bw, (x, y), (x+w, y+h), 127, 2)
        ROI = {'x': x, 'y': y, 'w': w, 'h': h, 'h2': y+h}

# We have to find the peaks; so we have to
# divide the bounding-box of shape into several
# ranges.
spaces = 5
sw = ROI['w']//spaces

# Each range can have a peak as a candidate point
candidates = [(ROI['x']+(sw*x)+sw//2, ROI['h']//2) for x in range(0, spaces)]

# Divide the object and find the highest point in
# each range
for s in tqdm(range(0, spaces)):
    l = ROI['x']+(sw*s)
    cv2.line(im, pt1=(l, ROI['y']), pt2=(l, ROI['h2']),
             color=127, thickness=2)
    for x in range(0, sw):
        for i in range(0, 200):
            pt = rndPt(l, ROI['y'], l+sw, ROI['h2']//4)
            if pt[1] < candidates[s][1] and bw[pt[1], pt[0]] == 0:
                candidates[s] = pt
l = ROI['x']+(sw*spaces)
cv2.line(im, pt1=(l, ROI['y']), pt2=(l, ROI['h2']), color=127, thickness=2)
print(candidates)

# We remove duplicate points and also sort the points
# according to the peak
candidates = list(set(candidates))
candidates.sort(key=lambda p: p[1])
print(candidates)
c = candidates

# Now that we have found two of the highest points, we can
# write a line equation for these two points
xA, xB = ROI['x'], ROI['x']+ROI['w']
x1, y1 = c[0][0], c[0][1]
x2, y2 = c[1][0], c[1][1]
m = (y2-y1)/(x2-x1)
# y=mx+b -> y-mx=b
b = y1-m*x1
yA = m*xA+b
yB = m*xB+b

# Convert images to BGR
im = cv2.cvtColor(im, cv2.COLOR_GRAY2BGR)
bw = cv2.cvtColor(bw, cv2.COLOR_GRAY2BGR)

# Make a copy of image to draw candidate points
marker = im.copy()
for p in candidates:
    cv2.circle(marker, (p[0],p[1]), 
        h//25, color=(50, 100, 200),thickness=4)

# Draw lines
cv2.line(im, pt1=intArr((xA, yA)), pt2=intArr((xB, yB)),
         color=(255, 0, 100), thickness=4, lineType=cv2.LINE_AA)
cv2.line(bw, pt1=intArr(c[0]), pt2=intArr(c[1]),
         color=(100, 0, 255), thickness=4, lineType=cv2.LINE_AA)

# Save final output
top = np.hstack((org, marker))
btm = np.hstack((bw, im))
cv2.imwrite(pth+'/out.png', np.vstack((top, btm)))