如何从表单中删除模板？

Question

给定以下两张图片：

Filled Form

Template

我想从此图像中删除模板，只留下笔迹。我有可以完美对齐这些图像的代码，但我在代码上苦苦挣扎以删除底层模板。

我目前的代码如下：

#Read in images and threshold
image = cv2.imread('image0.png')
template = cv2.imread('image1.png')
(thresh, im_bw) = cv2.threshold(image, 100, 255, cv2.THRESH_BINARY)
(thresh, temp_bw) = cv2.threshold(template, 100, 255, cv2.THRESH_BINARY)


#Convert temp from color to gray
graymask = cv2.cvtColor(temp_bw, cv2.COLOR_BGR2GRAY)

#Increase thickness of lines slightly
kernel = np.ones((2,2),np.uint8)
mask_crop = cv2.erode(graymask, kernel, iterations = 2)

(thresh, blackAndWhitemask) = cv2.threshold(mask_crop, 175, 255, cv2.THRESH_BINARY)
bw = cv2.bitwise_not(blackAndWhitemask)

#Inpaint
dst = cv2.inpaint(im_bw, bw, 3, cv2.INPAINT_NS)

问题是生成的图像 Output 看起来不干净。您可以清楚地看出那里有一个模板。有没有人有任何其他他们会推荐的技术？

Answer 1

差异图像解决了大部分问题，但获得干净的签名具有挑战性。

第一阶段——找到图像和模板的不同之处：

# Read in images and threshold
image = cv2.imread('image0.png', cv2.IMREAD_GRAYSCALE)  # Read image as grayscale
template = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)

diff = (image != template).astype(np.uint8)*255  # Find the difference and convert it to OpenCV mask format (255 where True).
cv2.imwrite('orig_diff.png', diff)

---

小改进：
找出绝对差大于 200 的地方：

thresh, diff = cv2.threshold(cv2.absdiff(image, template), 200, 255, cv2.THRESH_BINARY)

---

为了覆盖小的黑色间隙（假设所有签名都相同），我们可以使用以下步骤：

• 在diff 中查找轮廓 - 每个轮廓都应用一个签名。
• 找到所有签名（轮廓）的边界矩形。
• 迭代每个签名，并为每个签名迭代所有其他签名并放置两个签名中的最大值。
  通过放置两个签名的最大值，填充黑色间隙。
• 使用前一阶段的结果作为掩码。

结果并不完美，因为边界框没有完全对齐，而且原始差异太“粗”。

---

代码示例：

import cv2
import numpy as np

#Read in images and threshold
image = cv2.imread('image0.png', cv2.IMREAD_GRAYSCALE)  # Read image as grayscale
template = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)

#diff = (image != template).astype(np.uint8)*255  # Find the difference and convert it to OpenCV mask format (255 where True).
thresh, diff = cv2.threshold(cv2.absdiff(image, template), 200, 255, cv2.THRESH_BINARY)  # Find where absolute difference is above 200 and convert it to OpenCV mask format (255 where True).
cv2.imwrite('orig_diff.png', diff)

# Dilate diff for getting a gross place of the signatures
dilated_diff = cv2.dilate(diff, np.ones((51, 51), np.uint8))

# Find contours - each contour applies a signatures
cnts = cv2.findContours(dilated_diff, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]

rects = []

# Find bounding rectangles of all the signatures
for c in cnts:
    bounding_rect = cv2.boundingRect(c)
    rects.append(bounding_rect)

# Cover black parts in diff - asuume all the signatures are the same.
# Iterate each signature, and for each signature iterate all other signature and place the maximum of the two signatures
for rect in rects:
    x1, y1, w1, h1 = rect
    for rect in rects:
        x2, y2, w2, h2 = rect
        w3 = min(w1, w2)
        h3 = min(h1, h2)
        roi1 = diff[y1:y1+h3, x1:x1+w3]
        roi2 = diff[y2:y2+h3, x2:x2+w3]
        diff[y2:y2+h3, x2:x2+w3] = np.maximum(roi1, roi2)

dst = image.copy()
dst[(diff == 0) | (image > 50)] = 255  # Place white color whrere diff=0 and also where image is white.

cv2.imwrite('diff.png', diff)
cv2.imwrite('dilated_diff.png', dilated_diff)
cv2.imwrite('dst.png', dst)

cv2.imshow('diff', diff)
cv2.imshow('dilated_diff', dilated_diff)
cv2.imshow('dst', dst)
cv2.waitKey()
cv2.destroyAllWindows()

---

输出：

---

orig_diff.png:

dilated_diff.png:

diff.png：