好的。这是一个新的尝试，它可能只适用于您的特定任务，将这些面部的 3 张图像准确混合，前、左、右。
我使用这些输入：
前（i1）：
左（i2）：
右（i3）：
前掩码（m1）：（可选）：
这些图像的问题在于，正面图像仅覆盖一小部分，而左右重叠整个正面图像，这导致我的其他解决方案中的混合效果不佳。此外，图像的对齐不是很好（主要是由于透视效果），因此可能会出现混合伪影。
现在这个新方法的想法是，您肯定希望保留前面图像的部分，这些部分位于您的彩色“标记点”所跨越的区域内，这些部分不应混合。离该标记区域越远，应该使用来自左右图像的信息越多，因此我们创建一个带有 alpha 值的蒙版，它从 1（在标记区域内）线性降低到 0（在某个定义的距离处）从标记区域）。
所以标记所跨越的区域是这个：
既然我们知道左图基本上用在左标记三角形的左边区域，我们可以为左图和右图创建掩码，用于查找应该被前图另外覆盖的区域：
左：
右：
前标记区域以及不在左侧和右侧掩码中的所有内容：
这可以用可选的前置掩码输入来掩蔽，这样会更好，因为这个前置图像示例没有覆盖整个图像，但遗憾的是只覆盖了图像的一部分。
现在这是混合蒙版，alpha 值线性递减，直到与蒙版的距离为10 或更多像素：
现在我们首先创建仅覆盖左右图像的图像，复制大部分未混合的部分，但将左/右蒙版未覆盖的部分与0.5*left + 0.5*right混合在一起
blendLR:
最后我们通过计算将front 图像融合到blendLR 中：
blended = alpha*front + (1-alpha)*blendLR
一些改进可能包括从一些更高的信息（如重叠的大小或从标记三角形到面部边界的大小）计算 maxDist 值。
另一个改进是不计算0.5*left + 0.5*right，而是在这里也做一些阿尔法混合，从左边的图像中获取更多信息，我们在差距的左边。这将减少图像中间的接缝（前图像部分的顶部和底部）。
// idea: keep all the pixels from front image that are inside your 6 points area always unblended:
cv::Mat blendFrontAlpha(cv::Mat front, cv::Mat left, cv::Mat right, std::vector<cv::Point> sixPoints, cv::Mat frontForeground = cv::Mat())
{
// define some maximum distance. No information of the front image is used if it's further away than that maxDist.
// if you have some real masks, you can easily set the maxDist according to the dimension of that mask - dimension of the 6-point-mask
float maxDist = 10;
// to use the cv function to draw contours we must order it like this:
std::vector<std::vector<cv::Point> > contours;
contours.push_back(sixPoints);
// create the mask
cv::Mat frontMask = cv::Mat::zeros(front.rows, front.cols, CV_8UC1);
// draw those 6 points connected as a filled contour
cv::drawContours(frontMask,contours,0,cv::Scalar(255),-1);
// add "lines": everything left from the points 3-4-5 might be used from left image, everything from the points 0-1-2 might be used from the right image:
cv::Mat leftMask = cv::Mat::zeros(front.rows, front.cols, CV_8UC1);
{
cv::Point2f center = cv::Point2f(sixPoints[3].x, sixPoints[3].y);
float steigung = ((float)sixPoints[5].y - (float)sixPoints[3].y)/((float)sixPoints[5].x - (float)sixPoints[3].x);
if(sixPoints[5].x - sixPoints[3].x == 0) steigung = 2*front.rows;
float n = center.y - steigung*center.x;
cv::Point2f top = cv::Point2f( (0-n)/steigung , 0);
cv::Point2f bottom = cv::Point2f( (front.rows-1-n)/steigung , front.rows-1);
// now create the contour of the left image:
std::vector<cv::Point> leftMaskContour;
leftMaskContour.push_back(top);
leftMaskContour.push_back(bottom);
leftMaskContour.push_back(cv::Point(0,front.rows-1));
leftMaskContour.push_back(cv::Point(0,0));
std::vector<std::vector<cv::Point> > leftMaskContours;
leftMaskContours.push_back(leftMaskContour);
cv::drawContours(leftMask,leftMaskContours,0,cv::Scalar(255),-1);
cv::imshow("leftMask", leftMask);
cv::imwrite("x_leftMask.png", leftMask);
}
// add "lines": everything left from the points 3-4-5 might be used from left image, everything from the points 0-1-2 might be used from the right image:
cv::Mat rightMask = cv::Mat::zeros(front.rows, front.cols, CV_8UC1);
{
// add "lines": everything left from the points 3-4-5 might be used from left image, everything from the points 0-1-2 might be used from the right image:
cv::Point2f center = cv::Point2f(sixPoints[2].x, sixPoints[2].y);
float steigung = ((float)sixPoints[0].y - (float)sixPoints[2].y)/((float)sixPoints[0].x - (float)sixPoints[2].x);
if(sixPoints[0].x - sixPoints[2].x == 0) steigung = 2*front.rows;
float n = center.y - steigung*center.x;
cv::Point2f top = cv::Point2f( (0-n)/steigung , 0);
cv::Point2f bottom = cv::Point2f( (front.rows-1-n)/steigung , front.rows-1);
std::cout << top << " - " << bottom << std::endl;
// now create the contour of the left image:
std::vector<cv::Point> rightMaskContour;
rightMaskContour.push_back(cv::Point(front.cols-1,0));
rightMaskContour.push_back(cv::Point(front.cols-1,front.rows-1));
rightMaskContour.push_back(bottom);
rightMaskContour.push_back(top);
std::vector<std::vector<cv::Point> > rightMaskContours;
rightMaskContours.push_back(rightMaskContour);
cv::drawContours(rightMask,rightMaskContours,0,cv::Scalar(255),-1);
cv::imshow("rightMask", rightMask);
cv::imwrite("x_rightMask.png", rightMask);
}
// add everything that's not in the side masks to the front mask:
cv::Mat additionalFrontMask = (255-leftMask) & (255-rightMask);
// if we know more about the front face, use that information:
cv::imwrite("x_frontMaskIncreased1.png", frontMask + additionalFrontMask);
if(frontForeground.cols)
{
// since the blending mask is blended for maxDist distance, we have to erode this mask here.
cv::Mat tmp;
cv::erode(frontForeground,tmp,cv::Mat(),cv::Point(),maxDist);
// idea is to only use the additional front mask in those areas where the front image contains of face and not those background parts.
additionalFrontMask = additionalFrontMask & tmp;
}
frontMask = frontMask + additionalFrontMask;
cv::imwrite("x_frontMaskIncreased2.png", frontMask);
//todo: add lines
cv::imshow("frontMask", frontMask);
// for visualization only:
cv::Mat frontMasked;
front.copyTo(frontMasked, frontMask);
cv::imshow("frontMasked", frontMasked);
cv::imwrite("x_frontMasked.png", frontMasked);
// compute inverse of mask to take it as input for distance transform:
cv::Mat inverseFrontMask = 255-frontMask;
// compute the distance to the mask, the further away from the mask, the less information from the front image should be used:
cv::Mat dist;
cv::distanceTransform(inverseFrontMask,dist,CV_DIST_L2, 3);
// scale wanted values between 0 and 1:
dist /= maxDist;
// remove all values > 1; those values are further away than maxDist pixel from the 6-point-mask
dist.setTo(cv::Scalar(1.0f), dist>1.0f);
// now invert the values so that they are == 1 inside the 6-point-area and go to 0 outside:
dist = 1.0f-dist;
cv::Mat alphaValues = dist;
//cv::Mat alphaNonZero = alphaValues > 0;
// now alphaValues contains your general blendingMask.
// but to use it on colored images, we need to duplicate the channels:
std::vector<cv::Mat> singleChannels;
singleChannels.push_back(alphaValues);
singleChannels.push_back(alphaValues);
singleChannels.push_back(alphaValues);
// merge all the channels:
cv::merge(singleChannels, alphaValues);
cv::imshow("alpha mask",alphaValues);
cv::imwrite("x_alpha_mask.png", alphaValues*255);
// convert all input mats to floating point mats:
front.convertTo(front,CV_32FC3);
left.convertTo(left,CV_32FC3);
right.convertTo(right,CV_32FC3);
cv::Mat result;
// first: blend left and right both with 0.5 to the result, this gives the correct results for the intersection of left and right equally weighted.
// TODO: these values could be blended from left to right, giving some finer results
cv::addWeighted(left,0.5,right,0.5,0, result);
// now copy all the elements that are included in only one of the masks (not blended, just 100% information)
left.copyTo(result,leftMask & (255-rightMask));
right.copyTo(result,rightMask & (255-leftMask));
cv::imshow("left+right", result/255.0f);
cv::imwrite("x_left_right.png", result);
// now blend the front image with it's alpha blending mask:
cv::Mat result2 = front.mul(alphaValues) + result.mul(cv::Scalar(1.0f,1.0f,1.0f)-alphaValues);
cv::imwrite("x_front_blend.png", front.mul(alphaValues));
cv::imshow("inv", cv::Scalar(1.0f,1.0f,1.0f)-alphaValues);
cv::imshow("f a", front.mul(alphaValues)/255.0f);
cv::imshow("f r", (result.mul(cv::Scalar(1.0f,1.0f,1.0f)-alphaValues))/255.0f);
result2.convertTo(result2, CV_8UC3);
return result2;
}
int main()
{
// front image
cv::Mat i1 = cv::imread("blending/new/front.jpg");
// left image
cv::Mat i2 = cv::imread("blending/new/left.jpg");
// right image
cv::Mat i3 = cv::imread("blending/new/right.jpg");
// optional: mask of front image
cv::Mat m1 = cv::imread("blending/new/mask_front.png",CV_LOAD_IMAGE_GRAYSCALE);
cv::imwrite("x_M1.png", m1);
// these are the marker points you detect in the front image.
// the order is important. the first three pushed points are the right points (left part of the face) in order from top to bottom
// the second three points are the ones from the left image half, in order from bottom to top
// check coordinates for those input images to understand the ordering!
std::vector<cv::Point> frontArea;
frontArea.push_back(cv::Point(169,92));
frontArea.push_back(cv::Point(198,112));
frontArea.push_back(cv::Point(169,162));
frontArea.push_back(cv::Point(147,162));
frontArea.push_back(cv::Point(122,112));
frontArea.push_back(cv::Point(147,91));
// first parameter is the front image, then left (right face half), then right (left half of face), then the image polygon and optional the front image mask (which contains all facial parts of the front image)
cv::Mat result = blendFrontAlpha(i1,i2,i3, frontArea, m1);
cv::imshow("RESULT", result);
cv::imwrite("x_Result.png", result);
cv::waitKey(-1);
return 0;
}