I just erased the object using image repair (inpaint) (OpenCV: C ++)

Background I will introduce the inpaint function that exists only in C ++, following I erased the object using image repair (inpaint) (OpenCV: Python).

Introduction

I summarized it roughly in the table.

inpaint type year python include library thesis note
Navier Stokes-Stokes) method 2001 #include <opencv2/photo.hpp> opencv_photo Navier-stokes, fluid dynamics, and image and video inpainting Algorithm based on fluid mechanics
Method by Alexandru Telea 2004 #include <opencv2/photo.hpp> opencv_photo An image inpainting technique based on the fast marching method. An algorithm based on the Fast Marching Method that gradually repairs scratches inward from the boundary of the area
Patch Offset method 2012 × #include <opencv2/xphoto.hpp> opencv_xphoto Statistics of Patch Offsets for Image Completion Algorithm that infers and repairs similar images in an image
Method by fuzzy Fourier transform 2014 × #include <opencv2/fuzzy.hpp> opencv_fuzzy Image reconstruction by means of F-transform A method using the Fourier transform. For small scratchesONE_STEP, For big scratchesMULTI_STEP, Combined with small scratches and large scratchesITERATIVE3 patterns are prepared

Development

GNUmakefile


CXX = c++
CXXFLAGS =  -I/usr/local/Cellar/opencv/4.1.1_2/include/opencv4/
LDFLAGS = -L/usr/local/Cellar/opencv/4.1.1_2/lib/
LDLIBS =  -lopencv_core -lopencv_highgui -lopencv_imgcodecs -lopencv_imgproc -lopencv_fuzzy -lopencv_photo -lopencv_xphoto
CXXVERSION = -std=c++11

inpaint: inpaint.cpp
	$(CXX) $< -o $@ $(CXXFLAGS) $(CXXVERSION) $(LDFLAGS) $(LDLIBS)

clean :
	rm inpaint

inpaint.cpp


#include <opencv2/opencv.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/fuzzy.hpp>
#include <opencv2/photo.hpp>
#include <opencv2/xphoto.hpp>


int main(int argc, const char* argv[]) {
	cv::Mat src = cv::imread("./capture_2.png ");


	cv::Mat resizeMat, dst, mergeMat;
	cv::resize(src, resizeMat, cv::Size(), 0.5, 0.5);

	//Mask image background black-Repair section white
	cv::Mat mask_img = cv::Mat::zeros(resizeMat.size(), CV_8UC3);
	cv::rectangle(mask_img, cv::Point(185,255), cv::Point(470,300), cv::Scalar(255,255,255), -1, cv::LINE_AA); 
	cv::rectangle(mask_img, cv::Point(40,25), cv::Point(115,65), cv::Scalar(255,255,255), -1, cv::LINE_AA);
	cv::rectangle(mask_img, cv::Point(200,40), cv::Point(440,110), cv::Scalar(255,255,255), -1, cv::LINE_AA);

	cv::cvtColor(mask_img,mask_img,cv::COLOR_BGR2GRAY);

	cv::imwrite("mask.png ", mask_img);

	// ns
	cv::inpaint(resizeMat, mask_img, dst, 3, cv::INPAINT_NS);
	cv::imwrite("output_ns.png ", dst);
	dst.release();

	// telea
	cv::inpaint(resizeMat, mask_img, dst, 3, cv::INPAINT_TELEA);
	cv::imwrite("output_telea.png ", dst);
	dst.release();

	cv::Mat mask_inv = ~mask_img;
	resizeMat.copyTo(mergeMat, mask_inv);

	// xphoto
	cv::xphoto::inpaint(mergeMat, mask_inv, dst, cv::xphoto::INPAINT_SHIFTMAP);
	cv::imwrite("output_xphoto_shiftmap.png ", dst);
	dst.release();

	// fuzzy
	cv::ft::inpaint(mergeMat, mask_inv, dst, 3, cv::ft::LINEAR, cv::ft::ONE_STEP);
	cv::imwrite("output_fuzzy_linear_one_step.png ", dst);
	dst.release();

	cv::ft::inpaint(mergeMat, mask_inv, dst, 3, cv::ft::LINEAR, cv::ft::MULTI_STEP);
	cv::imwrite("output_fuzzy_linear_multi.png ", dst);
	dst.release();

	cv::ft::inpaint(mergeMat, mask_inv, dst, 3, cv::ft::LINEAR, cv::ft::ITERATIVE);
	cv::imwrite("output_fuzzy_linear_iterative.png ", dst);
	dst.release();

	src.release();
	mergeMat.release();
	resizeMat.release();
	mask_img.release();
	mask_inv.release();

   return 0;
}

Run


make
./inpaint

Supplement

mask.png

How to load the mask image is different for each function. cv :: inpaint makes the area to be repaired white, but cv :: xphoto :: inpaint cv :: ft :: inpaint makes it black. Therefore, it is inverted with cv :: Mat mask_inv = ~ mask_img;.

Result

library parameter image odds
photo INPAINT_NS output_ns.png
photo INPAINT_TELEA output_telea.png
xphoto INPAINT_SHIFTMAP output_xphoto_shiftmap.png
fuzzy ONE_STEP output_fuzzy_linear_one_step.png ×
fuzzy MULTI_STEP output_fuzzy_linear_multi.png
fuzzy ITERATIVE output_fuzzy_linear_iterative.png

If you want to erase the above time, it will be repaired based on the background image, so there is no problem with cv :: inpaint. As for the subtitles below, if you use cv :: inpaint, the crease will be sharp in the middle, but cv :: ft :: inpaint seems to be a little more relaxed. Although it is blurry than other parts, MULTI_STEP is finished naturally as a whole.

cv :: xphoto :: inpaint INPAINT_SHIFTMAP is pasted based on the section in the image, and it seems to be effective when there are many identical objects such as buildings and forests in the background image.

You can see that cv :: ft :: inpaint ONE_STEP is not supported because it is a parameter for small scratches.

Future Next time I will try to erase the subtitles in the video: smirk_cat:

Reference

-Image Inpainting --opencv-python doc -Repair image / remove unnecessary objects --opencv cookbook -Development memo # 69 Using Inpaint with OpenCV 3.2 with Contrib module

Recommended Posts

I just erased the object using image repair (inpaint) (OpenCV: C ++)
I just erased the object using image repair (inpaint) (OpenCV: Python)
I tried using the image filter of OpenCV
I tried "differentiating" the image with Python + OpenCV
I tried object detection using Python and OpenCV
I tried "binarizing" the image with Python + OpenCV
I tried to compress the image using machine learning
I tried "gamma correction" of the image with Python + OpenCV
[For beginners] I tried using the Tensorflow Object Detection API
I tried to process the image in "sketch style" with OpenCV
I tried to digitize the stamp stamped on paper using OpenCV
I tried to process the image in "pencil style" with OpenCV
[Python] Using OpenCV with Python (Image Filtering)
Judgment of backlit image using OpenCV
[Python] Using OpenCV with Python (Image transformation)
I tried using GrabCut of OpenCV
Try blurring the image with opencv2
I tried using the checkio API
I tried to transform the face image using sparse_image_warp of TensorFlow Addons
I tried to get the batting results of Hachinai using image processing
I want to display an image on Jupyter Notebook using OpenCV (mac)