Detect similar videos in Java and OpenCV rev.1


I had a chance to find out about "detecting similar images" for a moment, and in the process I was thinking about videos.

  1. Extract images from videos
  2. Compare images


I came up with the simple idea, and decided to make a little. ** Under development **

screen image image


Java (Java 9) is used on Windows, and Eclipse is used as the IDE. GUI creation is Scene Builder, or JavaFX.

Installing OpenCV will create a jar and a dll. Place them in a place where they can be read.

→ Copy to Windows \ System32 \

→ Copy under classpath, add from Eclipse

Furthermore, in Java, it was necessary to describe the load process in a class that uses OpenCV as shown below.

	static {

Extract images from videos

Extract some frames from the video as images. Here, I decided to extract 9 images. There is no particular basis, but it is a judgment that it looks good when arranged in GUI with 3x3. Since the first and last frames are likely to be darkened, save 9 frames separated by 10. -> FPS difference detection ...?

Also, all the extracted images are scaled so that the width is 200px before saving. -> Detection of size difference ...?

↓ It looks like this.

	static public VideoMetadata captureFrame(Config config, String videoFile) {
		// ------------------------------------------------------------------
		//Open the video
		// ------------------------------------------------------------------
		VideoCapture capture = new VideoCapture(videoFile);
		if (!capture.isOpened()) {
			System.out.println("file not opened.");
			return null;

		String outputDir = config.getTempDir() + "/" + String.valueOf(videoFile.hashCode()).replace("-", "_");
		File dir = new File(outputDir);
		if (!dir.exists()) {

		// ------------------------------------------------------------------
		//Acquisition of video information
		// ------------------------------------------------------------------
		double fps = capture.get(Videoio.CV_CAP_PROP_FPS);
		int resizeHeight = (int) (config.getImageWidth() / capture.get(Videoio.CV_CAP_PROP_FRAME_WIDTH)
				* capture.get(Videoio.CV_CAP_PROP_FRAME_HEIGHT));
		double allFrameCnt = capture.get(Videoio.CV_CAP_PROP_FRAME_COUNT);
		int captureInterval = (int) (allFrameCnt / (config.getCaptureCount() + 1));
		long time = (long) (allFrameCnt / fps);

		// ------------------------------------------------------------------
		//Get the number of frames specified in the settings
		// ------------------------------------------------------------------
		for (int i = 1; i < config.getCaptureCount() + 1; i++) {
			int frameIndex = captureInterval * i;
			Mat orgFrame = new Mat();
			Mat resizedFrame = new Mat();
			capture.set(Videoio.CV_CAP_PROP_POS_FRAMES, frameIndex - 1);
			if (! {
			Imgproc.resize(orgFrame, resizedFrame, new Size(config.getImageWidth(), resizeHeight));
			Imgcodecs.imwrite(outputDir + "/" + i + ".jpg ", resizedFrame);


		VideoMetadata metadata = new VideoMetadata();
		return metadata;

Compare images

In the previous section, we made 9 images per video. This time we will compare this.

For image comparison, I decided to compare histograms. I think there is much room for consideration in this regard.

The average of the comparison results of each of the nine images was used as the comparison result of the entire video.

The code is below.

	static public VideoComparison compareImages(Config config, VideoMetadata video1, VideoMetadata video2) {

		List<Double> histList = new ArrayList<Double>();
		// ------------------------------------------------------------------
		//Compare by image
		// ------------------------------------------------------------------
		for (int i = 1; i < config.getCaptureCount() + 1; i++) {
			String filename1 = video1.getFrameDirname() + "/" + i + ".jpg ";
			String filename2 = video2.getFrameDirname() + "/" + i + ".jpg ";
			File file1 = new File(filename1);
			File file2 = new File(filename2);
			if (!(file1.exists() && file2.exists())) {

			// ------------------------------------------------------------------
			// ------------------------------------------------------------------
			Mat img1 = Imgcodecs.imread(filename1);
			List<Mat> src1 = new ArrayList<Mat>();
			Mat hist1 = new Mat();
			Imgproc.calcHist(src1, new MatOfInt(0), new Mat(), hist1, new MatOfInt(256), new MatOfFloat(0, 256));

			Mat img2 = Imgcodecs.imread(filename2);
			List<Mat> src2 = new ArrayList<Mat>();
			Mat hist2 = new Mat();
			Imgproc.calcHist(src2, new MatOfInt(0), new Mat(), hist2, new MatOfInt(256), new MatOfFloat(0, 256));

			histList.add(Imgproc.compareHist(hist1, hist2, 0));


		VideoComparison videoComparison = new VideoComparison();
		return videoComparison;

Try to make it GUI

I want to make it a GUI because it handles images. I was worried, but I decided to make it with JavaFX to commemorate the release of Java9. (JavaFX is 8 ...)

In JavaFX, the design layout can be described in XML (fxml), and Scene Builder can edit this graphically. The XML and code are linked by the fx: id attribute in XML and the @FMXL annotation in .java.

For example, if the control of the execute button is arranged as follows,

<Button fx:id="executeBtn" layoutX="214.0" layoutY="31.0" mnemonicParsing="false" onAction="#doExecuteAction" prefHeight="38.0" prefWidth="115.0" text="execute" />

On the Java code side, do the following:

	void doExecuteAction(ActionEvent event) {
        // ...

Now you can attach the process that will be executed when you press the button.

The controls and actions used this time are as follows.



I wrote the code in Java for the first time in a long time, but there are many annoying aspects in personal use. Actually, I wrote it in Python at first, but I wanted to make it a GUI, what should I do with threads and so on ... → OpenCV can also be done in Java. So I chose Java.

I haven't had a GUI in Java since AWT, but JavaFX was better than I expected.

The rest is around the following.

--It takes time if there are many files

--Although it is processed by parallelStream, there seems to be room for tuning in terms of processing time.

--Visualization of processing status

--Implementation of progress bar

--Display processing status log

--Improved image comparison accuracy

--Comparison by features.

--Other ...?

Holiday programming was fun. end.


OpenCV: OpenCV modules

Client Technology: Java Platform, Standard Edition (Java SE) 8 Release 8

JavaFX Scene Builder 1.x Archive

Developing JavaFX8 in Eclipse using the designer --Qiita

bitWalk's: JavaFX: ListView with CheckBox

Recommended Posts

Detect similar videos in Java and OpenCV rev.2
Detect similar videos in Java and OpenCV rev.3
Detect similar videos in Java and OpenCV rev.1
Use OpenCV in Java
Encoding and Decoding example in Java
StringBuffer and StringBuilder Class in Java
Java + OpenCV 3.X in IntelliJ IDEA
Hello world in Java and Gradle
Difference between final and Immutable in Java
[Java] for Each and sorted in Lambda
I did OpenCV camera calibration in Java
Arrylist and linked list difference in java
Program PDF headers and footers in Java
Learn Flyweight patterns and ConcurrentHashMap in Java
Java Direction in C ++ Design and Evolution
Java to C and C to Java in Android Studio
Reading and writing gzip files in Java
Difference between int and Integer in Java
Discrimination of Enums in Java 7 and above
Regarding the transient modifier and serialization in Java
Create barcodes and QR codes in Java PDF
Use OpenCV_Contrib (ArUco) in Java! (Part 1-Build) (OpenCV-3.4.4)
Parallel and parallel processing in various languages (Java edition)
Difference between next () and nextLine () in Java Scanner
Differences in writing Java, C # and Javascript classes
Capture and save from selenium installation in Java
Add, read, and delete Excel comments in Java
Check static and public behavior in Java methods
[Java] Understand in 10 minutes! Associative array and HashMap
Basics of threads and Callable in Java [Beginner]
Distinguish between positive and negative numbers in Java
Java adds and removes watermarks in word documents
Represents "next day" and "previous day" in Java / Android
Questions in java exception handling throw and try-catch
Upload and download notes in java on S3
Encrypt / decrypt with AES256 in PHP and Java
Generate OffsetDateTime from Clock and LocalDateTime in Java
Changes in Java 11
Rock-paper-scissors in Java
Java and JavaScript
XXE and Java
Pi in Java
FizzBuzz in Java
[Android / Java] Screen transition and return processing in fragments
Convert JSON and YAML in Java (using Jackson and SnakeYAML)
Write ABNF in Java and pass the email address
Organize builds in C ++ / Java and Win / Linux combinations
Vectorize and image MNIST handwritten digit image data in Java
Write a class in Kotlin and call it in Java
Java classes and instances to understand in the figure
Java joins and splitting table cells in Word documents
This and that for editing ini in Java. : inieditor-java
Reverse Enum constants from strings and values in Java
Use of Abstract Class and Interface properly in Java
Gzip-compress byte array in Java and output to file