Detect similar videos in Java and OpenCV rev.2

We have made some improvements since the last time.

Detecting similar videos in Java and OpenCV rev.1 --Qiita

Change image comparison

Comparison by features

In addition to the histogram, the features are compared. There are various feature algorithms, but here we will use the one called "AKAZE". I don't know the details, but it seems to have been added from OpenCV3, and it looked good (miscellaneous).

Convert to grayscale before processing.

FeatureDetector detector = FeatureDetector.create(FeatureDetector.AKAZE);
DescriptorExtractor executor = DescriptorExtractor.create(DescriptorExtractor.AKAZE);

Mat gray = new Mat();
Imgproc.cvtColor(resizedFrame, gray, Imgproc.COLOR_RGB2GRAY);
MatOfKeyPoint point = new MatOfKeyPoint();
detector.detect(gray, point);
Mat desc = new Mat();
executor.compute(gray, point, desc);

Next, use Descriptor Matcher to compare the acquired feature amount Mat. You can choose several methods for this as well, but I'm not sure about this either, so I chose BRUTE FORCE.

DescriptorMatcher macher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);

MatOfDMatch feature = new MatOfDMatch();
macher.match(video1.getFeatureImg().get(i - 1), video2.getFeatureImg().get(i - 1), feature);
List<Double> distanceList = new ArrayList<>();
for (DMatch dMatch : feature.toList()) {

This time, I decided to handle the results of video comparison on average. Since it is "distance", it can be judged that the smaller value is "similar".

Comparison by playback time

For the purpose of reducing the processing time, we did not compare the ones with a difference in playback time of more than 10%.

long playtime1 = video1.getPlayTime();
long playtime2 = video2.getPlayTime();
long playtimeDiff = Math.abs(playtime1 - playtime2);
if ((playtimeDiff / playtime1) > 0.1) {
	return null;

Due to the current mechanism, it is unavoidable because it is not possible to compare things that are related to "included" videos.

Other changes

Open explorer from results

Added a context menu so that you can check the file after checking the result. Here's how to add a context menu to a TableView.

ContextMenu menu = new ContextMenu();
MenuItem mi = new MenuItem("Open in explorer");
mi.setOnAction(event -> {
	TableItem item = table.getSelectionModel().getSelectedItem();
	String command1 = "explorer /select," + item.getOrg().getVideo1().getFilename();
	String command2 = "explorer /select," + item.getOrg().getVideo2().getFilename();
	try {
	} catch (IOException e) {

Exception handling is appropriate ...

in conclusion

I was able to detect "similar videos" just by comparing the histograms more than I expected. I think that it is accurate enough to detect data related to compressed original data such as frame rate and resolution. I wonder if it will be deep learning if it becomes more than this.

The next challenge is the non-functional aspect.

The execution time was about 90 seconds for 70 files in an environment with Core i5-4690 and 4GB of memory allocated. I am running it while looking at jconsole, but the memory seems to be severe as the number of files increases, so there seems to be room for improvement.

I'm currently working on caching results and writing histograms and features to a file and not having them in memory, but that doesn't work.

I want to improve it.

The end.


Recommended Posts

Detect similar videos in Java and OpenCV rev.2
Detect similar videos in Java and OpenCV rev.3
Detect similar videos in Java and OpenCV rev.1
Use OpenCV in Java
Encoding and Decoding example in Java
StringBuffer and StringBuilder Class in Java
Understanding equals and hashCode in Java
Java + OpenCV 3.X in IntelliJ IDEA
Hello world in Java and Gradle
Difference between final and Immutable in Java
[Java] for Each and sorted in Lambda
I did OpenCV camera calibration in Java
Arrylist and linked list difference in java
Program PDF headers and footers in Java
Learn Flyweight patterns and ConcurrentHashMap in Java
Java Direction in C ++ Design and Evolution
Java to C and C to Java in Android Studio
Reading and writing gzip files in Java
Difference between int and Integer in Java
Discrimination of Enums in Java 7 and above
Regarding the transient modifier and serialization in Java
Create barcodes and QR codes in Java PDF
Use OpenCV_Contrib (ArUco) in Java! (Part 1-Build) (OpenCV-3.4.4)
Parallel and parallel processing in various languages (Java edition)
Difference between next () and nextLine () in Java Scanner
Differences in writing Java, C # and Javascript classes
Capture and save from selenium installation in Java
Add, read, and delete Excel comments in Java
Check static and public behavior in Java methods
[Java] Understand in 10 minutes! Associative array and HashMap
Basics of threads and Callable in Java [Beginner]
Distinguish between positive and negative numbers in Java
Java adds and removes watermarks in word documents
Represents "next day" and "previous day" in Java / Android
Questions in java exception handling throw and try-catch
Upload and download notes in java on S3
Encrypt / decrypt with AES256 in PHP and Java
Generate OffsetDateTime from Clock and LocalDateTime in Java
Partization in Java
Changes in Java 11
Rock-paper-scissors in Java
Java and JavaScript
XXE and Java
Pi in Java
FizzBuzz in Java
[Android / Java] Screen transition and return processing in fragments
Convert JSON and YAML in Java (using Jackson and SnakeYAML)
I tried Mastodon's Toot and Streaming API in Java
Write ABNF in Java and pass the email address
Organize builds in C ++ / Java and Win / Linux combinations
Vectorize and image MNIST handwritten digit image data in Java
Write a class in Kotlin and call it in Java
[Java] Difference between static final and final in member variables
Java classes and instances to understand in the figure
Java joins and splitting table cells in Word documents
This and that for editing ini in Java. : inieditor-java
How to convert A to a and a to A using AND and OR in Java
Reverse Enum constants from strings and values in Java
Use of Abstract Class and Interface properly in Java
Gzip-compress byte array in Java and output to file