Detect similar videos in Java and OpenCV rev.3

We are still working on it.

Detecting similar videos in Java and OpenCV rev.1 --Qiita

Detecting similar videos in Java and OpenCV rev.2 --Qiita

Result cache

By saving the result for the file that has been processed once, the image processing is not performed by OpenCV for the second time.

Save Mat

The Mat of the image should be saved as an image by imwrite, but the calculation result of the histogram etc. could not be saved by imwrite and read by imread.

There is a general way to save Mat to a file in C ++, but it seems that you have to create it yourself as there is no Java wrapped one.

Since Mat is Matrix, that is, matrix data, it can be expressed vertically, horizontally, and as an array of data. In the case of Mat, in addition to that, it will have a type code that represents the type of data.

For this purpose, the amount of data is not so big, so I will save it as json data. I used google's gson library to handle json.

This is the save part.

	 *Save Mat to a file
	 * @param filename
	 * @param mat
	 * @throws IOException
	public static void storeMat(String filename, Mat mat) throws IOException {
		String jsondata = matToJson(mat);
		PrintWriter writer = new PrintWriter(new BufferedWriter(new FileWriter(new File(filename))));

	 *Convert Mat to Json format
	 * @param mat
	 * @return
	public static String matToJson(Mat mat) {
		JsonObject obj = new JsonObject();

		if (mat.isContinuous()) {
			int cols = mat.cols();
			int rows = mat.rows();
			int elemSize = (int) mat.elemSize();
			int type = mat.type();
			String typeStr = CvType.typeToString(type);
			int dataSize = cols * rows * elemSize;
			if (rows == 0) {
				dataSize = cols * elemSize;

			String dataString = "";
			if (typeStr.contains("32F")) {
				float[] data = new float[dataSize];
				String[] strData = new String[dataSize];
				mat.get(0, 0, data);
				for (int i = 0; i < data.length; i++) {
					strData[i] = String.valueOf(data[i]);
				dataString = String.join(",", strData);
			} else if (typeStr.contains("8U")) {
				byte[] data = new byte[dataSize];
				String[] strData = new String[dataSize];
				mat.get(0, 0, data);
				for (int i = 0; i < data.length; i++) {
					strData[i] = String.valueOf(data[i]);
				dataString = String.join(",", strData);

			obj.addProperty("rows", mat.rows());
			obj.addProperty("cols", mat.cols());
			obj.addProperty("type", mat.type());
			obj.addProperty("data", dataString);

			Gson gson = new Gson();
			String json = gson.toJson(obj);

			return json;
		} else {
		return "{}";

This is the reading part.

	 *Read Mat from file
	 * @param filename
	 * @return
	 * @throws IOException
	public static Mat loadMat(String filename) throws IOException {
		InputStream input = new FileInputStream(filename);
		int size = input.available();
		byte[] buffer = new byte[size];;
		return matFromJson(new String(buffer));

	 *Create Mat from Json
	 * @param json
	 * @return
	public static Mat matFromJson(String json) {
		JsonParser parser = new JsonParser();
		JsonObject JsonObject = parser.parse(json).getAsJsonObject();

		int rows = JsonObject.get("rows").getAsInt();
		int cols = JsonObject.get("cols").getAsInt();
		int type = JsonObject.get("type").getAsInt();
		String typeStr = CvType.typeToString(type);

		String dataString = JsonObject.get("data").getAsString();
		Mat mat = new Mat(rows, cols, type);
		String[] splitedStr = dataString.split(",");
		if (typeStr.contains("32F")) {
			float[] data = new float[splitedStr.length];
			for (int i = 0; i < data.length; i++) {
				data[i] = Float.parseFloat(splitedStr[i]);
			mat.put(0, 0, data);
		} else if (typeStr.contains("8U")) {
			byte[] data = new byte[splitedStr.length];
			for (int i = 0; i < data.length; i++) {
				data[i] = Byte.parseByte(splitedStr[i]);
			mat.put(0, 0, data);
		} else {

		return mat;

The Mat type is only for 8U and 32F, and it is a little overkill for general purpose. If necessary, such as 64F → double, it is necessary to add for each type.

Python (pandas) is excellent in the ease of handling matrices and json around here.

Save other video information

I used java.util.Properties and saved it as a properties file.

Properties videoProperty = new Properties();
videoProperty.setProperty("playtime", String.valueOf(time));
videoProperty.setProperty("width", String.valueOf(width));
videoProperty.setProperty("height", String.valueOf(height));
videoProperty.setProperty("fps", String.valueOf(fps));
videoProperty.setProperty("size", String.valueOf(fileSize)); FileOutputStream(metadata.getMetaFilename()), "");

Save the comparison result

At first I was trying to save it in CSV, but finally I decided to save it in SQLite. I gave up on CSV because there are various libraries and I couldn't get the one I chose properly.

The created table.

	key text PRIMARY KEY, 
	file1 text, 
	file2 text, 
	timediff integer, 
	hist real, 
	feature real, 
	skip integer 

I prepared a field called "skip" here. This is a flag that indicates that this comparison result is excluded from the result, and is set when it is judged that it is different when visually confirmed, or when the playback time is different by 10% or more.

UI improvements

The following has been added for ease of use.

--Display metadata on label --Button to move the result selection up and down --Button to hide the result from the next --Play button --Recycle bin button

For playback, I want to start it with the associated application, but it doesn't work ... For the time being, I try to start it with a fixed application.

The screen after the change.


The design sense is delicate. ..


Diligent release

Don't forget to call Mat # release frequently when dealing with large numbers of images. Especially in the case of Java, be especially careful because you cannot see the resources of the Mat object that seems to be allocated by JNI.

The same is true for Video Capture.

Antivirus software settings

When I was creating a large number of files, the CPU usage was high, but the cause was antivirus software. It seems better to set it such as turning off real-time search of the target folder.


OpenCV Mat object serialization in java - Stack Overflow

Recommended Posts

Detect similar videos in Java and OpenCV rev.2
Detect similar videos in Java and OpenCV rev.3
Detect similar videos in Java and OpenCV rev.1
Use OpenCV in Java
StringBuffer and StringBuilder Class in Java
Understanding equals and hashCode in Java
Java + OpenCV 3.X in IntelliJ IDEA
Hello world in Java and Gradle
Difference between final and Immutable in Java
[Java] for Each and sorted in Lambda
I did OpenCV camera calibration in Java
Arrylist and linked list difference in java
Program PDF headers and footers in Java
Learn Flyweight patterns and ConcurrentHashMap in Java
Java Direction in C ++ Design and Evolution
Java to C and C to Java in Android Studio
Reading and writing gzip files in Java
Difference between int and Integer in Java
Discrimination of Enums in Java 7 and above
Regarding the transient modifier and serialization in Java
Create barcodes and QR codes in Java PDF
Use OpenCV_Contrib (ArUco) in Java! (Part 1-Build) (OpenCV-3.4.4)
Parallel and parallel processing in various languages (Java edition)
Difference between next () and nextLine () in Java Scanner
Differences in writing Java, C # and Javascript classes
Capture and save from selenium installation in Java
Add, read, and delete Excel comments in Java
Check static and public behavior in Java methods
[Java] Understand in 10 minutes! Associative array and HashMap
Basics of threads and Callable in Java [Beginner]
Distinguish between positive and negative numbers in Java
Java adds and removes watermarks in word documents
Represents "next day" and "previous day" in Java / Android
Questions in java exception handling throw and try-catch
Upload and download notes in java on S3
Encrypt / decrypt with AES256 in PHP and Java
Generate OffsetDateTime from Clock and LocalDateTime in Java
Partization in Java
Changes in Java 11
Rock-paper-scissors in Java
Java and JavaScript
XXE and Java
Pi in Java
FizzBuzz in Java
[Android / Java] Screen transition and return processing in fragments
I tried Mastodon's Toot and Streaming API in Java
Write ABNF in Java and pass the email address
Organize builds in C ++ / Java and Win / Linux combinations
Vectorize and image MNIST handwritten digit image data in Java
Write a class in Kotlin and call it in Java
[Java] Difference between static final and final in member variables
Java classes and instances to understand in the figure
Java joins and splitting table cells in Word documents
This and that for editing ini in Java. : inieditor-java
How to convert A to a and a to A using AND and OR in Java
Reverse Enum constants from strings and values in Java
Use of Abstract Class and Interface properly in Java
Gzip-compress byte array in Java and output to file