We are still working on it.

Result cache

By saving the result for the file that has been processed once, the image processing is not performed by OpenCV for the second time.

Save Mat

The Mat of the image should be saved as an image by imwrite, but the calculation result of the histogram etc. could not be saved by imwrite and read by imread.

There is a general way to save Mat to a file in C ++, but it seems that you have to create it yourself as there is no Java wrapped one.

Since Mat is Matrix, that is, matrix data, it can be expressed vertically, horizontally, and as an array of data. In the case of Mat, in addition to that, it will have a type code that represents the type of data.

For this purpose, the amount of data is not so big, so I will save it as json data. I used google's gson library to handle json.

This is the save part.

	/**
	 *Save Mat to a file
	 * @param filename
	 * @param mat
	 * @throws IOException
	 */
	public static void storeMat(String filename, Mat mat) throws IOException {
		String jsondata = matToJson(mat);
		PrintWriter writer = new PrintWriter(new BufferedWriter(new FileWriter(new File(filename))));
		writer.write(jsondata);
		writer.close();
	}

	/**
	 *Convert Mat to Json format
	 * @param mat
	 * @return
	 */
	public static String matToJson(Mat mat) {
		JsonObject obj = new JsonObject();

		if (mat.isContinuous()) {
			int cols = mat.cols();
			int rows = mat.rows();
			int elemSize = (int) mat.elemSize();
			int type = mat.type();
			String typeStr = CvType.typeToString(type);
			int dataSize = cols * rows * elemSize;
			if (rows == 0) {
				dataSize = cols * elemSize;
			}

			String dataString = "";
			if (typeStr.contains("32F")) {
				float[] data = new float[dataSize];
				String[] strData = new String[dataSize];
				mat.get(0, 0, data);
				for (int i = 0; i < data.length; i++) {
					strData[i] = String.valueOf(data[i]);
				}
				dataString = String.join(",", strData);
			} else if (typeStr.contains("8U")) {
				byte[] data = new byte[dataSize];
				String[] strData = new String[dataSize];
				mat.get(0, 0, data);
				for (int i = 0; i < data.length; i++) {
					strData[i] = String.valueOf(data[i]);
				}
				dataString = String.join(",", strData);
			}

			obj.addProperty("rows", mat.rows());
			obj.addProperty("cols", mat.cols());
			obj.addProperty("type", mat.type());
			obj.addProperty("data", dataString);

			Gson gson = new Gson();
			String json = gson.toJson(obj);

			return json;
		} else {
			System.err.println("");
		}
		return "{}";
	}

This is the reading part.

	/**
	 *Read Mat from file
	 * @param filename
	 * @return
	 * @throws IOException
	 */
	public static Mat loadMat(String filename) throws IOException {
		InputStream input = new FileInputStream(filename);
		int size = input.available();
		byte[] buffer = new byte[size];
		input.read(buffer);
		input.close();
		return matFromJson(new String(buffer));
	}

	/**
	 *Create Mat from Json
	 * @param json
	 * @return
	 */
	public static Mat matFromJson(String json) {
		JsonParser parser = new JsonParser();
		JsonObject JsonObject = parser.parse(json).getAsJsonObject();

		int rows = JsonObject.get("rows").getAsInt();
		int cols = JsonObject.get("cols").getAsInt();
		int type = JsonObject.get("type").getAsInt();
		String typeStr = CvType.typeToString(type);

		String dataString = JsonObject.get("data").getAsString();
		Mat mat = new Mat(rows, cols, type);
		String[] splitedStr = dataString.split(",");
		if (typeStr.contains("32F")) {
			float[] data = new float[splitedStr.length];
			for (int i = 0; i < data.length; i++) {
				data[i] = Float.parseFloat(splitedStr[i]);
			}
			mat.put(0, 0, data);
		} else if (typeStr.contains("8U")) {
			byte[] data = new byte[splitedStr.length];
			for (int i = 0; i < data.length; i++) {
				data[i] = Byte.parseByte(splitedStr[i]);
			}
			mat.put(0, 0, data);
		} else {
			System.err.println("");
		}

		return mat;
	}

The Mat type is only for 8U and 32F, and it is a little overkill for general purpose. If necessary, such as 64F → double, it is necessary to add for each type.

Python (pandas) is excellent in the ease of handling matrices and json around here.

Save other video information

I used java.util.Properties and saved it as a properties file.

Properties videoProperty = new Properties();
videoProperty.setProperty("playtime", String.valueOf(time));
videoProperty.setProperty("width", String.valueOf(width));
videoProperty.setProperty("height", String.valueOf(height));
videoProperty.setProperty("fps", String.valueOf(fps));
videoProperty.setProperty("size", String.valueOf(fileSize));
videoProperty.store(new FileOutputStream(metadata.getMetaFilename()), "");

Save the comparison result

At first I was trying to save it in CSV, but finally I decided to save it in SQLite. I gave up on CSV because there are various libraries and I couldn't get the one I chose properly.

The created table.

CREATE TABLE IF NOT EXISTS compare_video ( 
	key text PRIMARY KEY, 
	file1 text, 
	file2 text, 
	timediff integer, 
	hist real, 
	feature real, 
	skip integer 
)

I prepared a field called "skip" here. This is a flag that indicates that this comparison result is excluded from the result, and is set when it is judged that it is different when visually confirmed, or when the playback time is different by 10% or more.

UI improvements

The following has been added for ease of use.

--Display metadata on label --Button to move the result selection up and down --Button to hide the result from the next --Play button --Recycle bin button

For playback, I want to start it with the associated application, but it doesn't work ... For the time being, I try to start it with a fixed application.

The screen after the change.

The design sense is delicate. ..

Other

Diligent release

Don't forget to call Mat # release frequently when dealing with large numbers of images. Especially in the case of Java, be especially careful because you cannot see the resources of the Mat object that seems to be allocated by JNI.

The same is true for Video Capture.

Antivirus software settings

When I was creating a large number of files, the CPU usage was high, but the cause was antivirus software. It seems better to set it such as turning off real-time search of the target folder.

This is also a Windows server.

reference

OpenCV Mat object serialization in java - Stack Overflow

Detect similar videos in Java and OpenCV rev.3