I did OpenCV camera calibration in Java


I needed something called camera calibration, so I tried it. Note: I am self-taught and may contain inaccurate statements.

What is Camera Calibration?

There is an expression that can convert the position in space to the position on the image, but this requires a camera parameter. There are different types of camera parameters, which can be broadly divided into internal parameters and external parameters. There is also a lens distortion coefficient (distortion), which I don't know if it is included in the internal parameters.

Internal parameters

Internal parameters are "parameters determined by the camera". ・ Focal length f ・ Physical spacing of pixels δ u </ sub>, δ v </ sub> -Image center in image coordinates relative to normalized coordinates c u </ sub>, c v </ sub>

External parameters

External parameters are "parameters of camera position and orientation with respect to the world coordinate system". ・ Rotation matrix R 3 </ sub> ・ Translation vector t 3 </ sub>

What is camera calibration?

Now, camera calibration is to find the parameters of this camera. The internal and external parameters obtained at this time are collectively called the camera matrix or perspective projection matrix. Determine the parameters by applying the spatial points whose positions are known in advance and the projection points (that is, the coordinates on the image) on the image to the matrix formula.

Specifically, take a picture while moving the calibration target (such as a chess board). Since the length of the chessboard grid is known, the distance between the grid points is known and its position in space is determined. (Conversely, it determines the world coordinate system) It also gets the coordinates of the grid points on the image. (Acquisition of image coordinates) Take multiple images and use the least squares method to find the parameters.

Quote (reference) Source: "Digital image processing [revised new edition]", the expression has been modified for convenience. Reference: Camera calibration and 3D reconstruction --OpenCV.jp

Calibrate the camera with OpenCV

This time, I calibrated the camera with the chest board, which I think is relatively easy, according to the following rough procedure.

For the code, see OpenCV_CameraCalibration --GitHub please.

  1. Hold the chest board (calibration target) and take a picture while moving it.
  2. Determine the coordinates of the world coordinate system and the grid points in space
  3. Find the coordinates of the grid points of the chestboard on the image [using Calib3d.findChessboardCorners ()]
  4. Execute Calib3d.calibrateCamera ()
  5. Save the returned camera matrix and lens distortion factor The outline looks like this.

I will explain each in a little more detail.

Determine the coordinates of the world coordinate system and grid points in space

Arbitrarily set the world coordinate system so that the plane of the flat calibration target (chest board) is Z = 0. At this time, measure the spacing between the grid points and set the points in mm. I'm calling here.

List<Mat> objectPoints = getObjectPoints(outputFindChessboardCorners.size(), patternSize); //Three-dimensional coordinates of chessboard corners(z=0), For the number of captured images.

OpenCV_CameraCalibration_L68 - GitHub

The contents of getObjectPoints () are as follows.

	public List<Mat> getObjectPoints(int size, Size patternSize) {
		List<Mat> objectPoints = new ArrayList<>();
		for (int i = 0; i < size; i++) {
		return objectPoints;

	public MatOfPoint3f getObjectPoint(Size patternSize) {
		MatOfPoint3f objectPoint = new MatOfPoint3f();

		List<Point3> objectPoint_ = new ArrayList<>();
		// final Size patternSize = new Size(6, 9); //Number of corners to explore
		for (int row = 0; row < patternSize.height; row++) {
			for (int col = 0; col < patternSize.width; col++) {
				objectPoint_.add(getPoint(row, col));

		return objectPoint;

	public Point3 getPoint(int row, int col) {
		final double REAL_HEIGHT = 20.0, REAL_WIDTH = 20.0;
		return new Point3(col * REAL_WIDTH, row * REAL_HEIGHT, 0.0); //Maybe x, y,z looks like this.

OpenCV_CameraCalibration_L96 - GitHub

Find the coordinates of the grid points on the chestboard on the image

I wrote a method called findChessboardCorners () myself. The call looks like this:

		List<Mat> outputFindChessboardCorners = new ArrayList<>();
		try (DirectoryStream<Path> ds = Files.newDirectoryStream(picFolderPath)) {
			for (Path path : ds) {

				final Optional<Mat> outputMat = findChessboardCorners(path.toString(), imagePoints, patternSize);

				if (outputMat.isPresent()) {
					System.out.println("successful to find corners.");
				} else {
					System.err.println("unsuccessful to find corners.");
		} catch (IOException ex) {

OpenCV_CameraCalibration_L50 - GitHub

The contents of findChessboardCorners () are as follows. I used a method called Calib3d.findChessboardCorners (). In addition, the returned Mat contains an image that Calib3d.findChessboardCorners () was able to execute normally.

	public Optional<Mat> findChessboardCorners(String picPathString, List<Mat> imagePoints, Size patternSize) {
		Mat inputMat = Imgcodecs.imread(picPathString);
		Mat mat = inputMat.clone();
		// final Size patternSize = new Size(6, 9); //Number of corners to explore
		MatOfPoint2f corners = new MatOfPoint2f(); // in,Receives a vector of the two-dimensional coordinates of the detected corner.

		Imgproc.cvtColor(inputMat, inputMat, Imgproc.COLOR_BGR2GRAY);

		final boolean canFindChessboard = Calib3d.findChessboardCorners(inputMat, patternSize, corners);

		if (!canFindChessboard) {
			System.err.println("Cannot find Chessboard Corners.");
			return Optional.empty();


		Calib3d.drawChessboardCorners(mat, patternSize, corners, true);

		Path picPath = Paths.get(picPathString);
		Path folderPath = Paths.get("S:\\CameraCaliblation\\2018-12-31_output");
		Path path = Paths.get(folderPath.toString(), picPath.getFileName().toString());

		if (!Files.exists(folderPath) || !Files.isDirectory(folderPath)) {
			try {
				System.out.println("There was no folder, so it is createing a folder. : " + folderPath.toString());
			} catch (IOException ex) {

		Imgcodecs.imwrite(path.toString(), mat);

		return Optional.of(inputMat);

Run Calib3d.calibrateCamera ()

Calib3d.calibrateCamera () with the coordinates of the grid points in the space prepared earlier and the coordinates of the grid points of the chest board on the image obtained earlier as arguments. /camera_calibration_and_3d_reconstruction.html#cv-calibratecamera). Also, since this method returns the values of the camera matrix and lens distortion coefficient, create a variable to store them.

		//What to receive
		Mat cameraMatrix = new Mat(), distortionCoefficients = new Mat();
		List<Mat> rotationMatrixs = new ArrayList<>(), translationVectors = new ArrayList<>();

		Calib3d.calibrateCamera(objectPoints, imagePoints, imageSize,
				cameraMatrix, distortionCoefficients, rotationMatrixs, translationVectors);

OpenCV_CameraCalibration_L81 - GitHub

Save the returned camera matrix and lens distortion factor

I was able to get the camera matrix and lens distortion factor, but it's tedious and wasteful to do this every time, so let's save the matrix values. In the case of C and Python, there is something that inputs and outputs the Mat value on the OpenCV side, but since it is not the Java version, I made it myself this time. I used the XML input / output found in the Java standard library.

		Map<String, Mat> exportMats = new HashMap<>();
		exportMats.put("CameraMatrix", cameraMatrix);
		exportMats.put("DistortionCoefficients", distortionCoefficients);
		final Path exportFilePath = Paths.get("S:\\CameraCaliblation\\CameraCalibration_2018-12-31.xml");
		MatIO.exportMat(exportMats, exportFilePath);

OpenCV_CameraCalibration_L87 - GitHub

The contents of MatIO are OpenCV_CameraCalibration_MatIO.java --GitHub Please refer to.

in conclusion

For the time being, using this, [Posture estimation of AR marker of ArUco](https://qiita.com/smk7758/items/1ca1370f78cad1233d02#%E3%82%AB%E3%83%A1%E3%83%A9% E3% 82% 92% E7% 94% A8% E3% 81% 84% E3% 81% 9F% E3% 83% AA% E3% 82% A2% E3% 83% AB% E3% 82% BF% E3% 82% A4% E3% 83% A0marker% E5% A7% BF% E5% 8B% A2% E6% 8E% A8% E5% AE% 9A) and it worked fine. I am glad that the juniors of our information group also understood and implemented it and succeeded.

Reference link

OpenCV --How to calibrate the camera --PynoteCamera Calibration (OpenCV 1.0) --OpenCV.jp -Image processing and recognition by OpenCV and Visual C ++ -Code for camera calibration (obtaining internal parameters and distortion coefficient of one camera) with python and OpenCV (Plagiarism)Camera calibration -Camera calibration with OpenCV 2.4.11 & C ++-Technical singularity -Camera Calibration and 3D Reconstruction --OpenCV Official-There is a good explanation if you look at the official website again. -[CC_Controller.java --opencv-java / camera-calibration (GitHub)](https://github.com/opencv/opencv/blob/master/samples/android/camera-calibration/src/org/opencv/samples/ cameracalibration / CameraCalibrator.java) --It was helpful for the Java version. ・ Cameracalibration --opencv (GitHub)

Recommended Posts

I did OpenCV camera calibration in Java
Use OpenCV in Java
I made roulette in Java.
I tried metaprogramming in Java
I sent an email in Java
I created a PDF in Java.
I made an annotation in Java.
I tried using JWT in Java
Java + OpenCV 3.X in IntelliJ IDEA
[java] What I did when comparing Lists in my own class
[Java] I participated in ABC-188 of Atcorder.
I tried using Elasticsearch API in Java
I tried the new era in Java
I tried using OpenCV with Java + Tomcat
[* Java *] I participated in JJUG CCC 2019 Spring
I made a primality test program in Java
Detect similar videos in Java and OpenCV rev.2
Use OpenCV_Contrib (ArUco) in Java! (Part 1-Build) (OpenCV-3.4.4)
Partization in Java
I want to send an email in Java.
I did Java to make (a == 1 && a == 2 && a == 3) always true
Changes in Java 11
I tried to implement deep learning in Java
I wanted to make (a == 1 && a == 2 && a == 3) true in Java
I wrote a primality test program in Java
I made a rock-paper-scissors game in Java (CLI)
rsync4j --I want to touch rsync in Java.
Detect similar videos in Java and OpenCV rev.3
What I learned in Java (Part 2) What are variables?
I tried to output multiplication table in Java
What I did when I converted java to Kotlin
Detect similar videos in Java and OpenCV rev.1
I tried to create Alexa skill in Java
I wrote a prime factorization program in Java
Pi in Java
FizzBuzz in Java
I made a simple calculation problem game in Java
I tried Mastodon's Toot and Streaming API in Java
I want to do something like "cls" in Java
I tried using Google Cloud Vision API in Java
I wrote about Java downcast in an easy-to-understand manner
I want to use ES2015 in Java too! → (´ ・ ω ・ `)
# 2 [Note] I tried to calculate multiplication tables in Java.
I tried to create a Clova skill in Java
I tried to make a login function in Java
I dealt with Azure Functions not working in Java
I tried using an extended for statement in Java
I tried passing Java Silver in 2 weeks without knowing Java
I tried to implement the Euclidean algorithm in Java
~ I tried to learn functional programming in Java now ~
I tried to find out what changed in Java 9
[java] sort in list
Read JSON in Java
Interpreter implementation in Java
Make Blackjack in Java
Rock-paper-scissors app in Java
Constraint programming in Java
Put java8 in centos7
NVL-ish guy in Java
Combine arrays in Java
I first touched Java ②