[JAVA] Face recognition app with OpenCV + Android Studio
Thing you want to do
Creating a face recognition app using OpenCV ·Target ・ For the time being, learn how to install and use Android Studio (beginner) ・ Run OpenCV on Android ・ Both images taken with the camera and images saved in the gallery can be used.
reference
I tried to make a simple face recognition Android application using OpenCV https://qiita.com/Kuroakira/items/094ecb236da89949d702 Supplement on how to call the camera with an intent (mainly for Xperia 2.1 problem) https://gabu.hatenablog.com/entry/20101125/1290681748
What I used
Android Studio 3.5.3 OpenCV 4.2.0
Rough flow
As for the implementation, I will write it briefly because I refer to the above predecessors.
Press the camera and gallery buttons ↓ Get images from camera, gallery ↓ Convert the acquired image from bitmap to Mat format ↓ Pass Mat format image to OpenCV ↓ Acquires face recognition information and displays it on the screen
result
Coordinate information like that came out.
Things that can be used in the future
·camera If you want to acquire the image taken by the camera in high resolution, write it to the URL once and acquire the image from the URL. getImg = (Bitmap) data.getExtras (). get ("data"); can only get low resolution images for thumbnails. (The image for image recognition is obtained by getImg = (Bitmap) data.getExtras (). Get ("data") ;, but it seems to be inconvenient if the number of people increases or if it is a drawing image)
MainActivity.java
getImg = (Bitmap) data.getExtras().get("data");
MainActivity.java
/**Abbreviation**/
imageView.setImageURI(mImageUri);
/**Abbreviation**/
/**
*For photography
*Export the taken photo to URL once, and store the URL of the export destination in mImageUri
*/
protected void takePhoto(){
String filename = System.currentTimeMillis() + ".jpg ";
ContentValues values = new ContentValues();
values.put(MediaStore.Images.Media.TITLE,filename);
values.put(MediaStore.Images.Media.MIME_TYPE, "image/jpeg");
mImageUri = getContentResolver().insert(
MediaStore.Images.Media.EXTERNAL_CONTENT_URI, values
);
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
intent.putExtra(MediaStore.EXTRA_OUTPUT, mImageUri);
startActivityForResult(intent, RESULT_CAMERA);
}
What I want to achieve in the future
・ Class division only for face recognition Initially, I tried to implement it by classifying it, but I gave up because I did not know how to classify the following parts. (Can't access res / raw with the same description as MainActivity.java from another class?)
MainActivity.java
File cascadeDir = getDir("cscade", Context.MODE_PRIVATE);
File cascadeFile = new File(cascadeDir, "lbpcascade_frontalface.xml");
・ Image acquisition from ImageView By acquiring images from ImageVIew, in the case of cameras and galleries, implementation in each case becomes unnecessary, so it may be possible to reduce the code.
・ Implementation in Kotlin I implemented it in Java, which I was familiar with, because I wanted to get used to it for the time being, but I would like to use Kotlin when developing for Android in the future.
Source code
MainActivity.java
package com.example.myapplication;
import androidx.appcompat.app.AppCompatActivity;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import android.content.ContentValues;
import android.content.Context;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.BitmapFactory;
import android.graphics.Color;
import android.graphics.Paint;
import android.net.Uri;
import android.os.Bundle;
import android.provider.MediaStore;
import android.util.Log;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;
import android.widget.Toast;
import org.opencv.android.Utils;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Rect;
import org.opencv.objdetect.CascadeClassifier;
public class MainActivity extends AppCompatActivity {
private final static int RESULT_CAMERA = 1001; //For camera
private final static int REQUEST_GALLERY = 1000; //For gallery
private Uri mImageUri; //Instance variable to store the URL of the image
private Bitmap getImg = null; //Images from the camera or gallery
private ImageView imageView;//Image view declaration
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
imageView = findViewById(R.id.image_view);//First, associate ImageView with the layout view ID
Button cameraButton = findViewById(R.id.camera_button);
cameraButton.setOnClickListener(new View.OnClickListener() {//Implementation using a normal inner class
@Override
public void onClick(View v) {
takePhoto();
}
});
Button galleyButton = findViewById(R.id.gallery_button);
galleyButton.setOnClickListener(new View.OnClickListener() {//Implementation using a normal inner class
@Override
public void onClick(View v) {
Intent intent = new Intent();
intent.setType("image/*");
intent.setAction(Intent.ACTION_GET_CONTENT);
startActivityForResult(intent, REQUEST_GALLERY);
}
});
}
//I will paste the photo taken in ImageView from now on.
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
if(requestCode == RESULT_CAMERA || requestCode == REQUEST_GALLERY) {
TextView textView = findViewById(R.id.text1); //Text view(View to store the text to display)
String outputText = ""; //Text to display
MatOfRect faceRects; //Store face recognition data
//When selecting an image from the camera
if (requestCode == RESULT_CAMERA) {
//Including canceled cases
if (data.getExtras() == null) {
Log.d("debug", "cancel ?");
return;
} else {
imageView.setImageURI(mImageUri);
getImg = (Bitmap) data.getExtras().get("data");
}
}
//When selecting an image from the gallery
if (requestCode == REQUEST_GALLERY && resultCode == RESULT_OK) {
try {
InputStream in = getContentResolver().openInputStream(data.getData());
getImg = BitmapFactory.decodeStream(in);
in.close();
//Display the selected image
imageView.setImageBitmap(getImg);
} catch (Exception e) {
//When there is no file
textView.setText("File not found");
}
}
//Image pasted on imageView(Obtained from camera, gallery)Face recognition from
try {
faceRects = checkFaceExistence(getImg);
outputText = makeOutputText(faceRects);\
//Display the selected image
imageView.setImageBitmap(getImg);
} catch (IOException e) {
outputText = "Face recognition error";
e.printStackTrace();
}
Toast.makeText(this,outputText,Toast.LENGTH_LONG).show();
textView.setText(outputText);
}
}
/**
*For photography
*Export the taken photo to URL once, and store the URL of the export destination in mImageUri
*/
protected void takePhoto(){
String filename = System.currentTimeMillis() + ".jpg ";
ContentValues values = new ContentValues();
values.put(MediaStore.Images.Media.TITLE,filename);
values.put(MediaStore.Images.Media.MIME_TYPE, "image/jpeg");
mImageUri = getContentResolver().insert(
MediaStore.Images.Media.EXTERNAL_CONTENT_URI, values
);
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
intent.putExtra(MediaStore.EXTRA_OUTPUT, mImageUri);
startActivityForResult(intent, RESULT_CAMERA);
}
/**
*Perform face recognition based on the passed image
* @param bitmap Image for face recognition
* @return text Face recognition information
* @throws IOException
*/
protected MatOfRect checkFaceExistence(Bitmap bitmap) throws IOException{
System.loadLibrary("opencv_java4");
//Image data sent(bitmap)To Mat format
Mat matImg = new Mat();
Utils.bitmapToMat(bitmap, matImg);
String text = ""; //Text that stores face recognition information
//Generating a cascade classifier instance for face recognition(Export the file once and get the file path)
//Get the xml file once stored under raw
InputStream inStream = getResources().openRawResource(R.raw.haarcascade_frontalface_alt);
MatOfRect faceRects = new MatOfRect(); //Store face recognition data
try {
//Set the output xml file path as an argument of CascadeClassfle
CascadeClassifier faceDetetcor = outputCascadeFile(inStream);
//Face recognition by giving image data to the cascade classifier
faceDetetcor.detectMultiScale(matImg, faceRects);
}
catch (IOException e){
Toast.makeText(this,"Failed to open the analysis information file.",Toast.LENGTH_SHORT).show();
}
return faceRects;
}
/**
*Import the openCV classifier prepared in advance once and make it available for export.
* @param inStream Original data of classifier
* @return faceDetector classifier data
* @throws IOException
*/
protected CascadeClassifier outputCascadeFile(InputStream inStream) throws IOException {
File cascadeDir = getDir("cscade", Context.MODE_PRIVATE);
File cascadeFile = new File(cascadeDir, "lbpcascade_frontalface.xml");
//Output the acquired xml file to a specific directory
FileOutputStream outputStream = new FileOutputStream(cascadeFile);
byte[] buf = new byte[2048];
int rdBytes;
while ((rdBytes = inStream.read(buf)) != -1) {
try {
outputStream.write(buf, 0, rdBytes);
} catch (IOException e) {
e.printStackTrace();
}
}
//Set the output xml file path as an argument of CascadeClassfle
CascadeClassifier faceDetetcor = new CascadeClassifier((cascadeFile.getAbsolutePath()));
outputStream.close();
return faceDetetcor;
}
/**
*Text creation for output
*Set the coordinate information to text based on the face recognition information
* @param faceRects
* @return
*/
protected String makeOutputText(MatOfRect faceRects){
String text = "";
//Return face recognition result as String
if(faceRects.toArray().length > 0){
for(Rect face: faceRects.toArray()) {
text = "Vertical width of the face:" + face.height + "\n" +
"Width of face" + face.width + "\n" +
"Face position(Y coordinate)" + face.y + "\n" +
"Face position(X coordinate)" + face.x;
}
}
else{
text = "No face was detected.";
}
return text;
}
Recommended Posts