Let's move OpenCv a little. mac or linux
opencv
A standard library for image processing. Image processing is interesting. I want to do AR VR. The AR function is certainly an extension, so the installation method is different. I have to build from scratch. Moreover, it is troublesome that the settings are different for each extension function. Therefore, I will omit it this time.
This unity sample is a paid asset. The accuracy is poor because the parts such as eyes are estimated by the hog + cascade of opencv.
Rectangle detection
https://github.com/shibatch/rectdetect https://www.youtube.com/watch?v=HpaY0HOomAI https://www.youtube.com/watch?v=BLJEYui0XcY
Detected in motion
https://shibata.naist.jp/~n-sibata/software/baum/ https://www.youtube.com/watch?v=KtO5TxG4T0M
opencv videos http://miyamotok0105.hatenablog.com/entry/2017/03/12/182056
Opencv operating environment
mac linux win (I will omit this time.)
opencv language environment
I think it works in more languages, but I'll focus on the following. I can write it in c language, but in my case I had to use the g ++ compiler. However, it is possible to use the source written in C language.
C http://opencv.jp/opencv-2svn/c/ http://opencv.jp/opencv-2svn_org/c/ C++ http://opencv.jp/opencv-2svn/cpp/ http://opencv.jp/opencv-2svn_org/cpp/ python http://opencv.jp/opencv-2svn/py/ http://opencv.jp/opencv-2svn_org/py/
With opencv2 and opencv3
http://tatsyblog.sakura.ne.jp/wordpress/programming/cpp/662/
This area seems to have changed. cv::imread cv::cvtColor cv::TermCriteria cv::VideoCapture::get cv::PCA
Python development environment
Python installation (mac, linux)
There were a lot of great humors, so make a note of the shortest. For those who are python, it is the shortest to enter with conda. I haven't written about conda here. If you want to build from scratch, you should do it when you have a little time.
For conda
conda install -c menpo opencv=2.4.11
or
conda install -c menpo opencv3=3.1.0
For build
It takes a lot of time.
git clone https://github.com/Itseez/opencv.git
cd opencv
mkdir build
cd build
cmake ..
make -j4 or make -j8
sudo make install
I played with options when I was sick with azure.
cmake -D WITH_OPENMP=ON -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_opencv_python2=Off -D BUILD_opencv_python3=On -D PYTHON2_EXECUTABLE="" -D PYTHON2_INCLUDE_DIR="" -D PYTHON2_LIBRARY="" -D PYTHON2_PACKAGES_PATH="" -D OPENCV_FORCE_PYTHON_LIBS=On -D PYTHON3_PACKAGES_PATH=$PYTHON3_PACKAGES_PATH -D CMAKE_INSTALL_PREFIX=$LOCAL_PREFIX -D WITH_CUDA=Off -D BUILD_opencv_photo=OFF -D BUILD_opencv_shape=OFF -D BUILD_opencv_video=OFF -D ENABLE_NEON=OFF -D WITH_DC1394=ON -D WITH_FFMPEG=OFF ..
Uninstall python
If it doesn't work, I may put it in and erase it several times. In some cases, several versions overlapped, or in some cases it was better to erase them all once. Please be aware that the version may be out of date.
For conda
If there are many, erase them all
conda uninstall -c menpo opencv=2.4.11
conda uninstall opencv
conda uninstall opencv3
Check if everything has disappeared
conda list | grep opencv
For build
sudo make uninstall
Try running with Python
Operating environment you are trying
pyenv
anaconda
python2.7
mac
Camera caption
If you have a computer with a camera, your face will suddenly appear. If you don't have numpy, put it in.
pip install numpy
sample.py
# -*- coding: utf-8 -*-
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# Display the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Haar-Like features
Cut out a part of the image and calculate the local difference in brightness.
Categorize categories based on their characteristics.
Video of algorithm image. You can see it by looking at it. https://vimeo.com/12774628
Get xml from git and specify. https://github.com/opencv/opencv/blob/master/data/haarcascades
| filename | Contents |
|---|---|
| haarcascade_eye.xml | eyes |
| haarcascade_eye_tree_eyeglasses.xml | glasses |
| haarcascade_frontalcatface.xml | Cat face (front) |
| haarcascade_frontalcatface_extended.xml | Cat face (front) |
| haarcascade_frontalface_alt.xml | Face (front) |
| haarcascade_frontalface_alt2.xml | Face (front) |
| haarcascade_frontalface_alt_tree.xml | Face (front) |
| haarcascade_frontalface_default.xml | Face (front) |
| haarcascade_fullbody.xml | whole body |
| haarcascade_lefteye_2splits.xml | left eye |
| haarcascade_licence_plate_rus_16stages.xml | Russian license plate (whole) |
| haarcascade_lowerbody.xml | Lower body |
| haarcascade_profileface.xml | Face (ID photo) |
| haarcascade_righteye_2splits.xml | right eye |
| haarcascade_russian_plate_number.xml | Russian license plate (number) |
| haarcascade_smile.xml | Smile |
| haarcascade_upperbody.xml | upper body |
sample.py
#python2
import cv2
import sys
cascPath = sys.argv[1]
faceCascade = cv2.CascadeClassifier(cascPath)
video_capture = cv2.VideoCapture(0)
while True:
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.CASCADE_SCALE_IMAGE
)
# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Display the resulting frame
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
Template matching
Compare the two images to see if they look similar. Almost perfect match is required, but some accuracy is obtained. If it doesn't work, see ↓.
http://answers.opencv.org/question/187096/why-opencv-matchtemplate-api-not-detect-all-occurrence-of-image/
In addition to this, there is a method of extracting features and specifying the location.
Matching combined with deep learning is also interesting as a research field.
https://arxiv.org/pdf/1705.08593.pdf https://ricardodeazambuja.com/deep_learning/2017/03/05/easy-peasy_conv_deep_learning_two/ https://github.com/sunsided/convolution-template-matching https://www.jstage.jst.go.jp/article/transinf/E100.D/1/E100.D_2016EDP7233/_pdf http://cs231n.stanford.edu/reports/2017/pdfs/817.pdf
LBP features
Features are calculated from the histogram.
https://pebbie.wordpress.com/2011/11/10/local-binary-pattern-in-opencv-python/
sample.py
import cv
def calc_lbp(im):
"""
calculate LBP (Local Binary Pattern) image N8 neighborhood
"""
sz = cv.GetSize(im)
gr = cv.CreateImage(sz, 8, 1)
lbp = cv.CreateImage(sz, 8, 1)
#convert to grayscale
cv.CvtColor(im, gr, cv.CV_BGR2GRAY)
LBPMASK = [(0,-1),(1,-1),(1,0),(1,1),(0,1),(-1,-1),(-1,0),(-1,1)]
for y in xrange(1, sz[1]-2):
for x in xrange(1, sz[0]-2):
n = 0
gv = gr[y,x]
for i in xrange(len(LBPMASK)):
m = LBPMASK[i]
if gr[y+m[1], x+m[0]]>gv:
n += 1 << i
lbp[y,x] = n
return lbp
if __name__ == '__main__':
im = cv.LoadImage('jalan2.jpg')
lbpim = calc_lbp(im)
cv.ShowImage('lbp', lbpim)
key = cv.WaitKey(0)
HOG features
Feature calculation based on the distribution of brightness in the gradient direction.
sample.py
import cv2
converter = cv2.HOGDescriptor()
img = cv2.imread('test.png')
hog = hog.compute(img)
Make a cascade classifier
opencv_createsamples Make the correct image into one vector file. It is written that it is good to have 7,000 or more correct images and 3000 or more incorrect images, but it seems that it depends. http://kivantium.hateblo.jp/entry/2015/05/13/153122 https://www.pro-s.co.jp/engineerblog/opencv/post_6397.html
opencv_createsamples -info train.dat -vec train.vec -num 1000
opencv_traincascade Machine learning with Haar-Like features, LBP features, or HOG features. Exhale xml file. After that, you can read xml and use it.
opencv_traincascade -data cascade/ -vec train.vec -bg bg.dat -numPos 900 -numNeg 1000 -featureType LBP -mode ALL
Play the video
As for the video, I searched for my favorite video, converted the format to mp4 with ffmpeg, and lowered the frame. http://qiita.com/miyamotok0105/items/6de05e5a13e7ffd456fc
sample.py
# -*- coding: utf-8 -*-
import numpy as np
import cv2
cap = cv2.VideoCapture('sample.mp4')
fps = 15
size = (640,480)
cap.set(3, size[0]) # Width
cap.set(4, size[1]) # Heigh
cap.set(5, fps) # FPS
while(cap.isOpened()):
ret, frame = cap.read()
# gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Example using VideoWriter
The specification of OpenCV video loading is that "the latest image frames are newly loaded into the stream one by one".
# -*- coding: utf-8 -*-
#!/usr/bin/python
import cv2
import time
class Camera():
# Constructor...
def __init__(self):
self.cap = cv2.VideoCapture(0) # Prepare the camera...
print("Camera warming up ...")
time.sleep(1)
# if self.cap.isOpened():
# print(self.cap.get(3))
# print(self.cap.get(4))
w = int(self.cap.get(3)) # Frame width...
h = int(self.cap.get(4)) # Frame hight...
fps = 20.0 # Frames per second...
resolution = (w, h) # Frame size/resolution...
# Prepare Capture
self.ret, self.frame = self.cap.read()
# Prepare output window...
self.winName = "Motion Indicator"
cv2.namedWindow(self.winName, cv2.WINDOW_AUTOSIZE)
# Read three images first...
self.prev_frame = cv2.cvtColor(self.cap.read()[1],cv2.COLOR_RGB2GRAY)
self.current_frame = cv2.cvtColor(self.cap.read()[1],cv2.COLOR_RGB2GRAY)
self.next_frame = cv2.cvtColor(self.cap.read()[1],cv2.COLOR_RGB2GRAY)
# Define the codec and create VideoWriter object
self.fourcc = cv2.VideoWriter_fourcc(*'H264') # You also can use (*'XVID')
self.out = cv2.VideoWriter('output.avi',self.fourcc, fps, (w, h), True)
# Frame generation for Browser streaming wiht Flask...
def get_frame(self):
self.frames = open("stream.jpg ", 'w+')
s, img = self.cap.read()
if s: # frame captures without errors...
cv2.imwrite("stream.jpg ", img) # Save image...
return self.frames.read()
def diffImg(self, tprev, tc, tnex):
# Generate the 'difference' from the 3 captured images...
Im1 = cv2.absdiff(tnex, tc)
Im2 = cv2.absdiff(tc, tprev)
return cv2.bitwise_and(Im1, Im2)
def captureVideo(self):
# Read in a new frame...
self.ret, self.frame = self.cap.read()
# Image manipulations come here...
# This line displays the image resulting from calculating the difference between
# consecutive images...
diffe = self.diffImg(self.prev_frame, self.current_frame, self.next_frame)
cv2.imshow(self.winName,diffe)
# Put images in the right order...
self.prev_frame = self.current_frame
self.current_frame = self.next_frame
self.next_frame = cv2.cvtColor(self.frame, cv2.COLOR_RGB2GRAY)
return()
def saveVideo(self):
# Write the frame...
self.out.write(self.frame)
return()
def __del__(self):
self.cap.release()
cv2.destroyAllWindows()
self.out.release()
print("Camera disabled and all output windows closed...")
return()
def main():
# Create a camera instance...
cam1 = Camera()
while(True):
# Display the resulting frames...
cam1.captureVideo() # Live stream of video on screen...
cam1.saveVideo() # Save video to file 'output.avi'...
if cv2.waitKey(1) & 0xFF == ord('q'):
break
return()
if __name__=='__main__':
main()
Check with a browser
Mainly when running opencv plus http server in a terminal such as Raspberry Pi, you can create a server to check the status using the following. It's best to run it in the background using the screen or tmux commands, but you can put it on nginx or apache. Personally, apache is for beginners.
Example using BaseHTTPServer and SocketServer
#!/usr/bin/python
'''
Author:
A Simple mjpg stream http server
use python2
python simple_mjeg_streamer_...py
then see down
http://127.0.0.1:8080/cam.mjpg
'''
import cv2
from PIL import Image
import threading
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
from SocketServer import ThreadingMixIn
import StringIO
import time
capture=None
class CamHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.endswith('.mjpg'):
self.send_response(200)
self.send_header('Content-type','multipart/x-mixed-replace; boundary=--jpgboundary')
self.end_headers()
while True:
try:
rc,img = capture.read()
if not rc:
continue
imgRGB=cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
jpg = Image.fromarray(imgRGB)
tmpFile = StringIO.StringIO()
jpg.save(tmpFile,'JPEG')
self.wfile.write("--jpgboundary")
self.send_header('Content-type','image/jpeg')
self.send_header('Content-length',str(tmpFile.len))
self.end_headers()
jpg.save(self.wfile,'JPEG')
time.sleep(0.05)
except KeyboardInterrupt:
break
return
if self.path.endswith('.html'):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write('<html><head></head><body>')
self.wfile.write('<img src="http://127.0.0.1:8080/cam.mjpg"/>')
self.wfile.write('</body></html>')
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
def main():
global capture
capture = cv2.VideoCapture(0)
# capture.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH, 320);
# capture.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT, 240);
# capture.set(cv2.cv.CV_CAP_PROP_SATURATION,0.2);
global img
try:
server = ThreadedHTTPServer(('localhost', 8080), CamHandler)
print("server started")
server.serve_forever()
except KeyboardInterrupt:
capture.release()
server.socket.close()
if __name__ == '__main__':
main()
Example using flask
Store index.html under the templates folder. Use the common flask structure.
index.html
<html>
<head>
<title>Video Streaming Demonstration</title>
</head>
<body>
<h1>Video Streaming Demonstration</h1>
<img src="{{ url_for('video_feed') }}">
</body>
</html>
# -*- coding: utf-8 -*-
#python2
from flask import Flask, render_template, Response
#from camera import VideoCamera
from time import time
from PIL import Image
import cv2
import numpy as np
class Camera(object):
def __init__(self, cap):
rc,img = cap.read()
def get_frame(self):
self.frames = open("stream.jpg ", 'w+')
s, img = cap.read()
if s: # frame captures without errors...
cv2.imwrite("stream.jpg ", img) # Save image...
return self.frames.read()
cap=cv2.VideoCapture(0)
imagen = cap.read()
app = Flask(__name__)
@app.route('/')
def index():
return render_template('index.html')
def gen(camera):
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')
@app.route('/video_feed')
def video_feed():
return Response(gen(Camera(cap)),
mimetype='multipart/x-mixed-replace; boundary=frame')
if __name__ == '__main__':
app.run(host='0.0.0.0', port=8080, debug=True, threaded=False)
Transfer from browser to browser
A library that makes it easy to use p2p. Clients can communicate with each other without the need for a server. In detail, it seems that there are cases where it can be connected and cases where it cannot be connected, but this time I will not go into too much depth. It's interesting because there are a lot of samples.
WebRTC https://webrtc.github.io/samples/ https://tech-sketch.jp/2014/07/webrtcpeerjs1.html
C ++ preferences
C ++ installation (mac)
brew tap homebrew/science/
brew update
brew install opencv3 --with-contrib --with-python
brew install pkg-config
Installed below /usr/local/Celler/opencv3
cd /usr/local/Cellar/opencv3/3.1.0_4/lib/pkgconfig
sudo cp opencv.pc /usr/local/lib/pkgconfig/opencv.pc
Copy opencv.pc under pkgconfig.
Display image
python
#include <stdio.h>
#include <opencv2/opencv.hpp>
//g++ `pkg-config --cflags opencv` `pkg-config --libs opencv` 001.cpp -o 001
using namespace cv;
int main(int argc, char** argv ) {
Mat image;
image = imread("../../img/hari.jpeg ", 1 );
if ( !image.data ) {
printf("No image data \n");
return -1;
}
namedWindow("Display Image", WINDOW_AUTOSIZE );
imshow("Display Image", image);
waitKey(0);
return 0;
}
g++ `pkg-config --cflags opencv` `pkg-config --libs opencv` 001.cpp -o 001
# OpenCV 2.2 cheat sheet (C ++) It is for letting you know what kind of function it has.
An important OpenCV class
--Point 2D point class template --Point3 3D point class template --Size Size (width, height) Class template --Vec short vector class template --Matx small matrix class template --Scalar 4-Element vector --Rect rectangle --Range Range represented by an integer value --Mat 2D or multidimensional dense distribution (can store matrices, images, histograms, feature descriptors, voxel volumes, etc.) --SparseMat Multidimensional sparse array --Ptr smart pointer class template
Matrix basics
Creating a matrix
Mat image(240, 320, CV 8UC3);
(Re) placement of declared matrices
image.create(480, 640, CV 8UC3);
Initialize the matrix with a constant
Mat A33(3, 3, CV 32F, Scalar(5));
Mat B33(3, 3, CV 32F); B33 = Scalar(5);
Mat C33 = Mat::ones(3, 3, CV 32F)*5.;
Mat D33 = Mat::zeros(3, 3, CV 32F) + 5.;
Initialize the matrix with a specific value
double a = CV PI/3;
Mat A22 = (Mat <float>(2, 2) <<
cos(a), -sin(a), sin(a), cos(a));
float B22data[] = {cos(a), -sin(a), sin(a), cos(a)};
Mat B22 = Mat(2, 2, CV 32F, B22data).clone();
Initialize the matrix with random numbers
randu(image, Scalar(0), Scalar(256)); //Uniform distribution
randn(image, Scalar(128), Scalar(10)); //normal distribution
Convert matrices and other structures to each other
(No data copy)
Mat image alias = image;
float* Idata=new float[480*640*3];
Mat I(480, 640, CV 32FC3, Idata);
vector<Point> iptvec(10);
Mat iP(iptvec); // iP – 10x1 CV 32SC2 matrix
IplImage* oldC0 = cvCreateImage(cvSize(320,240),16,1);
Mat newC = cvarrToMat(oldC0);
IplImage oldC1 = newC; CvMat oldC2 = newC;
... (with data copy)
Mat newC2 = cvarrToMat(oldC0).clone();
vector<Point2f> ptvec = Mat <Point2f>(iP);
Access matrix elements
A33.at<float>(i,j) = A33.at<float>(j,i)+1;
Mat dyImage(image.size(), image.type());
for(int y = 1; y < image.rows-1; y++) {
Vec3b* prevRow = image.ptr<Vec3b>(y-1);
Vec3b* nextRow = image.ptr<Vec3b>(y+1);
for(int x = 0; y < image.cols; x++)
for(int c = 0; c < 3; c++)
dyImage.at<Vec3b>(y,x)[c] = saturate cast<uchar>(nextRow[x][c] - prevRow[x][c]);
}
Mat <Vec3b>::iterator it = image.begin<Vec3b>(),
itEnd = image.end<Vec3b>();
for(; it != itEnd; ++it)
(*it)[1] ^= 255;
Matrix operations: copy, shuffle, submatrix
--src.copyTo (dst) Copy to another matrix --src.convertTo (dst, type, scale, shift) Scaling and conversion to another data type --m.clone () Deep copy of the matrix --m.reshape (nch, narrows) Change matrix dimensions and number of channels without copying data --m.row (i), m.col (i) Matrix rows and columns
- m.rowRange(Range(i1,i2)) m.colRange (Range (j1, j2)) Matrix rows and columns (range specification) --m.diag (i) Diagonal elements of the matrix --m (Range (i1, i2), Range (j1, j2)), m (roi) submatrix --m.repeat (ny, nx) Create a large matrix from a small matrix --flip (src, dst, dir) Invert the order of rows (or columns) in a matrix --split (...) Split multi-channel matrix for each channel --merge (...) Create a multi-channel matrix from the matrix for each channel --mixChannels (...) General types of split () and merge () --randShuffle (...) Randomly shuffle matrix elements
Example 1. Smooth the image ROI
Mat imgroi = image(Rect(10, 20, 100, 100));
GaussianBlur(imgroi, imgroi, Size(5, 5), 1.2, 1.2);
Example 2. Linear algebra algorithm
m.row(i) += m.row(j)*alpha;
Example 3. Copy the image ROI to another image while converting
Rect r(1, 1, 10, 20);
Mat dstroi = dst(Rect(0,10,r.width,r.height));
src(r).convertTo(dstroi, dstroi.type(), 1, 0);
Simple matrix operation
There are many common matrix operations, logical operations, and other operations in OpenCV. Is implemented. For example • add(), subtract(), multiply(), divide(), absdiff(), bitwise and(), bitwise or(), bitwise xor(), max(), min(), compare()
– Sum, difference, multiplication for each element ... An operation equivalent to comparing two matrices or comparing a matrix and a scalar.
Example. Alpha composition function:
void alphaCompose(const Mat& rgba1, const Mat& rgba2, Mat& rgba dest)
{
Mat a1(rgba1.size(), rgba1.type()), ra1;
Mat a2(rgba2.size(), rgba2.type());
int mixch[]={3, 0, 3, 1, 3, 2, 3, 3};
mixChannels(&rgba1, 1, &a1, 1, mixch, 4);
mixChannels(&rgba2, 1, &a2, 1, mixch, 4);
subtract(Scalar::all(255), a1, ra1);
bitwise or(a1, Scalar(0,0,0,255), a1);
bitwise or(a2, Scalar(0,0,0,255), a2);
multiply(a2, ra1, a2, 1./255);
multiply(a1, rgba1, a1, 1./255);
multiply(a2, rgba2, a2, 1./255);
add(a1, a2, rgba dest);
}
• sum(), mean(), meanStdDev(), norm(), countNonZero(),minMaxLoc(), – Various statistics of matrix elements. • exp(), log(), pow(), sqrt(), cartToPolar(), polarToCart() – Classic arithmetic function. • scaleAdd(), transpose(), gemm(), invert(), solve(),determinant(), trace() eigen(), SVD, – Algebraic function + SVD class. • dft(), idft(), dct(), idct(), – Discrete Fourier transform, discrete cosine transform Some processes can use even more convenient algebraic notation. For example:
Mat delta = (J.t()*J + lambda*
Mat::eye(J.cols, J.cols, J.type()))
.inv(CV SVD)*(J.t()*err); This is the cornerstone of the Le Levenberg-Marquardt optimization algorithm.
Image processing
filtering
--filter2D () Non-separable linear filter --septFilter2D () Separation line filter --boxFilter (), GaussianBlur (), medianBlur (), bilateralFilter () Smoothing images with linear or non-linear filters --Sobel (), Scharr () Calculation of differential images --Laplacian () Laplacian calculation --erode (), dilate () Image contraction and expansion
Example. Apply a 3x3 highpass filter (Add 128 to the whole so that no negative values are lost):
filter2D(image, image, image.depth(), (Mat <float>(3,3)<<
-1, -1, -1, -1, 9, -1, -1, -1, -1), Point(1,1), 128);
Geometric transformation
--resize () Resize image --getRectSubPix () Extract image patch --warpAffine () Affine transformation of images --warpPerspective () Perspective transformation of images --remap () General-purpose image conversion --Map optimization for fast conversionMaps () remap ()
Example. Reduce image to half root:
Mat dst; resize(src, dst, Size(), 1./sqrt(2), 1./sqrt(2));
Various image conversions
--cvtColor () Convert image to another color space --threshold (), adaptive threshold () Convert grayscale image to binary image using fixed or variable threshold --Detection of connected components using floodFill () region expansion algorithm --Integral () Calculation of integral image --DistanceTransform () Construction of distance image or Voronoi diagram for binary image --Refer to the samples of watershed (), grabCut () marker-based image segmentation algorithm watershed.cpp and grabcut.cpp.
histogram
--calcHist () Image Histogram Calculation --calcBackProject () Histogram back projection --equalizeHist () Normalize image brightness and contrast --compareHist () Compare two histograms
Example. Calculate the hue-saturation histogram of an image:
Mat hsv, H; MatND tempH;
cvtColor(image, hsv, CV BGR2HSV);
int planes[]={0, 1}, hsize[] = {32, 32};
calcHist(&hsv, 1, planes, Mat(), tempH, 2, hsize, 0);
H = tempH;
Contour
See the samples in contours.cpp and squares.cpp for the meaning and usage of contours. Data input / output XML / YAML storage is a (nested) collection that can contain scalar values, structs, and various lists.
Writing data to YAML (or XML)
// File type is determined by extension
FileStorage fs("test.yml", FileStorage::WRITE);
fs << "i" << 5 << "r" << 3.1 << "str" << "ABCDEFGH";
fs << "mtx" << Mat::eye(3,3,CV 32F);
fs << "mylist" << "[" << CV PI << "1+1" <<
"{:" << "month" << 12 << "day" << 31 << "year"
<< 1969 << "}" << "]";
fs << "mystruct" << "{" << "x" << 1 << "y" << 2 <<
"width" << 100 << "height" << 200 << "lbp" << "[:";
const uchar arr[] = {0, 1, 1, 0, 1, 1, 0, 1};
fs.writeRaw("u", arr, (int)(sizeof(arr)/sizeof(arr[0])));
fs << "]" << "}";
Scalar values (integers, floating point numbers, strings), matrices, scalars using the << operator Writing empty values and other types of STL vectors to file storage I can Read data
//The file type is determined by the content
FileStorage fs("test.yml", FileStorage::READ);
int i1 = (int)fs["i"]; double r1 = (double)fs["r"];
string str1 = (string)fs["str"];
Mat M; fs["mtx"] >> M;
FileNode tl = fs["mylist"];
CV Assert(tl.type() == FileNode::SEQ && tl.size() == 3);
double tl0 = (double)tl[0]; string tl1 = (string)tl[1];
int m = (int)tl[2]["month"], d = (int)tl[2]["day"];
int year = (int)tl[2]["year"];
FileNode tm = fs["mystruct"];
Rect r; r.x = (int)tm["x"], r.y = (int)tm["y"];
r.width = (int)tm["width"], r.height = (int)tm["height"];
int lbp val = 0;
FileNodeIterator it = tm["lbp"].begin();
for(int k = 0; k < 8; k++, ++it)
lbp val |= ((int)*it) << k;
Read the scalar value using the FileNode cast operator. Procession and so Other types are read using the >> operator. When reading the list Use FileNodeIterator. Writing and reading raster images
imwrite("myimage.jpg ", image);
Mat image color copy = imread("myimage.jpg ", 1);
Mat image grayscale copy = imread("myimage.jpg ", 0);
Formats that these functions can read and write: BMP (.bmp), JPEG (.jpg, .jpeg), TIFF (.tif, .tiff), PNG (.png), PBM/PGM/PPM (.p? m), Sun Raster (.sr), JPEG 2000 (.jp2). Each format is Supports 8-bit, 1- or 3-channel images. 1 channel Also supports formats (PNG, JPEG 2000) that support 16-bit images There is. Load images from video files and cameras
VideoCapture cap;
if(argc > 1) cap.open(string(argv[1])); else cap.open(0);
Mat frame; namedWindow("video", 1);
for(;;) {
cap >> frame; if(!frame.data) break;
imshow("video", frame); if(waitKey(30) >= 0) break;
}
Simple GUI (highgui module)
--nameWindow (winname, flags) Creates a named highgui window --destroyWindow (winname) Destroys the specified window --imshow (winname, mtx) Show the image in the window --waitKey (delay) Waits for the key to be pressed for the specified time (or forever). Event processing is performed during the waiting time. Call this function several times a second Don't forget. --createTrackbar (...) Adds a trackbar (slider) to the specified window. --setMouseCallback (...) Sets a callback for mouse clicks and movements within the specified window. For how to use GUI functions, see camshiftdemo.cpp and OpenCV. See samples.
Camera calibration, attitude estimation, depth estimation
--calibrateCamera () Calibrate the camera using multiple images of the calibration pattern. --findChessboardCorners () Detects feature points on the checkerboard calibration pattern. --solvPnP () Find the attitude of the original object from the result of projecting the feature points. --stereoCalibrate () Calibrate the stereo camera. --stereoRectify () Performs parallelization of calibrated stereo camera images. --initUndistortRectifyMap () Calculates the parallelization map (for remap ()) for each stereo camera. --StereoBM, StereoSGBM This is a stereo correspondence point search engine that is executed on a parallelized stereo pair. --reprojectImageTo3D () Converts the parallax map into a 3D point cloud. --findHomography () Find the optimal perspective transformation between two-dimensional point sets.
Sample calibration.cpp and stereo calib.cpp can be used to calibrate the camera. Use the sample stereo match.cpp to get a parallax map and a 3D point cloud.
Object detection
--matchTemplate Finds the match map for the input template. --CascadeClassifier A cascade of boosting classifiers that uses Haar and LBP features proposed by Viola. See facedetect.cpp. --HOGDescriptor N. Dalal's HOG features It is a body detector. See peopledetect.cpp Please.
Anxious page
Send video to server with node https://github.com/na2hiro/webrtc-distribution Speed up opencv streaming http://nobutobook.blogspot.jp/2016/10/python-opencv.html face tracking https://01.org/developerjourney/face-tracking https://realpython.com/blog/python/face-detection-in-python-using-a-webcam/ http://manmade2.com/simple-home-surveillance-with-opencv-python-and-raspberry-pi/ Python live browser surveillance with motion detection https://github.com/Arri/Py_WebSurveillance streamming https://svds.com/streaming-video-analysis-python/ motion track https://github.com/svetlichnaya/Motion-Tracking Convert from rgb to low color hight color https://henrydangprg.com/2016/06/26/color-detection-in-python-with-opencv/
reference
https://www.pro-s.co.jp/engineerblog/opencv/post_6231.html http://qiita.com/hitomatagi/items/04b1b26c1bc2e8081427 http://derivecv.tumblr.com/post/73561473978 http://opencv.jp/opencv-2svn/opencv_cheatsheet.pdf Around the C ++ environment http://www2.meijo-u.ac.jp/~kohara/cms/technicalreport/opencv_intall4mac http://purple-apple.hatenadiary.jp/entry/2017/02/05/195109
Recommended Posts