Implement Non Maximum Suppression used for object detection.
Introduced here are the Intersection over Union and Non Maximum Suppression used in Computer Vision: Object Detection Part2-Single Shot Multi Detector.
Non Maximum Suppression (NMS) In object detection, as shown in the figure below, multiple candidate regions that are presumed to have an object may be obtained for one recognized object.
The part surrounded by the colored rectangle points to the candidate area, and the display on the upper left of each candidate area shows the category and certainty of the objects existing in the candidate area.
Non Maximum Suppression is a process that wants to keep only the candidate areas with the highest certainty among the multiple candidate areas obtained, that is, suppresses the ones that are not the maximum.
Now that the purpose of narrowing down the candidate areas has been decided, let's think about how to achieve it.
Intersection over Union (IoU) Non Maximum Suppression performs threshold processing based on Intersection over Union (IoU), which is an index for quantifying the degree of overlap, between the candidate region with the maximum object certainty and other candidate regions.
Considering the two candidate areas as shown in the figure below, IoU can be calculated from the following formula.
IoU = \frac{a \cap b}{a \cup b}
The symbols $ a \ cap b $ and $ a \ cup b $ used in set theory represent the colored areas in the figure below, respectively. When implementing, calculate the IoU by calculating the area of each candidate area, intersection, and total area.
IoU takes a value of [0, 1], the larger the overlap, the closer the value is to 1, and the smaller the overlap, the smaller the IoU value. Therefore, set the threshold value in advance and delete the candidate area with IoU larger than the threshold value. If the IoU value is less than the threshold, it is considered that another object is being detected and the candidate area is left.
-CPU Intel (R) Core (TM) i7-6700 4.00GHz
・ Windows 10 Pro 1909 ・ Python 3.6.6 ・ Numpy 1.17.3 ・ Opencv-contrib-python 4.1.2.30
The implemented program is published on GitHub.
non_maximum_suppression.py
When the threshold was set to 0.1 and run, the many initially displayed candidate areas were narrowed down to one each.
Computer Vision : Object Detection Part2 - Single Shot Multi Detector
Tatsuya Harada. "Machine Learning Professional Series Image Recognition", Kodansha, 2017.