This article is the 21st day article of Microsoft Azure Cognitive Services Advent Calendar 2020 (very late). This is an article for beginners who want to do Ruby and use image analysis by AI in a simple form.

What is Face API?

It is a human feature analysis function by pre-built AI provided by Microsoft Azure. I chose this service while AWS, GCP, etc. are providing cloud-based AI because the "age / gender" function can be used by default.

The summary article of Ledge was very helpful. -> What is face recognition/face detection? Compare the functions, accuracy, and price of 6 face recognition APIs! | Ledge.ai

The Face API demo page is on here, so you can feel free to try what parameters you can get.

By the way, in the past, age and gender were returned in "face attributes" in the response column on the right side, but recently (citation needed), null is returned.

Preparation for use

Let's register Azure from here. I wonder if the flow will be a little easier for those who have already obtained an email address with Office 365 etc. (Unconfirmed)

After registering, we will issue a subscription key to use the Face API.

Go to the Azure Portal (https://portal.azure.com/#home) and "Create a new resource".

The screen for creating a new resource looks like this To create an endpoint for FaceAPI, start from "Face" at the end of the arrow.

Follow the instructions on the screen

--Resource group (Service you want to use: FaceAPI in this case) --Regions (In most cases, it will be East Japan and West Japan, but the endpoints will change for each region.) --Name (Make it unique and unique) --Price level (Free is often sufficient. Please check the pricing in detail.)

You can create it in the flow by setting and pressing "Confirm and Create".

After creating it, when you return to the home of the portal earlier, the resource is created with the set name.

Click this to move to various setting screens. Click the Keys & Endpoints tab to see the keys to skip requests to the API. The endpoints displayed on this page will not be used this time.

Call API in Ruby

The top search hit Japanese documentation contains only samples in Python, C #, Go and REST (curl), but if you dig deeper into the Cognitive Services page, you'll see [in a little more languages]. There is a Documentation with Samples.

There is the following part at the bottom of the page, so copy and paste this and it will be the skeleton as it is.

The full text looks like this.

`Ruby sample`



require 'net/http'

uri = URI('https://japanwest.api.cognitive.microsoft.com/face/v1.0/detect')
uri.query = URI.encode_www_form({
    # Request parameters
    'returnFaceId' => 'true',
    'returnFaceLandmarks' => 'false',
    'returnFaceAttributes' => '{string}', #Change here ①
    'recognitionModel' => 'recognition_03',
    'returnRecognitionModel' => 'false',
    'detectionModel' => 'detection_02', #Change here ②
    'faceIdTimeToLive' => '86400'
})

request = Net::HTTP::Post.new(uri.request_uri)
# Request headers
request['Content-Type'] = 'application/json'
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}' #Change here ③
# Request body
request.body = "{body}" #Change here ④

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body

If you change the "change here" part in the code, you will be able to obtain the "age and gender" that is the purpose of this time as a response.

What kind of request do you want to skip

Before explaining how to change each value, you can see by looking at the sample of "Curl", but the request to be sent to the API takes the form of uri. The sample document has a console like the demo mentioned at the beginning of this article, where you can create a uri to throw with Curl.

In the Ruby code above, various character strings are entered as query parameters in the contents of uri.query.

Based on that, the uri actually sent is as follows.

https://japanwest.api.cognitive.microsoft.com/face/v1.0/detect?returnFaceId=true&returnFaceLandmarks=false&recognitionModel=recognition_03&returnRecognitionModel=false&detectionModel=detection_01&faceIdTimeToLive=86400

Note that this parameter is also actually thrown as a request on the console page, and if you enter the correct value, 200 ok will be returned.

Also, the endpoint to be specified is japanwest.api.cognitive.microsoft.com, but if the East Japan region is specified, it will be japaneast.api.cognitive.microsoft.com.

Change here ①

'returnFaceAttributes' => '{string}'

In this FaceAttributes, enter the attributes you want to return (= attributes you want to analyze) as a response.

For example, if you want to return "age and gender"

'returnFaceAttributes' => 'age,gender'

It would be good to do. In addition, if you specify accessories, you will be returned whether you are wearing masks or glasses, and if you specify make up, you will be returned whether you are wearing makeup or not. Read the documentation for more details.

Change here ②

'detectionModel' => 'detection_02'

Change the above detection_02 to detection_01.

To use the identification function such as sentiment analysis and age, use recognition_03 with the same query parameter recognitionModel, but at the moment, detection_02, which is a new human face detection model, is recognition_03. It does not correspond to. Therefore, use detection_01.

Change here ③

request['Ocp-Apim-Subscription-Key'] = '{subscription key}'

The Ocp-Apim-Subscription-Key is specified as" Key 1 "mentioned in the * Preparation for use * item, and this is used as the subscription key. If you just want to run it locally, you can write it as it is, but if you want to use it as a remote resource, you can describe it in a yaml file or add a mechanism to manage environment variables and refer to it outside the file. It is recommended to do so.

This is the end of preparation.

Change here ④

request.body = "{body}"

For body, you can specify the URL of the image or jpeg expanded as a binary. In this example, as a sample, I will borrow an image of Matz and Rocketta (Mr. Naruse and Mr. Shimizu) from the article by PR Times.

Image address: https://prtimes.jp/i/35716/3/resize/d35716-3-604425-6.jpg

If you want to send a local file

`Ruby`


file_path = ARGV.shift
file_data = File.open(file_path, "r+b")
request.body = file_data.read
file_data.close

I think it would be better to describe it as.

What kind of response will be returned?

After making the above preparations, if you execute the source code locally,

When the request is returned in json format and parsed as hash,

{"faceId"=>"a7d7d489-b657-4149-a2d8-e7b1b8af9aa5", "faceRectangle"=>{"top"=>69, "left"=>438, "width"=>62, "height"=>62}, "faceAttributes"=>{"gender"=>"male", "age"=>33.0}}
{"faceId"=>"b07990e7-d0b3-4030-bd2a-41bbcf86de7d", "faceRectangle"=>{"top"=>71, "left"=>126, "width"=>58, "height"=>58}, "faceAttributes"=>{"gender"=>"male", "age"=>23.0}}
{"faceId"=>"26876e61-d6fb-482c-b43b-3e7d4410df52", "faceRectangle"=>{"top"=>93, "left"=>286, "width"=>57, "height"=>57}, "faceAttributes"=>{"gender"=>"male", "age"=>44.0}}

It can be obtained in the form of.

Since the position information where the human face was detected is also obtained, drawing a rectangle based on these will be as follows.

The age of matz was 44, and he was about 10 years younger at the time of shooting. Hmmm, many engineers are young in appearance ...

Other points

It is more or less biased by the trained dataset. I haven't been able to dig deeper into the evaluations from documentation and technicians that image recognition and machine learning are not my specialty, or if the recognition model is also racially biased, it's East Asian. If it's a facial feature, it may tend to be younger. In addition, although gender is binary for men and women, there are cases where men with long hair and hidden contours are easily identified as women. These can also return the degree of estimation as a percentage, so if you can use it well and make a judgment with AI, I think that it will be an opportunity to acquire new knowledge.

At the end

The details of the services provided, including the UI, change quite frequently on this type of platform, so it is recommended that you search for the latest information as much as possible. (Preaching to Buddha)

Samples and Documents is available on Github, so please refer to it as well.

I borrowed the image sample used in the article from here. URL: https://prtimes.jp/main/html/rd/p/000000003.000035716.html All copyrights and portrait rights belong to the owner of the original image.

Congratulations on the release of Ruby 3.0.0! !! !! (2020/12/25)

Use Face API from Ruby