Introduction

Custom Vision Service is an image recognition service provided as one of the products of Microsoft Azure.

It is a service that tells you how close an image is to which tag by uploading a tag indicating the type of image and several images associated with it and learning it like this.

In the above example, after learning the violin, oboe, and tuba, and uploading a photo of a violin with a slightly different composition, it is recognized that there is a 99.9% chance that the photo is a violin.

This time, I will try to execute (one of) the API of this service from Python.

Overall flow

The general flow of using this service is

Register images for learning with tags
Perform learning
Upload the image to be identified and get the result

It is a very simple thing, and even a person like me who does not do machine learning or image processing at all can somehow easily move it. Of these, the following articles were very helpful for how to execute 1, 2 and 3 on the browser, so please refer to them.

I tried using Custom Vision Service --Qiita

This article describes how to do part 3 in Python.

Main subject

However, basically you just write and execute the program according to the reference.

Overview - Custom Vision Service

If you open the left menu "Reference"> "Custom Vision Prediction API Reference" on this page, you will see the Testing Console and the page with sample code in each programming language, along with the API specifications.

This time, select Python.

The top half of the displayed code is the 2.X code and the bottom half is the 3.x code. Create a python file by copying the one that suits your environment. (This time we will proceed with 3.x)

`customvisionapi.py`


import io, http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Content-Type': 'multipart/form-data',
    'Prediction-key': 'xxxxxxxxxxxxxxxxxxxxxxxx',
}

params = urllib.parse.urlencode({
    # Request parameters
    'iterationId': 'xxxxxxxxxxxxxxxxxxxxxxxx',
})

try:
    conn = http.client.HTTPSConnection('southcentralus.api.cognitive.microsoft.com')
    f = open("violin_a.jpg ", "rb", buffering=0)
    conn.request("POST", "/customvision/v1.0/Prediction/xxxxxxxxxxxxxxxxxxxxxxxx/image?%s" % params, f.readall(), headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    # print("[Errno {0}] {1}".format(e.errno, e.strerror))
    print(e)

Here, each parameter described in xxxxxxxxxxxxxxxxxxxxxxxx can be confirmed from the following.

Prediction-key
"Prediction Key:" field in the settings screen of the Custom Vision Service console
iterationId
In the "PERFORMANCE" screen of the console, click "Prediction URL" in the upper left, the end of the URL displayed, and the string after the "iterationId" parameter.
ProjectID
The part after "projects /" in the URL of the console screen itself

Perhaps there is a better way to look it up, but the one I found was the one above.

After setting each value, all you have to do is put the comparison file in the same directory as the Python file and execute it.

$ python customvisionapi.py
b'{"Id":"bxxxxxxxxxxxxxxxxxxxxxxxx","Project":"xxxxxxxxxxxxxxxxxxxxxxxx","Iteration":"xxxxxxxxxxxxxxxxxxxxxxxx","Created":"2017-06-06T13:59:23.5590916Z","Predictions":[{"TagId":"8fe34be4-eeff-495b-a83f-2a74bd25f035","Tag":"instrument","Probability":0.9999934},{"TagId":"bd2281d4-e4ff-48f1-a9ab-d525277479f9","Tag":"violin","Probability":0.9987756},{"TagId":"f33cdfdd-7eb2-47a2-8a30-2162a8f9e7fa","Tag":"oboe","Probability":3.22442238E-22},{"TagId":"b1490919-c0ab-4771-9564-13752bcfb96c","Tag":"tuba","Probability":7.720858E-24}]}'

You got the result. It is a little difficult to understand as it is, but you can see that the Probability data is returned for each tag, and the Probability of the Violin tag is as high as 0.9999934.

Now you can programmatically recognize the image! You can expand your dreams by incorporating it into an IoT device or combining it with a web application such as Django.

I get `BadRequestImageFormat`!

Well, this is the end of the procedure, but when I tried this, I was troubled by the phenomenon that BadRequestImageFormat appeared and the result could not be obtained for about an hour or two, so I will write down the solution. (The cause is not clear ...)

$ python customvisionapi.py
b'{"Code":"BadRequestImageFormat","Message":""}'

By the way, even if I asked Google teacher about this as it was, I was just told that I didn't know anything about it.

So here's the solution.

Although it is the sample code above, in fact, the code for pasting the image file to the request body is omitted in the document, only written as {body}.

conn.request("POST", "/customvision/v1.0/Prediction/{projectId}/image?%s" % params, "{body}", headers)

At first this part

imagefile = open("./violin_a.jpg ", "rb")
conn.request("POST", "/customvision/v1.0/Prediction/{projectId}/image?%s" % params, imagefile, headers)

I tried using the ʻopen ()` method, but it just didn't work.

So, while looking at io module reference and making trial and error, be careful of the following. I found that I could hit the API without any errors.

Put buffering = 0 in the third argument of ʻopen ()`
Pass the return value of the file object's readall () method to the request body
Use jpg images instead of ~~ png (!) ~~

__ (Corrected on June 25, 2017) What didn't work well with PNG images was a problem with the PNG file itself that I was using to check it. With the same source code, I was able to confirm that the result can be obtained even with a PNG image, so I will correct it. (/ Correct) __

I don't know the meaning of any of them, but I'll just mention that it worked for the time being. (In the case of Python2.x, it worked regardless of the third jpg. What a mess.)

As a result, the program that worked is as posted in the text.

Summary

that's all. Although each Azure service has the inconvenience of having little information, it seems that there are many APIs that can be used for something. There is also a free frame for 20,000 yen, so I will continue to try various things in the future.

[Azure] Hit Custom Vision Service with Python