Create a REST API using the model learned in Lobe and TensorFlow Serving.

In the article below, I explained how to learn with Lobe and how to export the learned model and use it from Python.

I tried "Lobe," which makes it easy to train machine learning models published by Microsoft. How to use the model learned in Lobe in Python

This time I would like to create a REST API in combination with TensorFlow Serving.

Preparation for operating TensorFlow Serving

See the article mentioned above for learning with Lobe and exporting models.

After trying various things, it seems that the name of the directory containing the model needs to be a number in order for TensorFlow Serving to work. (I haven't grasped this area yet, so I would appreciate it if anyone knows it.)

Therefore, create a directory named with numbers in the model directory exported from Lobe and copy or move the necessary files.

Launch TensorFlow Serving with Docker

The easiest way to use TensorFlow Serving is with Docker. The official Image is uploaded to docker hub, so use this. tensorflow/serving

The model directory used this time is as follows. TensorFlow Serving uses the model in the directory 1.

── sample_model
    ├── 1 #Create a new sample_Copy or move necessary files under model
    │   ├── saved_model.pb
    │   └── variables
    │       ├── variables.data-00000-of-00001
    │       └── variables.index
    ├── example
    │   ├── README.md
    │   ├── requirements.txt
    │   └── tf_example.py
    ├── saved_model.pb
    ├── signature.json
    └── variables
        ├── variables.data-00000-of-00001
        └── variables.index

Start docker as follows.

# docker run -t --rm -p 8501:8501 -v "`pwd`/sample_model:/models/sample_model" -e MODEL_NAME=sample_model tensorflow/serving

When the model is loaded and the operation starts normally, the following message will be displayed.

2020-11-03 01:04:51.142942: I tensorflow_serving/model_servers/server.cc:387] Exporting HTTP/REST API at:localhost:8501 ...
[evhttp_server.cc : 238] NET_LOG: Entering the event loop ...

If this message is output, TensorFlow Serving has started.

Try image classification with REST

With the steps up to this point, you are ready to try image classification with REST. Let's send a GET request to check the operation of TensorFlow Serving.

import requests
import json

url = 'http://localhost:8501/v1/models/sample_model'
res = requests.get(url)
print(json.loads(res.text))

If the following response is returned, it is working without any problem.

{'model_version_status': [{'version': '1', 'state': 'AVAILABLE', 'status': {'error_code': 'OK', 'error_message': ''}}]}

Forecast requests are made by POST.

import json

import cv2
import numpy as np
import requests

#Prediction request URL
url = 'http://localhost:8501/v1/models/sample_model:predict'

def predict(image_path):
    image = cv2.imread(image_path, cv2.IMREAD_COLOR)
    image = image.astype(np.float32) / 255
    image = image.tolist()

    headers = {"content-type": "application/json"}
    body = {"inputs": [image]}
    r = requests.post(url, data=json.dumps(body), headers=headers)
    r_json = json.loads(r.text)
    return r_json['outputs']['Prediction'][0]

You can get the prediction result by giving the image file as an argument to predict.

predict_label = predict('Image file')
print(predict_label)

Up to this point, you can create an environment that works at a minimum.