Let's publish the super resolution API using Google Cloud Platform
Introduction
This article is [here](https://aiotadiary.wp.xdomain.jp/2020/03/01/google-cloud-platform%E3%82%92%E4%BD%BF%E3%81%A3% E3% 81% A6% E8% B6% 85% E8% A7% A3% E5% 83% 8F% E5% BA% A6% E5% 8C% 96api% E3% 82% 92% E5% 85% AC% E9% 96% 8B% E3% 81% 97% E3% 81% A6% E3% 81% BF% E3% 82% 8B /) This is a rewrite of the blog article for qiita. Please take a look at this one as well. This time, I would like to publish the API using Google Cloud Platform, commonly known as GCP. I used to use GCP a while ago, so this time I'd like to keep it in mind and have the experience of deploying the service myself. By the way, the API is a super-resolution model using the previously implemented ESPCN. What I would like to touch on this time is a function called Cloud Run in GCP.
Register with GCP
To register for GCP, you can usually refer to the Google guide or other articles. There is no problem if you follow the start guide. The article below will be helpful. GCP (GCE) to start from now on, safely use the free tier Basically, register in the following order.
- Log in to GCP
- Press the "Register for Free Trial" button on the screen above
- Fill in the information according to the displayed instructions
Here, you will be required to enter information such as your credit card, but please be assured that it will not be withdrawn without permission unless you set it up as a paid account.
Completion of registration
Registration is now complete. With the free GCP tier, you can build a GCE instance of f1-micro for free, so please use it. By the way, in the free trial, you can use the service worth 300 $ for free for 12 months, so you may want to try various things.
Use Cloud Run
Cloud Run is a service that allows you to deploy Docker containers, and basically deploys the service using the Docker Image uploaded to the Container Registry. At this time, it seems that there is a way to build and deploy the container triggered by pulling to Github, but this time it seems a bit difficult, so this time I would like to upload the locally built image.
Install gcloud command
Since I am using mac, the following explanation is not helpful for anyone other than mac. First, download the following archive.
google-cloud-sdk-245.0.0-darwin-x86_64.tar.gz
Then execute the following command.
$ ./google-cloud-sdk/install.sh
Next, initialize the SDK.
$ gcloud init
Do you want to log in after that? A message will appear saying that, so enter Y. After that, select the project to connect to, but please be assured that if there is only one project, it will be selected arbitrarily. After this, you will be able to use the command if you enter it appropriately. ↓ is the official setup method.
Quickstart for macOS Quickstart for Windows Quickstart for Linux
Create a Docker image
Here, I will use the ESPCN model implemented last time and use the code that doubles the resolution and outputs it as an API. I use Flask and gunicorn.
As a flow of the whole operation
- Decode base64 of the input image from the request json
- Input the decoded image into the super-resolution function
- Encode to base64 and respond
It's as simple as that. First, create a Git repository for Espcn-API (Click here for the repository). The directory structure looks like the following.
ESPCN-API
├model
│ └pre_trained_mode.pth
├api.py
├requirements.txt
├Dockerfile
└networks.py
test
├test.py
└test.png
Test code
Let's start with the test code. There are two things to check in the test.
-A response is returned (status_code is 200) ・ The size of the image has been doubled We will implement the operation to confirm these.
import sys
import requests
import json
from io import BytesIO
from PIL import Image
import base64
def image_file_to_base64(file_path):
with open(file_path, "rb") as image_file:
data = base64.b64encode(image_file.read())
return 'data:image/png;base64,' + data.decode('utf-8')
def base64_to_img(img_formdata):
img_base64 = img_formdata.split(',')[1]
input_bin = base64.b64decode(img_base64)
with BytesIO(input_bin) as b:
img = Image.open(b).copy().convert("RGB")
return img
if __name__ == "__main__":
source_image_path = "test.png "
source_image = Image.open(source_image_path)
source_width, source_height = source_image.size
print("source_width :", source_width)
print("source_height :", source_height)
host_url = "http://0.0.0.0:8000"
data = {"srcImage":image_file_to_base64(source_image_path)}
json_data = json.dumps(data)
response = requests.post(host_url, json_data, headers={'Content-Type': 'application/json'})
assert response.status_code == 200, "validation error status code should be 200"
res_json = response.json()
res_image = base64_to_img(res_json["sresoImage"])
sreso_width, sreso_height = res_image.size
print("sreso_width :", sreso_width)
print("sreso_height :", sreso_height)
assert sreso_width == source_width * 2 and sreso_height == source_height * 2 , \
"validation error image size should be 2 times of input image"
res_image.show()
print("OK")
Main code
Next, we will implement the main api.py. First, we created the skeleton of the entire implementation. The contents of the function are not implemented at all.
from flask import Flask, request, jsonify
from networks import Espcn
import os
from io import BytesIO
import base64
from torchvision import transforms
from torch import load
import torch
import json
from PIL import Image
device = "cpu"
net = Espcn(upscale=2)
net.load_state_dict(torch.load(opt.model_path, map_location="cpu"))
net.to(device)
net.eval()
def b64_to_PILImage(b64_string):
"""
process convert base64 string to PIL Image
input: b64_string: base64 string : data:image/png;base64,{base64 string}
output: pil_img: PIL Image
"""
pass
def PILImage_to_b64(pil_img):
"""
process convert PIL Image to base64
input: pil_img: PIL Image
output: b64_string: base64 string : data:image/png;base64,{base64 string}
"""
pass
def expand(src_image, model=net, device=device):
pass
@app.route("/", methods=["POST"])
def superResolution():
pass
if __name__ == "__main__":
app.run(host="0.0.0.0", port=os.environ.get("PORT", 8000))
I am thinking of a structure in which the expand function performs the main super-resolution and executes it in the superResolution that runs for the request. Also, by defining the model used for super-resolution outside the function, I think that it is possible to save the trouble of loading the model when the worker processes multiple requests.
So, first, implement two input / output ~ to ~ functions.
def b64_to_PILImage(b64_string):
"""
process convert base64 string to PIL Image
input: b64_string: base64 string : data:image/png;base64,{base64 string}
output: pil_img: PIL Image
"""
b64_split = b64_string.split(",")[1]
b64_bin = base64.b64decode(b64_split)
with BytesIO(b64_bin) as b:
pil_img = Image.open(b).copy().convert('RGB')
return pil_img
def PILImage_to_b64(pil_img):
"""
process convert PIL Image to base64
input: pil_img: PIL Image
output: b64_string: base64 string : data:image/png;base64,{base64 string}
"""
with BytesIO() as b:
pil_img.save(b, format='png')
b.seek(0)
img_base64 = base64.b64encode(b.read()).decode('utf-8')
img_base64 = 'data:image/png;base64,' + img_base64
return img_base64
I had a hard time using BytesIO because it was unexpectedly difficult. Next is the expand function and the main superResolution function.
def tensor_to_pil(src_tensor):
src_tensor = src_tensor.mul(255)
src_tensor = src_tensor.add_(0.5)
src_tensor = src_tensor.clamp_(0, 255)
src_tensor = src_tensor.permute(1, 2, 0)
src_tensor = src_tensor.to("cpu", torch.uint8).numpy()
return Image.fromarray(src_tensor)
def expand(src_image, model=net, device=device):
src_tensor = transforms.ToTensor()(src_image).to(device)
if src_tensor.dim() == 3:
src_tensor = src_tensor.unsqueeze(0)
srezo_tensor = model(src_tensor).squeeze()
srezo_img = tensor_to_pil(srezo_tensor)
return srezo_img
@app.route("/", methods=["POST"])
def superResolution():
req_json = json.loads(request.data)
src_b64 = req_json["srcImage"]
# main process
src_img = b64_to_PILImage(src_b64)
srezo_img = expand(src_img)
srezo_b64 = PILImage_to_b64(srezo_img)
results = {"sresoImage":srezo_b64}
return jsonify(results)
if __name__ == "__main__":
app.run(host="0.0.0.0", port=os.environ.get("PORT", 8000))
At first, I wanted to fit it in two functions, but the conversion from torch.tensor to PIL Image became strange when using transforms.ToPILImage, so I decided to define it separately. Did. Also, regarding the argument at the time of the last app.run, it seems that it is necessary to set to read the environment variable PORT because another port is automatically assigned to each docker container due to the specifications of cloud run.
Now, build these locally checked operations as docker images.
$ docker build -t gcr.io/[project id]/espcn-api:0 .
Then upload the image to the container registry using the gcloud command.
$ gcloud docker -- push gcr.io/[project id]/espcn-api:0
You can actually see the image on the Container Registry on GCP.
Then deploy using the uploaded image.
$ gcloud beta run deploy SR-API --image=gcr.io/[project id]/espcn-api:0
The part where SR-API is entered after deploy is the name of the service, so you can add it as you like. It also seems to install the beta component the first time it runs. If you do not enter (--platform managed) at this time, on the console
[1] Cloud Run (fully managed)
[2] Cloud Run for Anthos deployed on Google Cloud
[3] Cloud Run for Anthos deployed on VMware
[4] cancel
Please enter your numeric choice: _
You will be asked to enter 1, so enter 1. It will be difficult to operate for free unless it is fully managed. Then you will be asked for the region.
[1] asia-east1
[2] asia-northeast1
[3] europe-north1
[4] europe-west1
[5] europe-west4
[6] us-central1
[7] us-east1
[8] us-east4
[9] us-west1
[10] cancel
Please enter your numeric choice: _
Here, let's select "us- *". In Cloud Run, downlink networking in North America is free up to 1GB, so if you call this from Cloud Functions (since Cloud Functions, downlink networking is free up to 5GB anywhere), you can use it almost free of charge.
After this, you will be asked if you want to allow unauthenticated access, but for now, the purpose is to move it, so let's set it to yes. If you enter as above, the service will be deployed and the following message will be output.
Deploying container to Cloud Run service [espcn-api] in project [studied-brace-261908] region [us-central1]
✓ Deploying new service... Done.
✓ Creating Revision...
✓ Routing traffic...
✓ Setting IAM Policy...
Done.
Service [espcn-api] revision [espcn-api-00001-guj] has been deployed and is serving 100 percent of traffic at https://espcn-api-~~~-uc.a.run.app
Eventually, a message saying where and where the url was deployed will appear, so when I tried to make a request using test.py here, I was able to confirm that the response was returned properly.
There was one thing that got stuck at this time, but when I first made the request, no response was returned. So, if you check the Cloud Run log,
Memory limit of 244M exceeded with 272M used. Consider increasing the memory limit, see https://cloud.google.com/run/docs/configuring/memory-limits
I received a message stating that there is not enough memory. It seems that the memory used in cloud run is decided, the size of the memory that can be specified is 128MB ~ 2GB, and 256MB is allocated by default. Since this error occurred with a 512 x 512 image, it seems that it can be solved by allocating 512MB etc., but if you allocate too much memory, it seems that the free frame will be exceeded soon. By the way, if you want to change the memory allocated to the application, use the following command.
$ gcloud beta run services update [service name] --memory 512M
At the end
We have created an API and published it on Cloud Run. In the current state, this API can be accessed from anywhere, and it is not security or wallet friendly, so next, after limiting access to Cloud Run from within GCP, use Cloud Functions to do this. I would like to change the structure to send a request to. I would like to touch on continuous deployment someday.
Recommended Posts