AWS Lambda with PyTorch [Lambda import]

Introduction

This article is a summary of an attempt to run PyTorch on AWS Lambda. It has a DeepLearning tag, but does not touch on learning. The goal is to try some inference on Lambda. (Click here for EFS settings](https://qiita.com/myonoym/items/5d360ed11931474cb640))

Rough summary

1. Try using EFS
2. PyTorch on Lambda ← This time here
3. Call from slack

It is a three-pronged stand.

Model running on Lambda

This time, I will try to run with Lambda EML-NET. See Github Project Page for more details, but SaliencyMap ) Is a model to generate. A Saliency Map is a heat map that represents the position where a person's line of sight is directed.

Quoted from the paper Figure 1

Put various things with pip

I mounted EFS on / mnt in previous article, but let's put the library you want to import from Lambda into EFS.

$ cd /mnt/lambda
$ sudo pip3 install -t . torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
$ sudo pip3 install --upgrade -t . opencv-python==4.4.0.42 scipy==1.5.2

by this,

• torch==1.6.0+cpu
• torchvision==0.7.0+cpu
• numpy==1.19.2
• future==0.18.2
• pillow==7.2.0
• opencv==4.4.0.42

Is now under / mnt / lambda.

Import with Lambda

Prepare Lambda

Click Create Function-> Create from scratch-> Enter Function Name-> Set Runtime to Python3.7-> Create Function

VPC Around here Something is written, but in order to connect Lambda to EFS from Lambda, VPC Must be placed inside. The default VPC should be fine. Regarding the security group, I think that there is an EC2 setting used to prepare the library in EFS, so specify that. When I click save, I get the following error: I don't seem to have enough authority. According to here

ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface

It seems that Lambda needs this permission. ʻThe AWS management policy is included in the AWSLambdaVPCAccessExecutionRole`, so let's attach this policy to the Lambda role. Then you should be able to set up your VPC.

Connect to EFS

Connect the prepared EFS and Lambda. Click Add File System fromFile Systemin the console. Specify the EFS file system and access point, set the local mount path to / mnt / lambda and click Save.

Tested with Lambda

Let's test if Lambda can load the library in EFS. EFS is mounted on / mnt / lambda, so add / mnt / lambda to your python path.

lambda_function.py

import json
import sys

sys.path.append("/mnt/lambda")

import torch
import torchvision
import PIL
import cv2
import numpy as np

def lambda_handler(event, context):
    print(f"torch:{torch.__version__}")
    print(f"torchvision:{torchvision.__version__}")
    print(f"PIL:{PIL.__version__}")
    print(f"cv2:{cv2.__version__}")
    print(f"numpy:{np.__version__}")

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

If you create an appropriate test event on the console and run the test, you will get the following result. You can read PyTorch properly.

START RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
torch:1.6.0+cpu
torchvision:0.7.0+cpu
PIL:7.2.0
cv2:4.4.0
numpy:1.19.2
END RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b
REPORT RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b	Duration: 29212.21 ms	Billed Duration: 29300 ms	Memory Size: 128 MB	Max Memory Used: 129 MB	

Move the model

Now that we're ready, let's move the model. The location of the trained model is listed in EML github page, so download it. According to the teaching of README, put 3 files, res_imagenet.pth, res_places.pth, res_decoder.pth in / mnt / lambda / backbone of EFS which is mounted on EC2 by scp.

Based on eval_combined.py, we will modify it so that it can be run on Lambda.

lambda_function.py

import sys
sys.path.append("/mnt/lambda")
import os

import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
import torchvision.transforms as transforms

from PIL import Image
import cv2
import numpy as np

import resnet
import decoder

#Isn't the environment variable around here unnecessary? I also feel that.
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

image_model_path = "/mnt/lambda/backbone/res_imagenet.pth"
place_model_path = "/mnt/lambda/backbone/res_places.pth"
decoder_model_path = "/mnt/lambda/backbone/res_decoder.pth"
size = (480, 640)
num_feat = 5


def normalize(x):
    x -= x.min()
    x /= x.max()


def post_process(pred):
    pred = cv2.GaussianBlur(pred, (5,5), 10.0)
    normalize(pred)
    pred_uint = (pred * 255).astype(np.uint8)
    return pred, pred_uint


def draw_heatmap(pred, img):
    #Convert the saliency map to the original image size.
    resized_pred = np.asarray(Image.fromarray(pred).resize((img.size[0], img.size[1])), dtype=np.uint8)
    resized_colormap = cv2.applyColorMap(resized_pred, cv2.COLORMAP_JET)
    resized_colormap = cv2.cvtColor(resized_colormap, cv2.COLOR_BGR2RGB)
    #Convert the original image to numpy ndarray.
    img_array = np.asarray(img)

    #Blending
    alpha = 0.5
    blended = cv2.addWeighted(img_array, alpha, resized_colormap, 1-alpha, 0)

    return blended


def predict(image_model_path, place_model_path, decoder_model_path, pil_img):
    img_model = resnet.resnet50(image_model_path).eval()
    pla_model = resnet.resnet50(place_model_path).eval()
    decoder_model = decoder.build_decoder(decoder_model_path, size, num_feat, num_feat).eval()

    preprocess = transforms.Compose([
        transforms.Resize(size),
        transforms.ToTensor(),
    ])

    processed = preprocess(pil_img).unsqueeze(0)


    with torch.no_grad():
        img_feat = img_model(processed, decode=True)
        pla_feat = pla_model(processed, decode=True)

        pred = decoder_model([img_feat, pla_feat])

    pred_origin = pred.squeeze().detach().cpu().numpy()
    pred, pred_uint = post_process(pred_origin)

    heatmap = draw_heatmap(pred_uint, pil_img)

    return heatmap


def lambda_handler(event, context):
    #Empty response
    empty_response = {
        "statusCode": 200,
        "body": "{}"
    }

    pil_img = Image.open("/mnt/lambda/image/examples/115.jpg ").convert("RGB")

    heatmap = predict(image_model_path, place_model_path, decoder_model_path, pil_img)
    print(heatmap.shape)

    return empty_response

It's a bit long, but now you can generate a SaliencyMap on Lambda. Since GPU cannot be used on Lambda, the description of the cuda related part is modified from the original code. When executed,

START RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
Model loaded /mnt/lambda/backbone/res_imagenet.pth
Model loaded /mnt/lambda/backbone/res_places.pth
Loaded decoder /mnt/lambda/backbone/res_decoder.pth
(511, 681, 3)
END RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec
REPORT RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec	Duration: 20075.02 ms	Billed Duration: 20100 ms	Memory Size: 1024 MB	Max Memory Used: 614 MB	

It seems that it can be executed safely. I wanted to output the file to the EFS side, but it was a bit annoying, so let's modify it so that it can be output to S3 while calling from Slack next time.

Summary

By connecting EFS to Lambda, you can now load libraries and model files. I think this will allow model inference to be performed without worrying about Lambda capacity as in the past. Next time, as a slack call edition, I will call this Lambda from slack and see it.

Serpentine

In writing this article, I created the configuration once again from scratch for confirmation, but I regret that if I did it with AWS CDK or something like that, the writing would have progressed. I will.