AWS Lambda with PyTorch [Lambda import]

Introduction

This article is a summary of an attempt to run PyTorch on AWS Lambda. It has a DeepLearning tag, but does not touch on learning. The goal is to try some inference on Lambda. (Click here for EFS settings](https://qiita.com/myonoym/items/5d360ed11931474cb640))

Rough summary

  1. Try using EFS
  2. PyTorch on Lambda ← This time here
  3. Call from slack

It is a three-pronged stand.

Model running on Lambda

This time, I will try to run with Lambda EML-NET. See Github Project Page for more details, but SaliencyMap ) Is a model to generate. A Saliency Map is a heat map that represents the position where a person's line of sight is directed.

Quoted from the paper Figure 1 スクリーンショット 2020-09-14 17.16.07.png

Put various things with pip

I mounted EFS on / mnt in previous article, but let's put the library you want to import from Lambda into EFS.

$ cd /mnt/lambda
$ sudo pip3 install -t . torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
$ sudo pip3 install --upgrade -t . opencv-python==4.4.0.42 scipy==1.5.2

by this,

Is now under / mnt / lambda.

Import with Lambda

Prepare Lambda

Click Create Function-> Create from scratch-> Enter Function Name-> Set Runtime to Python3.7-> Create Function

VPC Around here Something is written, but in order to connect Lambda to EFS from Lambda, VPC Must be placed inside. The default VPC should be fine. Regarding the security group, I think that there is an EC2 setting used to prepare the library in EFS, so specify that. When I click save, I get the following error: スクリーンショット 2020-09-15 12.04.42.png I don't seem to have enough authority. According to here

ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface

It seems that Lambda needs this permission. ʻThe AWS management policy is included in the AWSLambdaVPCAccessExecutionRole`, so let's attach this policy to the Lambda role. Then you should be able to set up your VPC.

Connect to EFS

Connect the prepared EFS and Lambda. Click Add File System fromFile Systemin the console. Specify the EFS file system and access point, set the local mount path to / mnt / lambda and click Save.

Tested with Lambda

Let's test if Lambda can load the library in EFS. EFS is mounted on / mnt / lambda, so add / mnt / lambda to your python path.

lambda_function.py


import json
import sys

sys.path.append("/mnt/lambda")

import torch
import torchvision
import PIL
import cv2
import numpy as np

def lambda_handler(event, context):
    print(f"torch:{torch.__version__}")
    print(f"torchvision:{torchvision.__version__}")
    print(f"PIL:{PIL.__version__}")
    print(f"cv2:{cv2.__version__}")
    print(f"numpy:{np.__version__}")

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

If you create an appropriate test event on the console and run the test, you will get the following result. You can read PyTorch properly.

START RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
torch:1.6.0+cpu
torchvision:0.7.0+cpu
PIL:7.2.0
cv2:4.4.0
numpy:1.19.2
END RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b
REPORT RequestId: 35329cd4-50f6-4eb7-8950-f27daf75462b	Duration: 29212.21 ms	Billed Duration: 29300 ms	Memory Size: 128 MB	Max Memory Used: 129 MB	

Move the model

Now that we're ready, let's move the model. The location of the trained model is listed in EML github page, so download it. According to the teaching of README, put 3 files, res_imagenet.pth, res_places.pth, res_decoder.pth in / mnt / lambda / backbone of EFS which is mounted on EC2 by scp.

Based on eval_combined.py, we will modify it so that it can be run on Lambda.

lambda_function.py


import sys
sys.path.append("/mnt/lambda")
import os

import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
import torchvision.transforms as transforms

from PIL import Image
import cv2
import numpy as np

import resnet
import decoder

#Isn't the environment variable around here unnecessary? I also feel that.
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

image_model_path = "/mnt/lambda/backbone/res_imagenet.pth"
place_model_path = "/mnt/lambda/backbone/res_places.pth"
decoder_model_path = "/mnt/lambda/backbone/res_decoder.pth"
size = (480, 640)
num_feat = 5


def normalize(x):
    x -= x.min()
    x /= x.max()


def post_process(pred):
    pred = cv2.GaussianBlur(pred, (5,5), 10.0)
    normalize(pred)
    pred_uint = (pred * 255).astype(np.uint8)
    return pred, pred_uint


def draw_heatmap(pred, img):
    #Convert the saliency map to the original image size.
    resized_pred = np.asarray(Image.fromarray(pred).resize((img.size[0], img.size[1])), dtype=np.uint8)
    resized_colormap = cv2.applyColorMap(resized_pred, cv2.COLORMAP_JET)
    resized_colormap = cv2.cvtColor(resized_colormap, cv2.COLOR_BGR2RGB)
    #Convert the original image to numpy ndarray.
    img_array = np.asarray(img)

    #Blending
    alpha = 0.5
    blended = cv2.addWeighted(img_array, alpha, resized_colormap, 1-alpha, 0)

    return blended


def predict(image_model_path, place_model_path, decoder_model_path, pil_img):
    img_model = resnet.resnet50(image_model_path).eval()
    pla_model = resnet.resnet50(place_model_path).eval()
    decoder_model = decoder.build_decoder(decoder_model_path, size, num_feat, num_feat).eval()

    preprocess = transforms.Compose([
        transforms.Resize(size),
        transforms.ToTensor(),
    ])

    processed = preprocess(pil_img).unsqueeze(0)


    with torch.no_grad():
        img_feat = img_model(processed, decode=True)
        pla_feat = pla_model(processed, decode=True)

        pred = decoder_model([img_feat, pla_feat])

    pred_origin = pred.squeeze().detach().cpu().numpy()
    pred, pred_uint = post_process(pred_origin)

    heatmap = draw_heatmap(pred_uint, pil_img)

    return heatmap


def lambda_handler(event, context):
    #Empty response
    empty_response = {
        "statusCode": 200,
        "body": "{}"
    }

    pil_img = Image.open("/mnt/lambda/image/examples/115.jpg ").convert("RGB")

    heatmap = predict(image_model_path, place_model_path, decoder_model_path, pil_img)
    print(heatmap.shape)

    return empty_response

It's a bit long, but now you can generate a SaliencyMap on Lambda. Since GPU cannot be used on Lambda, the description of the cuda related part is modified from the original code. When executed,

START RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
Model loaded /mnt/lambda/backbone/res_imagenet.pth
Model loaded /mnt/lambda/backbone/res_places.pth
Loaded decoder /mnt/lambda/backbone/res_decoder.pth
(511, 681, 3)
END RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec
REPORT RequestId: 6f9baccf-b758-4e9a-b43a-b92bdd9757ec	Duration: 20075.02 ms	Billed Duration: 20100 ms	Memory Size: 1024 MB	Max Memory Used: 614 MB	

It seems that it can be executed safely. I wanted to output the file to the EFS side, but it was a bit annoying, so let's modify it so that it can be output to S3 while calling from Slack next time.

Summary

By connecting EFS to Lambda, you can now load libraries and model files. I think this will allow model inference to be performed without worrying about Lambda capacity as in the past. Next time, as a slack call edition, I will call this Lambda from slack and see it.

Serpentine

In writing this article, I created the configuration once again from scratch for confirmation, but I regret that if I did it with AWS CDK or something like that, the writing would have progressed. I will.

Recommended Posts

AWS Lambda with PyTorch [Lambda import]
[AWS] Create API with API Gateway + Lambda
Using Lambda with AWS Amplify with Go
Notify HipChat with AWS Lambda (Python)
[AWS] Using ini files with Lambda [Python]
Play with PyTorch
[AWS] Link Lambda and S3 with boto3
Cross-validation with PyTorch
Beginning with PyTorch
Connect to s3 with AWS Lambda Python
[AWS] Do SSI-like things with S3 / Lambda
Python + Selenium + Headless Chromium with aws lambda
I just did FizzBuzz with AWS Lambda
[AWS SAM] Create API with DynamoDB + Lambda + API Gateway
Regular serverless scraping with AWS lambda + scrapy Part 1.8
Serverless scraping using selenium with [AWS Lambda] -Part 1-
LINE BOT with Python + AWS Lambda + API Gateway
Serverless application with AWS SAM! (APIGATEWAY + Lambda (Python))
[AWS] Try tracing API Gateway + Lambda with X-Ray
I tried connecting AWS Lambda with other services
Infrastructure construction automation with CloudFromation + troposphere + AWS Lambda
Make Lambda Layers with Lambda
Use RTX 3090 with PyTorch
Install torch-scatter with PyTorch 1.7
Tweet from AWS Lambda
AWS CDK with Python
Import tsv with Python
Try AWS Lambda Destinations
Dynamic HTML pages made with AWS Lambda and Python
Deploy Python3 function with Serverless Framework on AWS Lambda
Create a Layer for AWS Lambda Python with Docker
[AWS] Play with Step Functions (SAM + Lambda) Part.1 (Basic)
I want to AWS Lambda with Python on Mac!
Manage your Amazon CloudWatch loggroup retention with AWS Lambda
Make ordinary tweets fleet-like with AWS Lambda and Python
[AWS] Play with Step Functions (SAM + Lambda) Part.2 (Parameter)
[AWS] What to do when you want to pip with Lambda
Operate TwitterBot with Lambda, Python
[Python] Scraping in AWS Lambda
The simplest AWS Lambda implementation
Import vtk with brew python
Try an autoencoder with Pytorch
Deploy Django serverless with Lambda
[AWS] Try adding Python library to Layer with SAM + Lambda (Python)
Easily write if-elif with lambda
Try implementing XOR with PyTorch
Output CloudWatch Logs to S3 with AWS Lambda (Pythyon ver)
Implement PyTorch + GPU with Docker
Linux fastest learning with AWS
Prediction of Nikkei 225 with Pytorch 2
Machine learning Minesweeper with PyTorch
Try automating Start / Stop for EC2 instances with AWS Lambda
Dynamically move Amplify with Lambda
I just built a virtual environment with AWS lambda layer
Web scraping using AWS lambda
Create API with Python, lambda, API Gateway quickly using AWS SAM
Prediction of Nikkei 225 with Pytorch
Perform Stratified Split with PyTorch
I made Word2Vec with Pytorch
Server management with Jupyter (1) import
Site monitoring and alert notification with AWS Lambda + Python + Slack