[DOCKER] [AWS Lambda] Use any container Image in Python very simply

Overview

Container Image Support announced at AWS Lambda. What's new in AWS Lambda – Container Image Support (https://aws.amazon.com/jp/blogs/news/new-for-aws-lambda-container-image-support/)

However, the sample code in the AWS official documentation is The content is a little difficult to understand due to its habit.

In this article, I will write about the simplest deployment and execution method possible.

Target audience

--Knowledge of Docker and containers. --Has a rough knowledge of AWS Lambda. --Aws-cli is installed and set up.

Hands-on

Follow the procedure below.

  1. File preparation
  2. Image creation / execution locally
  3. Deploy Image
  4. Lambda settings and execution

This is the directory structure. I made it as simple as possible so that it can be applied easily.

#Directory structure
├── Dockerfile
├── entry.sh
└── app
    └── app.py

(1) File preparation

This time, let's use buster (debian image, OS of Linux distribution). Use an image that already has python installed. In fact, other images are fine.

The points are the following two points. --Installing aws lambdaric --Install runtime interface emulator if you want to run it locally These will be used in entry.sh (discussed below).

The source code of each file is shown below.

--Dockerfile: Create Image

# Dockerfile

FROM python:3.9-buster

#Installing runtime interface console
RUN pip install awslambdaric

#install runtime interface emulator to run locally
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
RUN chmod 755 /usr/bin/aws-lambda-rie
COPY entry.sh "/entry.sh"
RUN chmod 755 /entry.sh

#Place the executable file in the container.
ARG APP_DIR="/home/app/"
WORKDIR ${APP_DIR}
COPY app ${APP_DIR}

ENTRYPOINT [ "/entry.sh" ]
CMD [ "app.handler" ]

--app/app.py: Source code you want to run.

# app/app.py
def handler(event, context):
    return "Hello world!!"

--entry.sh: Determine if it's local or a container on AWS Lambda Use aws-lambda-rie or aws lambdaric. Official reference.

# app/entry.sh

if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
    exec /usr/bin/aws-lambda-rie /usr/local/bin/python -m awslambdaric $1
else
    exec /usr/local/bin/python -m awslambdaric $1
fi
  • Caution /usr/local/bin/python and /entry.sh are lambda specifications, and it seems that commands must be written on the absolute bus. If it is not an absolute path, it will work locally, but when I run it on Lambda I get the following error:
START RequestId: 80f9d98d-06b5-4ba8-b729-b2e6ac2abbe6 Version: $LATEST
IMAGE	Launch error: Couldn't find valid bootstrap(s): [python]	Entrypoint: []

(2) Creating an image and executing it locally

First, create an image.

docker build -t container_lambda .

Next, set up the container and the daemon will start up.

docker run -it --rm -p 9000:8080 container_lambda
> NFO[0000] exec '/usr/local/bin/python' (cwd=/home/app, handler=app.handler) 

To skip the event, throw a post at the following URL.

curl -XPOST "http://localhost:9002/2015-03-31/functions/function/invocations" -d '{}'
> "Hello world!!"% 

Then Hello World !! was returned! Successful execution locally.

If you want to change the file or function to be executed, change the handler specified in CMD.

(3) Deploy the image

Register the image in AWS's Elastic Container Registry (ECR) for it to actually work with Lambda. First, let's move to the ECR screen.

Press Create Repository. image.png

This time, create a repository with the name container_lambda. image.png After deciding on a name, press Create Repository at the bottom

Then push the local image to this repository.


#Specify the name of the image.
IMAGENAME=container_lambda

#Specify the URL of ECR.
REGISTRYURL=xxxxxxxxx.ecr.ap-northeast-1.amazonaws.com

#Log in to AWS ECR.
aws ecr get-login-password | docker login --username AWS --password-stdin $REGISTRYURL

#Create an image and deploy it to AWS ECR.
docker build -t ${IMAGENAME} .
docker tag ${IMAGENAME} 386617633989.dkr.ecr.ap-northeast-1.amazonaws.com/${IMAGENAME}
docker push 386617633989.dkr.ecr.ap-northeast-1.amazonaws.com/${IMAGENAME}

You have now deployed the image.

(4) Lambda setting / execution

Finally, let's run it on Lambda! First, from the AWS console, go to the Lambda screen and press Create Function.

  1. Select container image
  2. Give a function name (this time container_lambda)
  3. Specify the Docker Image you just deployed image.png Then press Create Function to change the screen. After a while, the upper band turns green and the lambda setup is complete.

image.png

If you define the test in the upper right and run it ... image.png Hello world !! is back!

Summary

So, I tried to summarize the execution in the original runtime.

In the official document, it seems complicated to use alpine system intentionally or multi stage buiid of Docker, but I think that it turned out to be easier in reality.

Lambda is extremely hot with the announcement that it supports up to 6 cores and 10GB of memory. It's also easier to create a simple machine learning API. AWS Lambda now supports up to 10 GB of memory and 6 vCPU cores for Lambda Functions

It's very convenient, so please give it a try!

Next, I would like to run an image recognition model called Detectron2 on AWS Lambda!

Recommended Posts

[AWS Lambda] Use any container Image in Python very simply
[Python] Scraping in AWS Lambda
Write AWS Lambda function in Python
Use print in a Python2 lambda expression
How to use Python Image Library in python3 series
Summary of studying Python to use AWS Lambda
Use python in Docker container as Pycharm interpreter
Use config.ini in Python
Use Valgrind in Python
Use profiler in Python
[Python] Retry process (Exponential Backoff) memo in AWS Lambda
Let's use def in python
Use let expression in Python
Use Measurement Protocol in Python
Use callback function in Python
Use parameter store in Python
Use HTTP cache in Python
Use list-keyed dict in Python
Use regular expressions in Python
Use Spyder in Python IDE
Tweet with image in Python
Image Processing Collection in Python
How to use Python lambda
Best practice for logging in JSON format on AWS Lambda / Python
[Python / AWS Lambda layers] I want to reuse only module in AWS Lambda Layers
I compared Node.js and Python in creating thumbnails using AWS Lambda
Install pip in Serverless Framework and AWS Lambda with Python environment
Summary if using AWS Lambda (Python)
Implemented image segmentation in python (Union-Find)
Create a DI Container in Python
How to use SQLite in Python
Run Python on Schedule on AWS Lambda
Recommended container image for Python applications
How to use Mysql in python
Use Python in pyenv with NeoVim
How to use ChemSpider in Python
How to use PubChem in Python
Use OpenCV with Python 3 in Window
Notify HipChat with AWS Lambda (Python)
Use PostgreSQL with Lambda (Python + psycopg2)
ImportError when trying to use gcloud package with AWS Lambda Python version
Use Resource API rather than Client API in AWS SDK for Python (Boto3)
Modules cannot be imported in Python on EC2 run from AWS Lambda
Upload what you got in request to S3 with AWS Lambda Python
[AWS] Using ini files with Lambda [Python]
[Introduction to Python] How to use class in Python?
Easily use your own functions in Python
Obtaining temporary AWS credentials in PHP, Python
Use cryptography library cryptography with Docker Python image
Don't use \ d in Python 3 regular expressions!
How to adjust image contrast in Python
How to use __slots__ in Python class
Easy image processing in Python with Pillow
Connect to s3 with AWS Lambda Python
Image sending / receiving memo in Python (Flask)
Use pathlib in Maya (Python 2.7) for upcoming Python 3.7
[Road to intermediate Python] Use lambda expressions
How to use regular expressions in Python
CG image quality evaluation memo in Python
Download files in any format using Python
Install the Python module in any directory