This is the 20th day article of AWS & Game Advent Calendar 2020.
agenda
At re: Invent in 2020, it was announced that Lambda will support containers. Until now, running Lambda with a custom runtime required a tedious setup, but I think this update will make it much easier to use.
In this article, I will explain how to use a container to operate a minimum function (using bash to display event data) on Lambda.
sam was installed using Homebrew
$ brew tap aws/tap
$ brew install aws-sam-cli
$ sam --version
SAM CLI, version 1.15.0
Create an ECR repository using awscli. The repository name is docker-lambda-bash.
$ aws ecr create-repository --repository-name docker-lambda-bash
The custom runtime tutorial is here (https://docs.aws.amazon.com/ja_jp/lambda/latest/dg/runtimes-walkthrough.html). The following three items are required to operate Lambda with bash.
We will proceed with the implementation using the official image of here.
bootstrap
See the tutorial for details. Simply put, turn while to use the Lambda API to get Lambda events. The acquired event data is returned as a response.
#!/bin/sh -xv
set -euo pipefail
#Loading handler function
source $LAMBDA_TASK_ROOT/"$(echo $_HANDLER | cut -d. -f1).sh"
while true
do
HEADERS="$(mktemp)"
#Get the event
EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)
#Execution of handler function
RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) "$EVENT_DATA")
#Return the result
curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response" -d "$RESPONSE"
done
function.sh
See the tutorial for more details. It is a script that just echoes the event data.
function.sh
function handler () {
EVENT_DATA=$1
echo "$EVENT_DATA" 1>&2;
RESPONSE="Echoing request: '$EVENT_DATA'"
echo $RESPONSE
}
To run this function locally, build the image using the following Dockerfile.
FROM public.ecr.aws/lambda/provided:al2
COPY bootstrap ${LAMBDA_RUNTIME_DIR}
COPY function.sh ${LAMBDA_TASK_ROOT}
CMD [ "function.handler" ]
The content is very simple, based on the image of Amazon Linux2 (al2), bootstrap is placed in Lambda's runtime Dir, function.sh is placed in Lambda's task Dir, and function.handler is executed in CMD.
The Lambda function is specified as (script name). (Function name).
In this example, it is necessary to execute the function handler in the script named function.sh, so this is the specification method.
Now that the dockekrfile is ready, build the image as follows.
$ docker build -t docker-lambda-bash .
Run using the built image.
$ docker run -p 9000:8080 docker-lambda-bash
Request data will be returned when you access port 9000 with curl.
$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"payload":"hello world!"}'
Echoing request: '{"payload":"hello world!"}'
Deploy to Lambda uses sam. I will omit the detailed usage of sam, but the general procedure is as follows.
To log in to ECR, execute the following command.
$ export aws_region=ap-northeast-1 #Set region in environment variable
$ export aws_account_id=xxxxxxxxxx #Set AWS account ID in environment variable
$ aws ecr get-login-password --region $aws_region | docker login --username AWS --password-stdin $aws_account_id.dkr.ecr.$aws_region.amazonaws.com
tagging & push
$ docker tag docker-lambda $aws_account_id.dkr.ecr.$aws_region.amazonaws.com/docker-lambda-bash
$ docker push $aws_account.dkr.ecr.$aws_region.amazonaws.com/docker-lambda-bash
You need to create a template to deploy Lambda with sam.
Since it is necessary to specify the environment variables aws_account_id and aws_region, replace (YOUR-AWS-ACCOUNT-ID) and (YOUR-AWS-REGION) as appropriate. Please give me.
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: lambda-custom-docker-sample
Globals:
Function:
Timeout: 3
Resources:
MyCustomDocker:
Type: AWS::Serverless::Function
Properties:
FunctionName: CustomDockerWithBash
ImageUri: (YOUR-AWS-ACCOUNT-ID).dkr.ecr.(YOUR-AWS-REGION).amazonaws.com/docker-lambda-bash:latest
PackageType: Image
Events:
CustomDockerEcho:
Type: Api
Properties:
Path: /echo
Method: get
Outputs:
MyCustomDockerApi:
Description: 'API Gateway endpoint URL'
Value: !Sub 'https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/echo/'
The container image pushed to ECR is specified by ImageUri.
Also, I am creating an endpoint so that I can check it using ApiGateway by specifying Outputs.
deploy
Deploy with sam deploy --guided. The log at the time of deploy is as follows.
$ sam deploy --guided
Configuring SAM deploy
======================
Looking for config file [samconfig.toml] : Not found
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]: lambda-custiom-docker-sample
AWS Region [us-east-1]: ap-northeast-1
Image Repository for MyCustomDocker: xxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/docker-lambda-bash
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: y
MyCustomDocker may not have authorization defined, Is this okay? [y/N]: y
Save arguments to configuration file [Y/n]: y
SAM configuration file [samconfig.toml]:
SAM configuration environment [default]:
Looking for resources needed for deployment: Not found.
Creating the required resources...
Successfully created!
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1b4wws4k1oxg
A different default S3 bucket can be set in samconfig.toml
Saved arguments to config file
Running 'sam deploy' for future deployments will use the parameters saved above.
The above parameters can be changed by modifying samconfig.toml
Learn more about samconfig.toml syntax at
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Deploying with following values
===============================
Stack name : lambda-custiom-docker-sample
Region : ap-northeast-1
Confirm changeset : True
Deployment image repository :
{
"MyCustomDocker": "xxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/docker-lambda-bash"
}
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1b4wws4k1oxg
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Signing Profiles : {}
Initiating deployment
=====================
MyCustomDocker may not have authorization defined.
Uploading to lambda-custiom-docker-sample/e1f6334b9552a49d5e9e07a9970bbae3.template 697 / 697.0 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
-------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Add MyCustomDockerCustomDockerEchoPermiss AWS::Lambda::Permission N/A
ionProd
+ Add MyCustomDockerRole AWS::IAM::Role N/A
+ Add MyCustomDocker AWS::Lambda::Function N/A
+ Add ServerlessRestApiDeploymentc8d8b9cf15 AWS::ApiGateway::Deployment N/A
+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage N/A
+ Add ServerlessRestApi AWS::ApiGateway::RestApi N/A
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:ap-northeast-1:xxxxxxxxxx:changeSet/samcli-deploy1608550235/9b8db82c-4197-4d15-9624-fc35f2e35bd8
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
2020-12-21 20:31:50 - Waiting for stack create/update to complete
CloudFormation events from changeset
-------------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
-------------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS AWS::IAM::Role MyCustomDockerRole -
CREATE_IN_PROGRESS AWS::IAM::Role MyCustomDockerRole Resource creation Initiated
CREATE_COMPLETE AWS::IAM::Role MyCustomDockerRole -
CREATE_IN_PROGRESS AWS::Lambda::Function MyCustomDocker -
CREATE_IN_PROGRESS AWS::Lambda::Function MyCustomDocker Resource creation Initiated
CREATE_COMPLETE AWS::Lambda::Function MyCustomDocker -
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi Resource creation Initiated
CREATE_COMPLETE AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeploymentc8d8b9cf15 -
CREATE_IN_PROGRESS AWS::Lambda::Permission MyCustomDockerCustomDockerEchoPermiss -
ionProd
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeploymentc8d8b9cf15 Resource creation Initiated
CREATE_IN_PROGRESS AWS::Lambda::Permission MyCustomDockerCustomDockerEchoPermiss Resource creation Initiated
ionProd
CREATE_COMPLETE AWS::ApiGateway::Deployment ServerlessRestApiDeploymentc8d8b9cf15 -
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage Resource creation Initiated
CREATE_COMPLETE AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_COMPLETE AWS::Lambda::Permission MyCustomDockerCustomDockerEchoPermiss -
ionProd
CREATE_COMPLETE AWS::CloudFormation::Stack lambda-custiom-docker-sample -
-------------------------------------------------------------------------------------------------------------------------------------------------------------
CloudFormation outputs from deployed stack
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Outputs
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Key MyCustomDockerApi
Description API Gateway endpoint URL
Value https://ddcq0ciwhb.execute-api.ap-northeast-1.amazonaws.com/Prod/echo/
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - lambda-custiom-docker-sample in ap-northeast-1
$
Now that it's successful, try accessing the Outputs URL.
$ curl 'https://ddcq0ciwhb.execute-api.ap-northeast-1.amazonaws.com/Prod/echo?param1=val1¶m2=val2'
{"message": "Go Serverless v1.0! Your function executed successfully!", "input": {"resource": "/echo", "path": "/echo", "httpMethod": "GET", "headers": {"Accept": "*/*", "CloudFront-Forwarded-Proto": "https", "CloudFront-Is-Desktop-Viewer": "true", "CloudFront-Is-Mobile-Viewer": "false", "CloudFront-Is-SmartTV-Viewer": "false", "CloudFront-Is-Tablet-Viewer": "false", "CloudFront-Viewer-Country": "JP", "Host": "ddcq0ciwhb.execute-api.ap-northeast-1.amazonaws.com", "User-Agent": "curl/7.64.1", "Via": "2.0 06f6824c0d57ccd48408cb017c7bce76.cloudfront.net (CloudFront)", "X-Amz-Cf-Id": "-lVoexDRq5sBmapw-aMSE8l0ypW4IXMZK3kECxkp4qZVT2ZpLgsAZQ==", "X-Amzn-Trace-Id": "Root=1-5fe08a9d-26bf7d24044bf6127630150f", "X-Forwarded-For": "124.213.96.14, 70.132.19.137", "X-Forwarded-Port": "443", "X-Forwarded-Proto": "https"}, "multiValueHeaders": {"Accept": ["*/*"], "CloudFront-Forwarded-Proto": ["https"], "CloudFront-Is-Desktop-Viewer": ["true"], "CloudFront-Is-Mobile-Viewer": ["false"], "CloudFront-Is-SmartTV-Viewer": ["false"], "CloudFront-Is-Tablet-Viewer": ["false"], "CloudFront-Viewer-Country": ["JP"], "Host": ["ddcq0ciwhb.execute-api.ap-northeast-1.amazonaws.com"], "User-Agent": ["curl/7.64.1"], "Via": ["2.0 06f6824c0d57ccd48408cb017c7bce76.cloudfront.net (CloudFront)"], "X-Amz-Cf-Id": ["-lVoexDRq5sBmapw-aMSE8l0ypW4IXMZK3kECxkp4qZVT2ZpLgsAZQ=="], "X-Amzn-Trace-Id": ["Root=1-5fe08a9d-26bf7d24044bf6127630150f"], "X-Forwarded-For": ["124.213.96.14, 70.132.19.137"], "X-Forwarded-Port": ["443"], "X-Forwarded-Proto": ["https"]}, "queryStringParameters": {"param1": "val1", "param2": "val2"}, "multiValueQueryStringParameters": {"param1": ["val1"], "param2": ["val2"]}, "pathParameters": null, "stageVariables": null, "requestContext": {"resourceId": "co37s7", "resourcePath": "/echo", "httpMethod": "GET", "extendedRequestId": "X5qYlFZ8tjMFcbg=", "requestTime": "21/Dec/2020:11:44:29 +0000", "path": "/Prod/echo", "accountId": "800832305859", "protocol": "HTTP/1.1", "stage": "Prod", "domainPrefix": "ddcq0ciwhb", "requestTimeEpoch": 1608551069250, "requestId": "f7c4fb6d-9621-44c3-8c60-03a0d259ce67", "identity": {"cognitoIdentityPoolId": null, "accountId": null, "cognitoIdentityId": null, "caller": null, "sourceIp": "124.213.96.14", "principalOrgId": null, "accessKey": null, "cognitoAuthenticationType": null, "cognitoAuthenticationProvider": null, "userArn": null, "userAgent": "curl/7.64.1", "user": null}, "domainName": "ddcq0ciwhb.execute-api.ap-northeast-1.amazonaws.com", "apiId": "ddcq0ciwhb"}, "body": null, "isBase64Encoded": false}}
$
The Lambda event data is displayed.
I introduced a sample to run a custom runtime in a docker container. The program itself doesn't make much sense, but it's a lot simpler than the custom runtime tutorial example. Also, I am happy that the operation is guaranteed because the execution environment matches locally and in the cloud (because it runs on the same container).
sample code
https://github.com/ohr486/lambda-container-image-sample
Recommended Posts