Summary of how to write AWS Lambda

There are times when you want to do ○○ using AWS Lambda.

At that time, I had a hard time with low skills in Lambda and programming languages, so I will summarize the strategy for myself and the world.

The only language used in this article is Python.

Basic

Organize what you want to do

First, let's sort out what you want to do with Lambda. (Assuming that it can be realized with other services)

Here is an example.

-** I want to take a snapshot of RDS every 2 hours **

Create an automatic snapshot because it only takes a backup once a day. All you need to do is get a list of RDS instances, get the name of the tagged target RDS instance, and take a snapshot. Alternatively, specify the RDS name and execute the snapshot.

-** I want to take a snapshot of EBS and copy it to another region **

EBS snapshots can be automated using Amazon Data Lifecycle Manager (DLM), but region copies are not possible, so create them. All you need to do is select EBS snapshot completion in the CloudWatch event, set the target to Lambda, and use that as a trigger to get the target EBS name and perform a DR copy.

Let's organize what we want to do in this way. Many of the things I want to do with Lambda seem to be for automation and inter-service collaboration. In the case of automation, think about what part of the operation on the console you want to automate, and in the case of inter-service cooperation, think about the beginning and end. All operations on the AWS console can be operated using APIs, and inter-service cooperation should be able to be connected well using CloudWatch, SNS, etc., so it is relatively easy to organize.

Find the action you want to do in the boto3 documentation

https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

Once you've decided what you want to do with Lambda, look at the boto3 documentation to find what you want to do.

For example, if you want to notify via SNS, refer to the following URL.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sns.html#SNS.Client.publish

** Request Syntax ** is the syntax and ** Returns ** is the result.

It's easy to find, just select the target service in "API reference" from the top page of boto3 and search for the action you want to do. Now let's write Python according to the syntax.


import boto3

client = boto3.client('sns')

response = client.publish(
    TopicArn='xxxxxx',
    Message='Hello',
    Subject='This is test'
)

A description of each parameter is also provided in the documentation. In the above case, "TopicArn" is the ARN of the SNS topic, "Message" is the body, and "Subject" is the subject.

Also, don't forget to create an instance object when using boto3. If you look at the documentation, you'll see a table of contents, but if you select it, it's probably listed first. After that, there is a list of each action, so don't forget to create the instance object you need for what you want to use (it's not ** boto3.client ** for anything). By the way, in the case of "publish ()" of the SNS used above, the following is referred to.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sns.html#client

Use environment variables

Let's make it a general-purpose function by using environment variables. It is easier to manage if you do not write information in the code as much as possible.

In the case of Lambda that notifies the SNS earlier, you can do as follows.


import boto3
import os

client = boto3.client('sns')
arn_sns = os.environ['ARN']

response = client.publish(
    TopicArn=arn_sns,
    Message='Hello',
    Subject='This is test'
)

By doing this, if you want to change the target SNS topic, you only have to modify the environment variables, so you will not touch the code. Therefore, it leads to reduction of human error. Let's make use of it.

Frequently used techniques

Receive the value of a CloudWatch event

For example, if you want to use the value passed from CloudWatch when using CloudWatch as a trigger, the event variable contains the value. The following is triggered by the completion of an EBS snapshot.


import boto3

def lambda_handler(event,context):
    snaphost_create_time = event['time']
    print(snaphost_create_time)

If you google for "cloud watach event ec2 snapshot", the AWS document will be a hit. You can check what is passed, so let's check what you want to get. If you search for other CloudWatch events with the basic "cloudwatch event xxx", AWS documents will appear in the search results, so it's a good idea to see in advance what values will be passed.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-cloud-watch-events.html#create-snapshot-complete

By the way, there are cases where you don't have to create a useless function by selecting "Constant (JSON text)" to control the behavior of Lambda in the target setting of CloudWatch event. The following is Lambda that starts and stops EC2.

Constant {"Action": "start", "Instances": ["************"]}


import boto3

ec2 = boto3.client('ec2')
 
def lambda_handler(event, context):
    instances = event['Instances']
    if event['Action'] == 'start':
        ec2.start_instances(InstanceIds=instances)
        print ('started your instances: ' + ", ".join(instances))
    elif event['Action'] == 'stop':
        ec2.stop_instances(InstanceIds=instances)
        print ('stopped your instances: ' + ", ".join(instances))

This method is based on the following article.

https://www.simpline.co.jp/tech/awslambda%E3%81%A7ec2%E3%81%AE%E8%B5%B7%E5%8B%95%E3%83%BB%E5%81%9C%E6%AD%A2%E3%82%92%E5%88%B6%E5%BE%A1%E3%81%99%E3%82%8B/

Trigger SNS

You can link with various services by selecting the trigger as the SNS topic.

For example, if you want to make a DR copy triggered by a snapshot completion notification of an RDS event subscription, you cannot set the RDS event subscription as a trigger for Lambda, but you can notify the SNS topic once and use that SNS topic as a trigger for Lambda. Can be executed.

In other words, by putting ** SNS topics in between **, each service and Lambda can be linked. SNS can be linked with various services, so if you think that you can not link with Lambda, you may want to check if you can link with SNS.

Read a file from S3

Sometimes I want to read a file from S3.

In such a case, please try as follows.


import boto3

s3 = boto3.client('s3')
bucket_name = os.environ['BUCKET']

def lambda_handler(event, context):
    file_name = 'syukujitsu.csv' 
    
    response = s3.get_object(Bucket=bucket_name, Key=file_name)
    body = response['Body'].read()

    bodystr = body.decode('utf-8')
    lines = bodystr.split('\r\n')
    
    print(lines)

The above is reading a CSV file that lists Japanese holidays.

This method is based on the following article.

https://www.kabegiwablog.com/entry/2018/01/17/090000

What I do personally

-** How to write an instance object, region name is not basic **


region_name = "ap-northeast-1"
ec2 = boto3.client('ec2', region_name=region)

I often see region names written as above, but I don't write them because I think I need to use another region.

-** Environment variables should be placed outside the function **


import boto3

s3 = boto3.client('s3')
bucket_name = os.environ['BUCKET']

def lambda_handler(event, context):
    response = s3.get_object(Bucket=bucket_name, Key=file_name)
    print(response['Body'])

Put the constants outside the function. This is simply not to make a mistake.

-** Avoid using external libraries as much as possible **

If you use an external library, you will have to upload it in a zip file, so implement it in the standard library as much as possible. When testing Lambda, it's just annoying to upload each time.


We hope you find this helpful.

Thanks.

Recommended Posts

Summary of how to write AWS Lambda
Summary of studying Python to use AWS Lambda
Summary of how to use pandas.DataFrame.loc
Summary of how to use pyenv-virtualenv
Summary of how to use csvkit
Summary of how to write .proto files used in gRPC
[Python] Summary of how to use pandas
[Python2.7] Summary of how to use unittest
Summary of how to use Python list
[Python2.7] Summary of how to use subprocess
Summary of how to import files in Python 3
[AWS / Lambda] How to load Python external library
XPath Basics (2) -How to write XPath
[Blender] Summary of how to install / update / uninstall add-ons
How to write a list / dictionary type of Python3
How to set layer on Lambda using AWS SAM
Summary of points I was addicted to running Selenium on AWS Lambda (python)
[Python] Summary of how to specify the color of the figure
How to use Python lambda
Summary of how to share state with multiple functions
[Python] Summary of eval / exec functions + How to write character strings with line breaks
Offline real-time how to write Python implementation example of E14
[python] Summary of how to retrieve lists and dictionary elements
How to enable Read / Write of net.Conn with context with golang
[Linux] [C / C ++] Summary of how to get pid, ppid, tid
[Python] Summary of how to use split and join functions
Summary if using AWS Lambda (Python)
How to write soberly in pandas
Flask reuse How to write html
Write AWS Lambda function in Python
Tweet WakaTime Summary using AWS Lambda
How to write Docker base image
How to add AWS EBS volume
How to write Django1.9 environment-independent wsgi.py
How to get the information of organizations, Cost Explorer of another AWS account with Lambda (python)
How to make AWS Lambda Layers when running selenium × chrome on AWS Lambda
[Hugo] Summary of how to add pages to sites built with Learn
Offline real-time how to write Python implementation example of E15 problem
Basic grammar of Python3 system (how to use functions, closures, lambda functions)
Here's a brief summary of how to get started with Django
How to create a serverless machine learning API with AWS Lambda
scikit-learn How to use summary (machine learning)
How to speed up instantiation of BeautifulSoup
Qiita (1) How to write a code name
How to set optuna (how to write search space)
How to get rid of long comprehensions
How to write Python document comments (Docstrings)
Regularly post to Twitter using AWS lambda!
How to access RDS from Lambda (python)
Jupyter Notebook Basics of how to use
Basics of PyTorch (1) -How to use Tensor-
Connect to s3 with AWS Lambda Python
Summary of vtkThreshold (updated from time to time)
How to write this process in Perl?
How to write Ruby to_s in Python
[AWS] Summary of CLI installation error countermeasures
How to write pydoc and multi-line comments
[Question] How to use plot_surface of python
Tweet in Chama Slack Bot ~ How to make a Slack Bot using AWS Lambda ~
I tried to notify Zabbix Server of execution error of AWS Lambda function
A memorandum of how to write pandas that I tend to forget personally