[AWS] Try tracing API Gateway + Lambda with X-Ray
What is X-Ray?
A service for analyzing and debugging distributed applications. Click here for details. AWS X-Ray
Premise
Again, I will try to build a sample with the following configuration.
- SAM
- Python3.8
For basic knowledge such as SAM, please refer to here.
-[AWS] Serverless Application Model (SAM) Basic Summary -[AWS] Create API Gateway + Lambda + DynamoDB sample with Serverless Application Model (SAM)
Let's make a sample
Project creation
As usual, we'll create a Hello World-based project in SAM.
$ sam init --runtime=python3.8
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template Location
Choice: 1
Project name [sam-app]:
Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git
AWS quick start application templates:
1 - Hello World Example
2 - EventBridge Hello World
3 - EventBridge App from scratch (100+ Event Schemas)
4 - Step Functions Sample App (Stock Trader)
5 - Elastic File System Sample App
Template selection: 1
-----------------------
Generating application:
-----------------------
Name: sam-app
Runtime: python3.8
Dependency Manager: pip
Application Template: hello-world
Output Directory: .
Next steps can be found in the README file at ./sam-app/README.md
Addition of X-Ray
Add decorator to source code
First, add the X-Ray decorator to your code.
Try the following, including refactoring the hello_world / app.py generated by default.
hello_world/app.py
import json
from aws_xray_sdk.core import xray_recorder
@xray_recorder.capture('hello world')
def lambda_handler(event, context):
return {
"statusCode": 200,
"body": json.dumps({
"message": "hello world",
})
}
Import SDK
To get the X-Ray SDK library into Lambda, modify hello_world / requirements.txt as follows.
hello_world/requirements.txt
aws-xray-sdk
Modify SAM template
Finally, in order to enable tracing of Lambda function and API Gateway, modify the Globals part as follows.
template.before yml modification
Globals:
Function:
Timeout: 3
template.after yml modification
Globals:
Function:
Timeout: 3
Tracing: Active
Api:
TracingEnabled: True
Build
$ sam build
Building function 'HelloWorldFunction'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided
Deploy
$ sam deploy --guided
Configuring SAM deploy
======================
Looking for samconfig.toml : Found
Reading default arguments : Success
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]:
AWS Region [us-east-1]: ap-northeast-1
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [Y/n]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: y
HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Save arguments to samconfig.toml [Y/n]: y
Looking for resources needed for deployment: Found!
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
A different default S3 bucket can be set in samconfig.toml
Deploying with following values
===============================
Stack name : sam-app
Region : ap-northeast-1
Confirm changeset : True
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Initiating deployment
=====================
Saved arguments to config file
Running 'sam deploy' for future deployments will use the parameters saved above.
The above parameters can be changed by modifying samconfig.toml
Learn more about samconfig.toml syntax at
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Deploying with following values
===============================
Stack name : sam-app
Region : ap-northeast-1
Confirm changeset : True
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Initiating deployment
=====================
HelloWorldFunction may not have authorization defined.
Uploading to sam-app/8476bddd8c14756a7c801a61352b828d.template 1142 / 1142.0 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType
------------------------------------------------------------------------------------------------
+ Add HelloWorldFunctionHelloWorldPe AWS::Lambda::Permission
rmissionProd
+ Add HelloWorldFunctionRole AWS::IAM::Role
+ Add HelloWorldFunction AWS::Lambda::Function
+ Add ServerlessRestApiDeployment47f AWS::ApiGateway::Deployment
c2d5f9d
+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage
+ Add ServerlessRestApi AWS::ApiGateway::RestApi
------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:ap-northeast-1:************:changeSet/samcli-deploy1597324152/18c44433-12d1-4e03-9fb8-00737d018991
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
2020-08-13 22:09:28 - Waiting for stack create/update to complete
CloudFormation events from changeset
-------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
-------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole Resource creation
Initiated
CREATE_COMPLETE AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction -
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction Resource creation
Initiated
CREATE_COMPLETE AWS::Lambda::Function HelloWorldFunction -
CREATE_IN_PROGRESS AWS::ApiGateway::RestA ServerlessRestApi Resource creation
pi Initiated
CREATE_IN_PROGRESS AWS::ApiGateway::RestA ServerlessRestApi -
pi
CREATE_COMPLETE AWS::ApiGateway::RestA ServerlessRestApi -
pi
CREATE_IN_PROGRESS AWS::ApiGateway::Deplo ServerlessRestApiDeplo -
yment yment47fc2d5f9d
CREATE_IN_PROGRESS AWS::ApiGateway::Deplo ServerlessRestApiDeplo Resource creation
yment yment47fc2d5f9d Initiated
CREATE_IN_PROGRESS AWS::Lambda::Permissio HelloWorldFunctionHell Resource creation
n oWorldPermissionProd Initiated
CREATE_IN_PROGRESS AWS::Lambda::Permissio HelloWorldFunctionHell -
n oWorldPermissionProd
CREATE_COMPLETE AWS::ApiGateway::Deplo ServerlessRestApiDeplo -
yment yment47fc2d5f9d
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdS -
tage
CREATE_COMPLETE AWS::Lambda::Permissio HelloWorldFunctionHell -
n oWorldPermissionProd
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdS Resource creation
tage Initiated
CREATE_COMPLETE AWS::ApiGateway::Stage ServerlessRestApiProdS -
tage
CREATE_COMPLETE AWS::CloudFormation::S sam-app -
tack
-------------------------------------------------------------------------------------------------
CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------
Key HelloWorldFunctionIamRole
Description Implicit IAM Role created for Hello World function
Value arn:aws:iam::************:role/sam-app-HelloWorldFunctionRole-IIPXQC9S1XKJ
Key HelloWorldApi
Description API Gateway endpoint URL for Prod stage for Hello World function
Value https://tws0qc6nbc.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
Key HelloWorldFunction
Description Hello World Lambda Function ARN
Value arn:aws:lambda:ap-northeast-1:************:function:sam-app-
HelloWorldFunction-1OH75PSLQUSLC
-------------------------------------------------------------------------------------------------
Successfully created/updated stack - sam-app in ap-northeast-1
Run
curl https://tws0qc6nbc.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message": "hello world"}
Succeeded.
Verification
Now, let's check the X-Ray Service Map, Traces, and analytics in the management console.
Service Map
You can check how the API Gateway and Lambda are called in order from the client and the latency of each.
Traces
Here you can check the status of the trace. Here you can filter according to each condition.
- URL
- StatusCOde
- Method
- User
- UserAgent
- ClientIP
- InstanceId
- ResourceARN
- Response time root causes
- Error root causes
- Error root cause messages
- Fault root causes
- Fault root cause messages
- Annotation aws:api id
Analytics
Finally, analytics, as the name implies, can perform a variety of analyzes and comparisons.
Try tracing AWS services
Now let's trace the AWS service. This time, let's add a trace for PutItem to DynamoDB.
For details, refer to "[AWS] Try creating an API Gateway + Lambda + DynamoDB sample with Serverless Application Model (SAM)". I would like to explain only the part related to X-Ray with points.
Add code
First, create a directory for Function separately from HelloWorld, and create files in it.
$ mkdir sam_ddb
$ touch sam_ddb/app.py
$ touch sam_ddb/requirements.txt
First, the body of the Lambda function, but the important part here is the patch ['boto3'] part.
By doing this, you will be able to trace boto3 AWS service calls.
sam_ddb/app.py
import json
import boto3
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch
from datetime import datetime
patch(['boto3'])
@xray_recorder.capture('put_item ddb')
def lambda_handler(event, context):
event_body = json.loads(event["body"])
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("Demo")
table.put_item(
Item={
"Key": event_body["key"],
"CreateDate": datetime.utcnow().isoformat()
}
)
return {
"statusCode": 200,
"body": json.dumps({
"message": "succeeded",
}),
}
sam_ddb/requirements.txt
aws-xray-sdk
boto3
Finally, edit template.yml.
The parts that need to be changed are Resources and ʻOutputs`.
Please note that IAM Role requires permissions for X-Ray.
template.yml before change
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sam-app
Sample SAM Template for sam-app
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Tracing: Active
Api:
TracingEnabled: True
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.8
Events:
HelloWorld:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /hello
Method: get
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
HelloWorldApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
HelloWorldFunction:
Description: "Hello World Lambda Function ARN"
Value: !GetAtt HelloWorldFunction.Arn
HelloWorldFunctionIamRole:
Description: "Implicit IAM Role created for Hello World function"
Value: !GetAtt HelloWorldFunctionRole.Arn
template.after changing yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sam-app
Sample SAM Template for sam-app
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Tracing: Active
Api:
TracingEnabled: True
Resources:
DynamoTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: Demo
AttributeDefinitions:
- AttributeName: Key
AttributeType: S
- AttributeName: CreateDate
AttributeType: S
KeySchema:
- AttributeName: Key
KeyType: HASH
- AttributeName: CreateDate
KeyType: RANGE
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.8
Events:
HelloWorld:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /hello
Method: get
SamDdbFunction:
Type: AWS::Serverless::Function
Properties:
Role: !GetAtt SamDdbFunctionIamRole.Arn
CodeUri: sam_ddb/
Handler: app.lambda_handler
Runtime: python3.8
Events:
SamDdb:
Type: Api
Properties:
Path: /ddb
Method: post
SamDdbFunctionIamRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- 'lambda.amazonaws.com'
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/CloudWatchLogsFullAccess'
Policies:
- PolicyName: 'SamDdbPolicy'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- dynamodb:PutItem
Resource: !GetAtt DynamoTable.Arn
- Effect: Allow
Action:
- xray:PutTraceSegments
- xray:PutTelemetryRecords
Resource: '*'
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
HelloWorldApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
HelloWorldFunction:
Description: "Hello World Lambda Function ARN"
Value: !GetAtt HelloWorldFunction.Arn
HelloWorldFunctionIamRole:
Description: "Implicit IAM Role created for Hello World function"
Value: !GetAtt HelloWorldFunctionRole.Arn
SamDdbApi:
Description: "API Gateway endpoint URL for Prod stage for SAM DDB function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/ddb/"
Build and deploy
Run sam build and sam deploy --guided.
(Details omitted)
Run
Since we have added one API this time, we will also call the Hello World API for comparison.
$ curl https://fp8nyhpv87.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message": "hello world"}
Next, let's call the Post API to write to DynamoDB.
$ curl -X POST -H "Content-Type: application/json" -d '{"key": "demo-data"}' https://fp8nyhpv87.execute-api.ap-northeast-1.amazonaws.com/Prod/ddb/
{"message": "succeeded"}
X-Ray confirmation
First, from the Service Map. The arrangement is twisted for some reason, but you can see that the access part to DynamoDB is cut out and displayed. You can see that the writing part of the DB takes 620ms.
The same is true for Traces. The DynamoDB part can be traced independently.
Check DynamoDB just in case
The data is registered perfectly.
Summary
Tracing with X-Ray enables various analyses. You can easily add it after the fact, but if possible, we recommend that you introduce it from an early stage.
Sample code repository
https://github.com/hito-psv/sam-demo-003
Recommended Posts