Notes on accessing SQS from AWS VPC Lambda via endpoint

Conclusion

The endpoint_url option is required when deploying Lambda in a VPC on AWS and using boto3 SQS via an endpoint.

import boto3
sqs = boto3.client("sqs", endpoint_url="https://sqs.ap-northeast-1.amazonaws.com")
sqs.send_message(...)

Without this option, you will not be able to connect and will time out.

Details

Make a note of the mess when deploying Lamba to a VPC on AWS and using SQS with a private connection. For details, please refer to the following site. When I tried it, the symptom only occurred when using boto3 on Lambda.

Under my conditions, Lambda's boto3 was version 1.14.48. Also, the version I tried with EC2 was 1.16.8, but the same phenomenon occurred here as well.

  1. Precautions when using SQS VPC Endpoint with AWS CLI etc .-- https://blog.serverworks.co.jp/tech/2019/03/18/sqsvpcendpoint/
  2. Unable Connect do SQS if using a VPC -- https://github.com/boto/boto3/issues/1900

According to the article in 1., the AWS CLI (including boto3 to see this phenomenon) ** tries to connect to the legacy endpoint even if QueueUrl is specified when connecting to SQS **. It seems to be a problem. When I create an SQS endpoint in a VPC, sqs.ap-northeast-1.amazonaws.com (current endpoint) is assigned a private IP, but ap-northeast-1.queue.amazonaws.com (legacy end) Point) remains the public IP. In that state, when I try to use SQS with boto3, it seems that the cause is that I tried to use a legacy endpoint when connecting, but I could not connect and timed out.

The solution using Session is shown in 2. GitHub, but if you specify the current endpoint in the ʻendpoint_urloption as above when executingboto3.client ()`, the problem is solved. For reference, the solution code by Session in the article on GitHub is as follows.

import boto3
session = boto3.Session()
sqs_client = session.client(
    service_name='sqs',
    endpoint_url='https://sqs.ap-northeast-1.amazonaws.com',
)
sqs_client.send_message(...)

... I got stuck for about 2 hours.

Recommended Posts

Notes on accessing SQS from AWS VPC Lambda via endpoint
Notes on accessing dashDB from python
Tweet from AWS Lambda
Things to note when running Python on EC2 from AWS Lambda
Notes on using MeCab from Python
Run Python on Schedule on AWS Lambda
Python calling Google Cloud Vision API from LINE BOT via AWS Lambda
Modules cannot be imported in Python on EC2 run from AWS Lambda
[Python] Run Headless Chrome on AWS Lambda