For example, if you regularly pull data from somewhere with AWS Lambda + Amazon CloudWatch Events and put it in Amazon RDS appropriately, it costs at least several thousand yen / month for a normal Provisioned instance. However, if you use Aurora Serverless, it will sleep without permission when you are not using it, so it may be cheap. [^ pricing] [^ serverless]
[^ pricing]: I tried to calculate the usage cost of Amazon Aurora Serverless | Developers.IO [^ serverless]: What to watch out for when installing Aurora Serverless | Developers.IO
However, there is one problem with this idea: ** Aurora Serverless is slow to happen **.
Aurora Serverless, which was sleeping when trying to connect from a DB client, starts booting, but sometimes the client times out before it can accept the connection.
Therefore, when using it, you should send a request to start Aurora Serverless (increase Capacity) in advance, and connect from the DB client when it starts (Capacity becomes 0 or more).
Try implementing a Lambda Function that waits for Aurora Serveless to occur and then does nothing.
The runtime used Python 3.8.
import asyncio
import boto3
def is_aurora_serverless_up(client, identifier: str) -> bool:
"""Returns whether Aurora Serverless is running"""
response = client.describe_db_clusters(DBClusterIdentifier=identifier)
assert response['ResponseMetadata']['HTTPStatusCode'] == 200
assert len(response['DBClusters']) > 0
assert response['DBClusters'][0]['EngineMode'] == 'serverless'
return response['DBClusters'][0]['Capacity'] > 0
async def wake_aurora_serverless_up(client, identifier: str, capacity: int = 2):
"""Start Aurora Serverless"""
if is_aurora_serverless_up(client, identifier):
return
response = client.modify_current_db_cluster_capacity(DBClusterIdentifier=identifier, Capacity=capacity)
assert response['ResponseMetadata']['HTTPStatusCode'] == 200
for i in range(10):
await asyncio.sleep(i ** 2)
if is_aurora_serverless_up(client, identifier):
return
raise TimeoutError()
async def main():
client = boto3.client('rds')
await wake_aurora_serverless_up(client, 'mycluster')
def lambda_handler(event, context):
asyncio.get_event_loop().run_until_complete(main())
What I need to do later.
--Because you need the privileges of rds: DescribeDBClusters
and rds: ModifyCurrentDBClusterCapacity
to execute, create an IAM Role like that and assign it to the Lambda Function.
--If the default value of the Lambda Function timeout setting is 3 seconds, it will never end, so set it to 3 minutes.
This time it is implemented in asyncio, but it doesn't make much sense because it is only used in asyncio.sleep. You can use time.sleep as usual. Implementing with asyncio has the advantage that asyncpg can be selected as the PostgreSQL client.
If you try it, you will find the following.
--It takes about 15 seconds for Aurora Serverless to start --Even if you start it (capacity> 0), you cannot connect from the DB client immediately, and wait another 10 seconds from there.
It hasn't changed much since I waited after all, but this is fine because the DB client no longer times out.
After doing so far, I realized that "Isn't this solved by just setting the connection timeout of the DB client longer?"
import psycopg2
with psycopg2.connect('postgresql://...', connect_timeout=120) as conn:
...
(In case of psycopg2) Now it works fine.
Eh
Ah, yes ...