[RUBY] AWS Lambda says Container Image Support, so pseudo Cloud Run

Introduction

Merry Christmas! !! : christmas_tree:: christmas_tree:: santa:: christmas_tree :: christmas_tree:

The 25th day of Ateam Lifestyle Advent Calendar 2020 Ateam Lifestyle Inc. Executive Officer CTO @tsutorm is in charge!

I heard that Lambda supports container images

What should I do with this year's ad-care: thinking:

It will be a poem system, but let's write something like "development productivity" of engineers who can easily make mistakes in business.

Christmas: christamas_tree: That's right. It's better to have a code that will be a gift

Where I thought ...

image.png

[Breaking News] Container images are now supported as a Lambda package format! !! #reinvent

I thought it was Cloud Ru ..., but is it a little different?

-Try Lambda Container Support with AWS SAM CLI -Explosive speed? !! I tried to verify the cold start of Lambda deployed from the container image #reinvent

There are already some articles that I touched (everyone is early), but it seems that it is not Cloud Run as I expected.

――I thought it was a Cloud Run-like guy with API Gateway on Lambda. -BaseImage is like Amazon Linux2, so if you create a Dockerfile properly, Rails will work in principle. --There are Aurora Serverless and RDS Proxy, and I think I can make Web API.

...

Let's try it together!

Rails API on AWS Lambda Run (provisional) It works

$ #Please do something like REST or seeds because Rails is used for data stuffing.
$ curl https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts  # GET /posts
[{"id":1,"title":"test","body":"hoge","published":null,"created_at":"2020-12-23T11:31:39.626Z","updated_at":"2020-12-23T11:31:39.626Z"},{"id":2,"title":"test2","body":"hoge","published":null,"created_at":"2020-12-23T12:11:04.785Z","updated_at":"2020-12-23T12:11:04.785Z"},{"id":3,"title":"test3","body":"hoge","published":null,"created_at":"2020-12-23T13:29:04.307Z","updated_at":"2020-12-23T13:29:04.307Z"}]

START RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40 Version: $LATEST
I, [2020-12-24T06:35:23.744874 #8]  INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Started GET "/posts" for 64.252.172.83 at 2020-12-24 06:35:23 +0000
I, [2020-12-24T06:35:23.746365 #8]  INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Processing by PostsController#index as */*
I, [2020-12-24T06:35:23.960053 #8]  INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Completed 200 OK in 214ms (Views: 212.8ms | ActiveRecord: 27.8ms | Allocations: 6188)
END RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40
REPORT RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40	Duration: 220.86 ms	Billed Duration: 3193 ms	Memory Size: 1024 MB	Max Memory Used: 158 MB	Init Duration: 2971.41 ms

Overall feeling

Overall view of composition

It's not as complicated as it sounds, but it looks like this in the figure undefined.png

Only that

code

I uploaded it to github.

As an application, it's a very simple one that just adds a certain post model to something like rails new --api, so what happens if you fit an existing Rails application that you made tightly? I think that will be another problem. ..

I threw everything around deploy to serverless. Very convenient.

The docker build is partly shelled, but the official documentation didn't seem too difficult.

I did only the construction of Aurora Serverless manually, but since it is not such a difficult story, I will omit the procedure explanation here.

build & deploy

I'm sorry I couldn't script it completely, but it's about below

--build to create an image ./build.sh --Enter the information required for deploy, including image hash, in dev.yml

$ aws lambda invoke --function-name lambda-run-dev-migrate --payload '{}' response.json # migration

Where I was addicted, where I stumbled

Dockerfile.lambda

I don't know the choice of my image including Base Image or Ruby runtime interface clients provided by AWS.

-Official document It's a story that can be read properly. ――At first I wrote a Dockerfile based on Alpine, but I don't like having to put aws_lambda_ric because it was invaded by Gemfile. Recreated with official Base Image --AWS-provided BaseImage is only Ruby 2.5/2.7

FROM public.ecr.aws/lambda/ruby:2.7.2020.12.18.21

BaseImage has aws_lambda_ric specified in advance in entrypoint, so if you do docker run -it public.ecr.aws/lambda/ruby: 2.7/bin/sh, it will not accept input: thinking:

before

$ docker run --rm -it public.ecr.aws/lambda/ruby:2.7 /bin/sh
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
#like this

after

$ docker run --rm --entrypoint="" -it public.ecr.aws/lambda/ruby:2.7 /bin/sh
sh-4.2#

I'm writing Dockerfile while doing yum install and playing with Areya Koreya

serverless

Basically, only I referred to it because there was a very easy-to-understand entry

Overwriting of CMD and ENTRYPOINT is not yet supported

Avoid by preparing another function to rails db: migrate with another image

# Dockerfile.lambda
#Abbreviation
FROM app as migrate

WORKDIR /var/task

#Omission

CMD ["lambda.App::Handler.migrate"]

# lambda.rb
#Abbreviation
    def self.migrate(event:, context:)
      p `bin/rails db:create`
      p `bin/rails db:migrate`
      response = {
        'statusCode' => 204
      }
    end
end

If you want to connect with Aurora, you have to use VPC Lambda

I completely forgot and made it normally, so I remade it. It was serverless, so it's easy with serverless remove && serverless deploy

# serverless.yml
  vpc:
    securityGroupIds:
      - ${self:custom.environment.${self:provider.stage}.MY_AWS_SECURITY_GROUP}
    subnetIds:
      - ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_0}
      - ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_1}
      - ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_2}
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ec2:CreateNetworkInterface"
        - "ec2:DescribeNetworkInterfaces"
        - "ec2:DeleteNetworkInterface"
      Resource:
        - "*"

Rails on Lambda

How do you get Rails running from Lambda in the first place? ??

I tried Ruby on Rails on AWS Lambda and API Gateway by Serverless Framework was saved because I had a predecessor. Almost this. Thank you very much.

It works with aws_lambda_ric → lambda.rb → config.ru.

The code passed from the handler to rack changed only the reference path of config.ru based on the following.

https://github.com/aws-samples/serverless-sinatra-sample/blob/master/lambda.rb

Therefore, Rails does not start puma, which is a common application server:: exclamation:: exclamation:

As a result, I feel that it has the benefit of reducing the puma startup overhead that was expected to occur during invoke.

Scales from the eyes. Botamochi from the shelf.

Since / var/task / of WORKDIR is write-protected, all the processing to write to Rails.root.join ('tmp') is an error

The main thing I was addicted to this time was the cache file in bootsnap and tmp/log/*. Log

bootsnap cache cannot be written to tmp/cache

Avoid using environment variables by changing the output destination of the bootsnap cache to / tmp as per the bootsnap documentation

# Dockerfile.lambda
ENV BOOTSNAP_CACHE_DIR=/tmp/cache
Changed to output log to STDOUT because log cannot be written to tmp/log/development.log

I'm having trouble getting on cloudwatch logs normally, so I just do this, and there is a story that it was RAILS_ENV = development at that time. You should also write it in config/environments/development.rb

# config/environments/development.rb
  if ENV["RAILS_LOG_TO_STDOUT"].present?
    logger           = ActiveSupport::Logger.new(STDOUT)
    logger.formatter = config.log_formatter
    config.logger    = ActiveSupport::TaggedLogging.new(logger)
  end

# Dockerfile.lambda
ENV RAILS_LOG_TO_STDOUT=1

How it feels when loaded

Resource status when only one request is thrown

cold start

$ hay -c 1 -n 1 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts

Summary:
  Total:        3.7599 secs
  Slowest:      3.7597 secs
  Fastest:      3.7597 secs
  Average:      3.7597 secs
  Requests/sec: 0.2660

  Total data:   408 bytes
  Size/request: 408 bytes

Response time histogram:
  3.760 [1]     |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |
  3.760 [0]     |


Latency distribution:
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.1575 secs, 3.7597 secs, 3.7597 secs
  DNS-lookup:   0.1406 secs, 0.1406 secs, 0.1406 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0001 secs
  resp wait:    3.6014 secs, 3.6014 secs, 3.6014 secs
  resp read:    0.0006 secs, 0.0006 secs, 0.0006 secs

Status code distribution:
  [200] 1 responses

Warm start

$ hay -c 1 -n 1 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts

Summary:
  Total:        0.0556 secs
  Slowest:      0.0556 secs
  Fastest:      0.0556 secs
  Average:      0.0556 secs
  Requests/sec: 17.9856

  Total data:   408 bytes
  Size/request: 408 bytes

Response time histogram:
  0.056 [1]     |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |
  0.056 [0]     |


Latency distribution:
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs
  0% in 0.0000 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0159 secs, 0.0556 secs, 0.0556 secs
  DNS-lookup:   0.0052 secs, 0.0052 secs, 0.0052 secs
  req write:    0.0003 secs, 0.0003 secs, 0.0003 secs
  resp wait:    0.0386 secs, 0.0386 secs, 0.0386 secs
  resp read:    0.0005 secs, 0.0005 secs, 0.0005 secs

Status code distribution:
  [200] 1 responses

Aurora works with capacity 1

image.png

When loaded (300 parallel / 3000 requests)

$ hay -c 300 -n 3000 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts

Summary:
  Total:        7.3692 secs
  Slowest:      5.1843 secs
  Fastest:      0.0223 secs
  Average:      0.4378 secs
  Requests/sec: 407.1016

  Total data:   370236 bytes
  Size/request: 123 bytes

Response time histogram:
  0.022 [1]     |
  0.538 [2645]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.055 [59]    |■
  1.571 [3]     |
  2.087 [0]     |
  2.603 [2]     |
  3.120 [5]     |
  3.636 [20]    |
  4.152 [255]   |■■■■
  4.668 [9]     |
  5.184 [1]     |


Latency distribution:
  10% in 0.0351 secs
  25% in 0.0404 secs
  50% in 0.0490 secs
  75% in 0.0728 secs
  90% in 0.8516 secs
  95% in 3.8275 secs
  99% in 4.0212 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0331 secs, 0.0223 secs, 5.1843 secs
  DNS-lookup:   0.0079 secs, 0.0000 secs, 0.1565 secs
  req write:    0.0009 secs, 0.0000 secs, 0.2680 secs
  resp wait:    0.3979 secs, 0.0222 secs, 4.7219 secs
  resp read:    0.0004 secs, 0.0000 secs, 0.1758 secs

Status code distribution:
  [200] 907 responses
  [500] 2093 responses

Don't be afraid. The cause is insufficient capacity on the RDS side, so if you stand for a certain period of time ...

$ hay -c 300 -n 3000 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts

Summary:
  Total:        1.3416 secs
  Slowest:      0.7870 secs
  Fastest:      0.0215 secs
  Average:      0.1167 secs
  Requests/sec: 2236.2066

  Total data:   844968 bytes
  Size/request: 281 bytes

(Abbreviation)

Status code distribution:
  [200] 2071 responses
  [500] 929 responses

image.png

Aurora is also serverless, so the capacity increases without permission and it can be handled without permission! !! !!

After a certain amount of time ...

image.png

I'll scale in towards zero

Where I couldn't touch

ActiveJob (sidekiq) will also be Lambda

It seems that you can just use aws-sdk-rails gem to make the handler Aws :: Rails :: SqsActiveJob.lambda_job_handler

Cost comparison with ECS Fargate/EKS

Lambda should be bad in the case of commercial use where there is a certain workload above a certain level. I think.

Summary

――I'm glad it was fun. -I feel good with Cloud Run. I'm fully aware of the story There are also Jets. I have no regrets. ――I don't know if it's a huge Rails. Insufficient verification.

in conclusion

This is the end of Ateam Lifestyle Advent Calendar 2020. How was that?

Thank you @engabesi, @asasigure for moving to the operation!

2020 was a turbulent year, mainly with Covid-19.

What kind of year will 2021 be next year? I'm looking forward to it!

Then everyone Happy Holidays !!!: tada:

Recommended Posts

AWS Lambda says Container Image Support, so pseudo Cloud Run
AWS Lambda supports container images so I tried Puppeteer
Run lambda with custom docker image
Run C binaries on AWS Lambda