[RAILS] Deploy to EC2 with CircleCi + Capistrano

Reason for writing the article

This is my first article creation. I was able to deploy the method for deploying to EC2 with CircleCi + Capistrano by referring to other articles, so I summarized the contents as an article creation practice.

Prerequisites

--CircleCi GUI is as of the time of writing (January 19, 2021). --Deployment to EC2 instance is completed locally with Capistrano. --RSpec and RuboCop already work on CircleCi. (If you haven't done so already, here was helpful.)

Note: This article is for an SSH connection to an EC2 instance with any IP address. (We will add it as soon as it is supported.)

Preparation (SSH key format change)

First, change the format of the SSH key to register with CircleCi. You probably created it to access your EC2 instance in the past, referring to the article here. To change the format of your SSH key (~/.ssh/your_app_key_rsa), do the following:

$ chmod 700 ~/.ssh/your_app_key_rsa
$ ssh-keygen -p -m PEM -f ~/.ssh/your_app_key_rsa
$ chmod 400 ~/.ssh/your_app_key_rsa
$ cat ~/.ssh/your_app_key_rsa  #Register the contents of the file displayed by this command in CircleCi.

It is OK if the contents of the file are "BEGIN RSA PRIVATE KEY" as shown below. -----BEGIN RSA PRIVATE KEY----- A list of letters including numbers -----END RSA PRIVATE KEY-----

Reference: [AWS] Resolution of errors addicted to CircleCI/CD automatic deployment [Capistrano]

Operation with CircleCi (SSH key registration environment variable registration)

SSH key registration

Next, perform the following operations to register the SSH key whose format has been changed to CircleCi.

1.Click "Project Settings" at the top right of the screen of the repository page registered with CircleCi
2.Select "SSH KEYS" on the left side of the screen.
3.Select "Add SSH Key" to the right of the "Additional SSH Keys" item at the bottom of the screen.
4.Enter the "Elastic IP" assigned to the EC2 instance in the "Host name" of the displayed item.
5.「Private Key*To ""~/.ssh/your_app_key_Copy and paste the contents of "rsa".
6."Hostname" and "Private Key"*After entering, register the SSH key with "Add SSH Key".

Supplement

In 5, please copy not only the enumeration of numbers and letters but also the following. -----BEGIN RSA PRIVATE KEY----- -----END RSA PRIVATE KEY----- When registration is completed in 6, "Elastic IP" will be added to "Hostname" in the "Additional SSH Keys" item. The Fingerprint should say something like "12:34:56: ab ...". The displayed "Fingerprint" will be used to register the "environment variables" that will be performed later. (Environment variables do not include ":".)

When the file is generated, it is generated as ~/.ssh/id_rsa_ followed by Fingerprint without:. If you use the registered SSH key, it will be automatically added to "ssh-agent". Even if you are using "ssh-agent" to connect to GitHub from your server environment, you only need to register one SSH key to use to connect to your EC2 instance locally.

Environment variable registration

Then add an environment variable to Circleci.

1.Current screen("SSH KEYS screen")Select "Environment Variables" to the left of.
2.Then select Add Environment Variable.
3.An item similar to the SSH key registration will appear, so "Name"*As a variable name in "PRODUCTION"_SSH_Enter "KEY".
4. 「Value*To ""~/.ssh/id_rsa_Like "123456abcd"~/.ssh/id_rsa_I registered behind
From the contents of "Fingerprint" of the SSH key, ":"Is omitted.
5.Register the environment variable with "Add Environment Variable".

Supplement

Even if multiple SSH keys are registered in CircleCi, they are saved as the file name of "id_rsa_Fingerprint: omitted" under "~/.ssh", so the path to the SSH key is registered as an environment variable. If you want to check, please refer to the official Debugging with SSH.

Editing Capistrano files

It's just a change of keys :, so it's done right away.

config/deploy/production.rb


server Rails.application.credentials.dig(:amazon, :ec2_ip),
       user: Rails.application.credentials.dig(:amazon, :ec2_user),
       roles: %w[web app db]

set :ssh_options, {
  keys: (ENV['PRODUCTION_SSH_KEY']), #Edit only here
  forward_agent: true
}

Editing CircleCi files

I edited it for myself by referring to the contents of Running SystemSpec (RSpec) and Rubocop on CircleCI.

yml:.circleci/config.yml(Before editing)


version: 2.1

orbs:
  ruby: circleci/[email protected]

jobs:
  build:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps

  test:
    parallelism: 3
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - DB_HOST: 127.0.0.1
          - RAILS_ENV: test
          - BUNDLER_VERSION: 2.1.4
      - image: circleci/mysql:8.0
        command: |
          mysqld --default-authentication-plugin=mysql_native_password
        environment:
          - MYSQL_ROOT_PASSWORD: password
          - MYSQL_USER: root
          - MYSQL_ROOT_HOST: '%'
      - image: selenium/standalone-chrome:latest
        name: chrome

    steps:
      - checkout
      - ruby/install-deps
      - run: mv config/database.yml.ci config/database.yml
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:3306 -timeout 1m
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load
      # Run rspec in parallel
      - ruby/rubocop-check
      - ruby/rspec-test

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - test:
          requires:
            - build

Add the following part to this by referring to I tried to automatically deploy with CircleCI + Capistrano + AWS (EC2) + Rails.

yml:.circleci/config.yml(Additional points)


#abridgement
  deploy:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps
      - add_ssh_keys:
          fingerprints:"XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      - deploy:
          name: Capistrano deploy
          command: bundle exec cap production deploy

workflows:
  version: 2
  build_accept_deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

yml:.circleci/config.yml(Overall)


version: 2.1

orbs:
  ruby: circleci/[email protected]

jobs:
  build:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps

  test:
    parallelism: 3
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - DB_HOST: 127.0.0.1
          - RAILS_ENV: test
          - BUNDLER_VERSION: 2.1.4
      - image: circleci/mysql:8.0
        command: |
          mysqld --default-authentication-plugin=mysql_native_password
        environment:
          - MYSQL_ROOT_PASSWORD: password
          - MYSQL_USER: root
          - MYSQL_ROOT_HOST: '%'
      - image: selenium/standalone-chrome:latest
        name: chrome

    steps:
      - checkout
      - ruby/install-deps
      - run: mv config/database.yml.ci config/database.yml
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:3306 -timeout 1m
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load
      # Run rspec in parallel
      - ruby/rubocop-check
      - ruby/rspec-test

  deploy:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps
      - add_ssh_keys:
          fingerprints: "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      - deploy:
          name: Capistrano deploy
          command: bundle exec cap production deploy

workflows:
  version: 2
  build_accept_deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

Supplement

      - add_ssh_keys:
          fingerprints: "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"

The part of


1.Click "Project Settings" at the top right of the screen of the repository page registered with CircleCi
2.Select "SSH KEYS" on the left side of the screen.
3.There is an SSH key added at the bottom of the screen, so Fingerprint (12):34:56:ab:cd:ef:...Like)
Copy and paste.

Note

Since the IP address used by CircleCi is not always the same, it is not possible to deploy if the security group limits the IP address that can be SSH connected. (Can be deployed if not limited) The next step is to address this security issue.

Reference: Run SystemSpec (RSpec) and Rubocop on CircleCI I tried automatic deployment with CircleCI + Capistrano + AWS (EC2) + Rails

Allow deployment only with specific IP addresses

Next, we will make it possible to deploy only with the IP address (My IP) limited by the security group and the IP address used on CircleCi.

Settings on AWS

First, create an IAM user by referring to this article and be sure to download the CSV (to register the contents of the CSV as an environment variable in CircleCi). Then do the following:

1.Select "EC2" from the service.
2.Select "Security Group" from the EC2 dashboard.
3.Select "Create Security Group" at the top right of the screen.
4."CircleCi" in "Security Group Name" and "Description"-Enter "Security Group".
5.Select the VPC that has the deployment destination instance in "VPC".
6.Create by selecting "Create Security Group".
7.Select "Instance" from the EC2 dashboard.
8.Select "Change Security Group" from "Action" of the instance to deploy to
9.Add the created security group to "Associated Security Group".

This completes the settings on AWS.

Supplement

There is no need to add inbound rules when creating a security group. The ID of the created security group will be registered in CircleCi later as an environment variable.

Settings on CircleCi

CircleCI adds the following four as environment variables. 「AWS_ACCESS_KEY_ID」「AWS_SECRET_ACCESS_KEY」「AWS_DEFAULT_REGION」「SECURITY_GROUP_ID」

Register the contents of the CSV downloaded earlier with the following names.

Name to "AWS_ACCESS_KEY_Register the value of "Access key ID" in "ID" and Value.
Name to "AWS_SECRET_ACCESS_Register the values ​​of "KEY" and "Secret access key".

Register the remaining two environment variables.

Name to "AWS_DEFAULT_"REGION", Value to "ap"-northeast-Enter "1" to register.
Name is "SECURITY"_GROUP_Register "ID of the security group created earlier" in "ID" and Value.

This completes the settings on CircleCi.

Supplement

I think that the value of "AWS_DEFAULT_REGION" is basically "ap-northeast-1", but if it is different, please register an appropriate region name.

Reference: I want to allow a specific IP only when accessing EC2 from CircleCI 2.0

Edit files and create shell scripts

Now let's edit the configuration file. (At the very end there is the entire source code)

yml:.circleci/config.yml(changes)


orbs:
  ruby: circleci/[email protected]
  aws-cli: circleci/[email protected] #add to

jobs:
#abridgement
  deploy:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps
      - aws-cli/install #add to
      - add_ssh_keys:
          fingerprints: "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      - deploy:
          name: Capistrano deploy
          command: ./deploy.sh #Changed to run shell script

Next, I created a shell script with the content "deploy.sh" based on this article. (The content has been edited a little with reference to this article.)

deploy.sh


#!/bin/sh
set -ex

IP=`curl -f -s ifconfig.me`

trap "aws ec2 revoke-security-group-ingress --group-id ${SECURITY_GROUP_ID} --protocol tcp --port 22 --cidr ${IP}/32" 0 1 2 3 15
aws ec2 authorize-security-group-ingress --group-id ${SECURITY_GROUP_ID} --protocol tcp --port 22 --cidr ${IP}/32
bundle exec cap production deploy

Finally, change the authority of "deploy.sh" with the following command.

$ chmod +x deploy.sh

Supplement

You can now deploy from CircleCi to an EC2 instance, but as a confirmation, the IP address of CircleCi is added to the security group created at the time of deployment, and the added IP address is deleted after the deployment is completed or when it fails. Please confirm that.

Reference: Story of deploying from CircleCI Deploy from CircleCI to AWS (EC2) using Capistrano

Source code

I will put the source code according to the predecessor.

yml:.circleci/config.yml


version: 2.1

orbs:
  ruby: circleci/[email protected]
  aws-cli: circleci/[email protected]

jobs:
  build:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps

  test:
    parallelism: 3
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          - DB_HOST: 127.0.0.1
          - RAILS_ENV: test
          - BUNDLER_VERSION: 2.1.4
      - image: circleci/mysql:8.0
        command: |
          mysqld --default-authentication-plugin=mysql_native_password
        environment:
          - MYSQL_ROOT_PASSWORD: password
          - MYSQL_USER: root
          - MYSQL_ROOT_HOST: '%'
      - image: selenium/standalone-chrome:latest
        name: chrome

    steps:
      - checkout
      - ruby/install-deps
      - run: mv config/database.yml.ci config/database.yml
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:3306 -timeout 1m
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load
      # Run rspec in parallel
      - ruby/rubocop-check
      - ruby/rspec-test

  deploy:
    docker:
      - image: circleci/ruby:2.7.2-node
        environment:
          BUNDLER_VERSION: 2.1.4
    steps:
      - checkout
      - ruby/install-deps
      - aws-cli/install
      - add_ssh_keys:
          fingerprints: "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      - deploy:
          name: Capistrano deploy
          command: ./deploy.sh

workflows:
  version: 2
  build_accept_deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

deploy.sh


#!/bin/sh
set -ex

IP=`curl -f -s ifconfig.me`

trap "aws ec2 revoke-security-group-ingress --group-id ${SECURITY_GROUP_ID} --protocol tcp --port 22 --cidr ${IP}/32" 0 1 2 3 15
aws ec2 authorize-security-group-ingress --group-id ${SECURITY_GROUP_ID} --protocol tcp --port 22 --cidr ${IP}/32
bundle exec cap production deploy

config/deploy/production.rb


server 'Elastic IP',
       user: 'username',
       roles: %w[web app db]

set :ssh_options, {
  keys: (ENV['PRODUCTION_SSH_KEY']),
  forward_agent: true
}

config/deploy.rb


# config/credentials.yml.Make the contents of enc available
require File.expand_path('./environment', __dir__)
# config valid for current version and patch releases of Capistrano
lock '~> 3.15.0'
#Application name
set :application, 'app_name'
#github url. Specify the git hosting destination for your project
set :repo_url, '[email protected]:user_name/app_name.git'
#The directory of the server to deploy to. Specify with full path
set :deploy_to, '/var/www/rails/app_name'

#Specify Ruby version
set :rbenv_ruby, '2.7.2'

#Specify a symbolic link file, specifically a file that goes into shared
append :linked_files, 'config/master.key'
#Generate a directory of symbolic links
append :linked_dirs, 'log', 'tmp/pids', 'tmp/cache', 'tmp/sockets'
#Required when doing sudo etc. in a task
set :pty, true
#Number of versions to keep(* See below)
set :keep_releases, 3
#The level of the log to output.
set :log_level, :debug

# puma
set :puma_init_active_record, true

#Fixed Nginx config file name and location
set :nginx_sites_enabled_path, '/etc/nginx/conf.d'
set :nginx_config_name, "#{fetch(:application)}.conf"

Gemfile


source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.7.2'

gem 'bcrypt'
gem 'bootsnap', require: false
gem 'bootstrap'
gem 'devise'
gem 'devise-i18n'
gem 'image_processing'
gem 'jbuilder'
gem 'jquery-rails'
gem 'kaminari'
gem 'kaminari-bootstrap'
gem 'mini_magick'
gem 'mysql2'
gem 'puma', '< 5'
gem 'rails', '~> 6.0.3', '>= 6.0.3.4'
gem 'rails-i18n'
gem 'sassc-rails'
gem 'turbolinks'
gem 'uglifier'

group :development, :test do
  gem 'byebug', platforms: %i[mri mingw x64_mingw]
  gem 'factory_bot_rails'
  gem 'rspec-rails'
end

group :development do
  gem 'bcrypt_pbkdf'
  gem 'capistrano'
  gem 'capistrano3-puma', '< 5'
  gem 'capistrano-bundler'
  gem 'capistrano-rails'
  gem 'capistrano-rbenv'
  gem 'capistrano-rbenv-vars'
  gem 'ed25519'
  gem 'listen'
  gem 'rubocop', require: false
  gem 'rubocop-performance', require: false
  gem 'rubocop-rails', require: false
  gem 'rubocop-rspec'
  gem 'spring'
  gem 'spring-watcher-listen'
  gem 'sshkit-sudo'
  gem 'web-console'
end

group :test do
  gem 'capybara'
  gem 'capybara-email'
  gem 'database_cleaner'
  gem 'rspec_junit_formatter'
  gem 'selenium-webdriver'
end

group :production do
  gem 'aws-sdk-s3', require: false
end

Recommended Posts

Deploy to EC2 with CircleCi + Capistrano
Deploy with EC2 / Docker / Laravel
I tried automatic deployment with CircleCI + Capistrano + AWS (EC2) + Rails
Deploy SpringBoot application to AWS EC2
For beginners! Automatic deployment with Rails6 + CircleCI + Capistrano + AWS (EC2)
Connect to Amazon EC2 with SSH (Ubuntu)
Easy deployment with Capistrano + AWS (EC2) + Rails
Deploy to heroku with Docker (Rails 6, MySQL)
Deploy your application with VPC + EC2 + Docker.
How to deploy
Deploy Java web app to Azure with maven
Deploy Rails to ECS Fargate with AWS Copilot
Steps to deploy to Heroku
Introducing CircleCI to Rails
Pass User credential to deployment destination with Jenkins + Capistrano
How to deploy Java to AWS Lambda with Serverless Framework
Run Rubocop and RSpec on CircleCI and deploy to ECS
Steps to deploy Struts 2.5.8
Automatically deploy to WildFly with Jenkins when SVN commits
Deploy RAILS on EC2
How to check CircleCI code and automatically deploy to Heroku
[Part 1] How to deploy Docker containers and static files with CircleCI + ECS + ECR + CloudFront
Deploy Vapor Project to Heroku
Java to play with Function
How to deploy on heroku
Hot deploy with IntelliJ IDEA
How to number (number) with html.erb
How to update with activerecord-import
Connect to DB with Java
Connect to MySQL 8 with Java
What to do if an SSH key authentication error occurs during automatic deployment to EC2 with Capistrano
Deploy your application to WildFly
Connect to oracle with eclipse!