Docker the development environment of Ruby on Rails project

This is the article on the 16th day of ACCESS Advent Calendar 2020.

Overview

There is a project to convert the development environment of the Ruby on Rails project currently in operation to Docker, and the procedure of the migration work performed at that time is shown.

About Docker

Docker is a platform that enables you to build and execute a virtual environment called a container.

This is the first time I've touched Docker, and [Introduction to Docker](PYTHON_T VDDQ) was very helpful in understanding Docker.

Premise

version

• macOS Catalina 10.15.7
• Docker 19.03.13
• docker-compose 1.27.4
• Ruby 2.4.5
• mongoDB 3.0.15
• postgres 10

System configuration

We will make it Docker according to the system configuration diagram here. (* Oita simplified)

First, create a Rails application Docker Container and Then with the orchestration tool docker-compose Connect your Rails application to mongoDB and postgreSQL.

procedure

Docker installation

Introduce Docker Desktop for Mac from Docker Hub.

It is OK if you can execute the following two commands in the terminal.

$ docker -v
Docker version 19.03.13, build 4484c46d9d
$ docker-compose -v
docker-compose version 1.27.4, build 40524192

Creating the required files

Create the files needed to run Docker and docker-compose in the root of your project.

Dockerfile I got a Docker image called ruby: 2.4.5-slim with the ruby ​​2.4.5 environment pre-installed from Docker Hub and set up the Rails environment on that image.

Dockerfile

FROM ruby:2.4.5-slim

#Specify the project root on the Docker container
ENV APP_ROOT=/app
RUN mkdir $APP_ROOT
WORKDIR $APP_ROOT

# apt-Suppress warnings when installing utils
# https://qiita.com/haessal/items/0a83fe9fa1ac00ed5ee9
ENV DEBCONF_NOWARNINGS yes

#Install apt package
RUN apt-get update -y -qq && \
    apt-get install -y -qq build-essential libpq-dev libmagickwand-dev

#Rails setup
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN gem install bundler -v 1.17.3 && bundle install

#Copy the project directory to Docker Image
COPY . $APP_ROOT

docker-compose.yml Define three services, postgres, mongo, web, It makes web dependent on postgres and mongo.

docker-compose.yml

version: "3"

services:
    #definition of postgreSQL container
    postgres: 
        image: postgres:10
        ports:
            # <Host Port>:<Container Port>
            - "5432:5432"
        environment:
            POSTGRES_USER: xxxxxx
            POSTGRES_PASSWORD: xxxxxx
    
    #definition of mongoDB container
    mongo:
        image: mongo:3.0.15
        ports:
           - "27017:27017"
    
    #Rails app container definition
    web: 
        build: . 
        env_file: .env
        #server to avoid pid error.Run rails s after removing pid
        # https://qiita.com/sakuraniumarete/items/ac07d9d56c876601748c
        command: /bin/sh -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
        #Dependency definition(When you build the web, postgres and mongo are built at the same time)
        depends_on:
            - postgres
            - mongo

Build execution

When you create these two files,

$ docker-compose build

You will be able to build the container with.

DB persistence

In the current state, the DB is generated in the storage inside the container, If you delete the container and rebuild it, all the data saved in the DB will be lost.

To persist the data on the DB, use the function called volume provided by Docker. volume is a data storage area that is generated independently of the Docker Container life cycle. By creating a DB on volume, the data on the DB will continue to remain even if the container is rebuilt.

Source: https://matsuand.github.io/docs.docker.jp.onthefly/storage/volumes/

To use volume, add the following contents to docker-compose.yml.

docker-compose.yml

 services:
    postgres: 
+        volumes:
+            - "postgres-data:/var/lib/postgresql/data"
    mongo:
+        volumes:
+            - "mongo-data:/data/db"
+ volumes:
+    postgres-data:
+    mongo-data:

If you add the above and build again, Docker Volume should be created.

$ docker volume ls
local               mongo-data
local               postgres-data

Synchronize host and container source code

In the current state, when you build the Dockerfile, all the data on the host is copied to the Image.

Dockerfile

COPY . $APP_ROOT

In other words, if you change the source code on the host side after this, stop the running container once and You need to start over from docker-compose build.

It is very inefficient to start over from the build every time in the development environment, so Allows host code changes to be immediately reflected in the container.

A common method is to mount the project root directory as an anonymous volume on the container, as shown below.

docker-compose.yml

services:
    # ${APP_ROOT}Is an environment variable defined in ENV in the Dockerfile
    web:.:${APP_ROOT}

However, the above method has the problem that the slow Volume I/O peculiar to Docker for Mac affects the performance. (This issue is discussed in the docker/for-mac GitHub repository](https://github.com/docker/for-mac/issues/77).) The effect on RSpec is particularly noticeable in the environment at hand, and when mounted by the above method, ** the test that normally completed in about 5 minutes took about 30 minutes ** ...

docker-sync

A third-party library called docker-sync is available as a solution to this problem.

Create new docker-sync.yml and docker-compose-dev.yml. I referred to docker-sync documentation when creating it.

docker-sync.yml

version: "2"

syncs:
  sync-volume:
    src: "."
    sync_excludes:
      - "log"
      - "tmp"
      - ".git"

docker-compose-dev.yml

version: "3"

services:
  web:
    volumes:
      - "sync-volume:/app:nocopy"

volumes:
  sync-volume:
    external: true

DB setup

Set up postgreSQL and mongoDB with the following command.

$ docker-compose run --rm -e RAILS_ENV=development -T web rake db:setup
$ docker-compose run --rm -e RAILS_ENV=devlopment -T web rake db:mongoid:create_indexes

docker-compose run --rm <container name> <command> is This command starts the specified container service, executes an arbitrary command, and then deletes the container. Since the DB is persisted in volume, it's okay to create a container just for setup and then delete it.

Run Rails server

After performing the steps up to this point, execute the following command to start the Rails server.

$ docker-sync-stack start

This is a shortened version of the following command.

$ docker-sync start
$ docker-compose -f docker-compose.yml -f docker-compose.yml up

If you specify multiple docker-compose files with the -f option, You can overwrite various parameters when creating a container. Reference: Adding and overwriting settings --Docker-docs-ja

Also, the above is the command to start in the foreground, To start it in the background, execute the following command.

#Start-up
$ docker-sync start
$ docker-compose -f docker-compose.yml -f docker-compose.yml up
#Stop
$ docker-compose down
$ docker-sync stop

Test run

To run RSpec, run the following command with the server running.

$ docker-compose exec -e COVERAGE=true -T web bundle exec rspec

You can run any command on the running Docker container with docker-compose exec.

Alternatively, you can run it in a container as shown below.

$ docker-compose exec web bash
root@container:/app# bundle exec rspec

CI compatible

I am using Jenkins as a CI tool.

Shell script to run the test

In the build job, the automatic test is performed by executing the following shell.

#Processing at the end
# docker-Post-process so that no garbage remains on the Jenkins build machine even if compose fails
# https://qiita.com/ryo0301/items/7bf1eaf00b037c38e2ea
function finally {
	# Clean project
 	docker-compose down --rmi local --volumes --remove-orphans
}
trap finally EXIT

#Jenkins environment variable as the project name for parallel execution$BUILD_Specify TAG
export COMPOSE_PROJECT_NAME=$BUILD_TAG

#Pre-built docker with environment variables-By specifying the compose file,
# -The specification by the f option can be omitted.
# https://docs.docker.jp/compose/reference/envvars.html
export COMPOSE_PATH_SEPARATOR=:
export COMPOSE_FILE=docker-compose.yml:docker-compose-test.yml

# Build Container
docker-compose build --no-cache
docker-compose up -d

# Setup DB
docker-compose exec -e RAILS_ENV=test -T web rake db:setup
docker-compose exec -e RAILS_ENV=test -T web rake db:mongoid:create_indexes

# Run RSpec
docker-compose exec  -e COVERAGE=true -T web bundle exec rspec

Allow build jobs to run in parallel

As a problem when Rails project is not Dockerized When two or more build jobs are executed in parallel, DB conflicts occur on the same machine, and There was a problem that caused an error.

By making it Docker, each build will be executed in an independent container, The error will not occur even if it is executed in parallel. However, to avoid duplicate container port numbers when running parallel builds You need to change the port forwarding settings. Reference: Assign container port on host-Docker-docs-ja

export COMPOSE_FILE=docker-compose.yml:docker-compose-test.yml

I'm doing that with the contents of the docker-compose-test.yml specified in.

docker-compose-test.yml

version: "3"

services:
    postgres:
        ports:
            - "5432"
    mongo:
        ports:
            - "27017"
    web:
        ports:
            - "3000"

You also need to move the port number listed in docker-compose.yml to docker-compose-dev.yml.

docker-compose.yml

services:
    postgres:
-        ports:
-            - "5432:5432"
    mongo:
-        ports:
-            - "27017:27017"
    web:
-        ports:
-            - "3000:3000"

docker-compose-dev.yml

services:
    postgres:
+        ports:
+            - "5432:5432"
    mongo:
+        ports:
+            - "27017:27017"
    web:
+        ports:
+            - "3000:3000"

The reason is that if you run docker-compose up -d as it is, docker-compose.yml and docker-compose-test.yml will be merged. As a result, the port specification becomes as follows, and the meaning of specifying the port in docker-compose-test.yml is lost.

services:
    postgres:
        ports:
            - "5432:5432"
            - "5432"
    mongo:
        ports:
            - "27017:27017"
            - "27017"
    web:
        ports:
            - "3000:3000"
            - "3000"

In the Run Rails Server section,

If you specify multiple docker-compose files with the -f option, Various parameters can be overridden when creating a container.

However, please note that in the case of parameters that can be specified multiple times, the setting values ​​will not be overwritten and will be ** merged **.

Bonus: Support for RubyMine

JetBrains IDE RubyMine is fully compatible with the Ruby on Rails development environment on Docker Container and can be set by following the steps below. Tutorial: Docker Compose as a remote interpreter — RubyMine

Summary

Newly created file this time

The newly created files for converting the development environment to Docker are as follows.

.
├── Dockerfile
├── docker-compose.yml
├── docker-compose-dev.yml
├── docker-compose-test.yml
└── docker-sync.yml

Other

This is the first time I have come into contact with container technology, and I have repeated various trials and errors in converting it to Docker. There may be better ways to deal with it, or there may be some incorrect ways to deal with it, but I would appreciate it if you could point out that.