[CircleCI] I was addicted to the automatic test of CircleCI (rails + mysql) [Memo]

Introduction

Note that I was addicted to trying to do automated testing of Rails apps using CircleCI. Especially the connection with the mysql container didn't work. To keep the contents of various setting files and countermeasures against errors as a memorandum.

environment

CircleCI  2.1
Bundler  2.1.4
Rails 6.0.3.4
ruby 2.6.5
MySQL 5.7

The development environment is built with Docker-compose. The relationship between the service name and the container name is as follows.

setting file

The app name is webapp.

database.yml and db.env

Database configuration file. Since this setting could not be used well on the CircleCI side, I decided to create a database setting file for CircleCI this time by referring to the following article. Therefore, the settings here have nothing to do with CircleCI. [CircleCI 2.0 settings memo](https://qiita.com/toyoken/items/16f3b6df06bbe393e644#mysql%E3%81%A8%E3%81%AE%E7%96%8E%E9%80%9A%E8% A8% AD% E5% AE% 9A% E3% 81% AE% E5% 89% 8D% E3% 81% AB "CircleCI 2.0 Settings Memo")

config/database.yml

default: &default
  adapter: mysql2
  encoding: utf8
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: <%= ENV.fetch('MYSQL_USER') { 'root' } %>
  password: <%= ENV.fetch('MYSQL_PASSWORD') { 'rootpass' } %>
  port: 3306
  host: db
  
development:
  <<: *default
  database: webapp_development
    
test:
  <<: *default
  database: webapp_test

environments/db.env

MYSQL_ROOT_USER=root
MYSQL_ROOT_PASSWORD=rootpass
MYSQL_USER=kagamiya
MYSQL_PASSWORD=kagamiya

You may not need to set the port. host: db is the service name for Docker-compose. I don't think you need {'root'} from ENV.fetch.

database.yml.ci Database configuration file for CI. Use this for automated testing on CircleCI.

yaml:config/database.yml.ci

test:
  adapter: mysql2
  encoding: utf8
  pool: 5
  username: 'root'
  port: 3306
  host: '127.0.0.1'
  database: webapp_test

config.yml The essential CircleCI configuration file. Click here for reference. Automate testing with CircleCI

.circleci/config.yml

version: 2.1 #Version specification

executors:  
  default:  
    working_directory: ~/webapp  
    docker:  
      - image: circleci/ruby:2.6.5-node
        environment:  
          BUNDLER_VERSION: 2.1.4
          RAILS_ENV: test
          DB_HOST: 127.0.0.1
          DB_USERNAME: 'root'
          DB_PASSWORD: ''
      - image: circleci/mysql:5.7 
        environment:
          MYSQL_ROOT_HOST: '%'
          MYSQL_ALLOW_EMPTY_PASSWORD: 'true'

commands:  
  setup:  
    steps:  
      - checkout  
      - run:  
          name: Update bundler #Update bundler to version 2
          command: gem update bundler  

      - run:  
          name: Which bundler? #Version confirmation
          command: bundle -v  

      - restore_cache: #Read cache
          keys:  
            - gem-cache-v1-{{ checksum "Gemfile.lock" }}  
            - gem-cache-v1-  

      - run:  
          name: Bundle Install  
          command: bundle check --path vendor/bundle || bundle install --deployment  

      - save_cache: #Save cache
          key: gem-cache-v1-{{ checksum "Gemfile.lock" }}  
          paths:  
            - vendor/bundle  
      
      #The following is required when using Webpacker with Rails 6
      - restore_cache:  
          keys:  
            - yarn-cache-v1-{{ checksum "yarn.lock" }}  
            - yarn-cache-v1-  

      - run:  
          name: Yarn Install  
          command: yarn install --cache-folder ~/.cache/yarn  

      - save_cache:  
          key: yarn-cache-v1-{{ checksum "yarn.lock" }}  
          paths:  
            - ~/.cache/yarn  

jobs:  
  test:  
    executor: default  
    environment:  
      RAILS_ENV: test  
    steps:  
      - checkout  
      - setup  
      - run:  
          name: Wait for DB  
          command: dockerize -wait tcp://127.0.0.1:3306 -timeout 90s  
      
      - run:
          name: Use specific database.yml #Use database settings for CircleCI
          command: mv config/database.yml.ci config/database.yml

      - run:  
          name: Database setup  
          command: |
            bin/rails db:create
            bin/rails db:schema:load --trace  

      - run: #Run regular tests and system tests
          name: Rails Test  
          command: |  
            bin/rails test  
            bin/rails test:system  

workflows:  
  build_and_test:  
    jobs:  
      - test  

(When using MySQL 8 series, it seems that it is necessary to add a description, so I decided to use 5.7. Also, the ruby image could not use the yarn command without a node. )

I was particularly addicted to the settings around the database. Because I didn't really understand the MySQL specs (or even now) ... Initially, following the reference article,

python

DB_USER: 'kagamiya'
DB_PASSWORD: 'kagamiya'

MYSQL_USER: 'kagamiya'
MYSQL_PASSWORD: 'kagamiya'

I was saying. However, a user named kagamiya is not registered in CircleCI's MySQL image in the first place. (Of course, it is in the db container of the development environment) Therefore, root is inevitably used, so I did the following.

python

DB_USER: 'root'
DB_PASSWORD: 'rootpass'

MYSQL_USER: 'root'
MYSQL_PASSWORD: 'rootpass'

But it didn't work. That's because the root user's default password isn't set (probably). That's why emptying the password doesn't work.

python

DB_USER: 'root'
DB_PASSWORD: ''

MYSQL_USER: 'root'
MYSQL_PASSWORD: ''

So it seems that the user must connect as root and the password as MYSQL_ALLOW_EMPTY_PASSWORD:'true' (allow empty?).

python

      - image: circleci/ruby:2.6.5-node
        environment:  
          BUNDLER_VERSION: 2.1.4
          RAILS_ENV: test
          DB_HOST: 127.0.0.1
          DB_USERNAME: 'root'
          DB_PASSWORD: ''
      - image: circleci/mysql:5.7 
        environment:
          MYSQL_ALLOW_EMPTY_PASSWORD: 'true'

I threw an error again here.

Access denied for user 'root'@'127.0.0.1'

I had a hard time understanding here, but it seems that there is no user'root'@ '12 7.0.0.1'. Looking at the mysql user list, I see'root' @'%' and'root' @'localhost', but not'root' @ '12 7.0.0.1'. Whether to use localhost or 127.0.0.1 for the mysql hostname (DB_HOST) seems to be another issue. However, since connecting is the highest priority now, we will specify% as the root host. (% Means "all hosts", but I'm not sure)

python

MYSQL_ROOT_HOST: '%'
MYSQL_ALLOW_EMPTY_PASSWORD: 'true'

Whatever the reason, I was finally able to connect.

Next, the CircleCI image uses its own database settings, so overwrite database.yml with database.yml.ci. Add processing before setting up the database.

python

      - run:
          name: Use specific database.yml #Use database settings for CircleCI
          command: mv config/database.yml.ci config/database.yml

In my environment, I was angry that the specified database (webapp_test) did not exist, so I created the database.

python

      - run:  
          name: Database setup  
          command: |
            bin/rails db:create
            bin/rails db:schema:load --trace  

(This may cause an error in the next execution ...?)

Summary

I don't fully understand the MySQL specifications and the contents of config. Many people have uploaded config samples, but it seems that there are few that explain in detail. Therefore, if an error occurs in a part that you do not understand, there is a high possibility that you will be addicted to it again ...