[DOCKER] [CircleCI] I will explain the stupid configuration file (config.yml) that I wrote for the first time.

Guchoku: Being honest and unable to act flexibly. Also, that way. Stupid honesty. -goo Japanese dictionary

Introduction

Hello, this is Masuyama. I will be in charge of the 20th day of CircleCI Advent Calendar 2020.

This time, I tried to write an article explaining the configuration file config.yml used in the web application I made as a personal development. Day 16's article and content were all round.

So, this time, I would like to explain the ** configuration file that I wrote for the first time when using CircleCI.

First of all, the code does not have a ** stylish look ** or ** ingenuity that emphasizes actual operation **. By deliberately using the theme of ** the configuration file I wrote for the first time , The purpose is to make people who have never used CircleCI think " You may understand this **".

Trigger

When using any tool, even if it seems convenient, preparation in advance is complicated If you don't understand what you're doing, why don't you just avoid it? Also, even if there is an explanation, wouldn't it be "no more" just because it seems complicated at first glance?

For example, when you try to write code for some purpose, you may search Google to find code that may be helpful. At that time ** the amount of code is reduced to the limit and it is cleaner than the code ** ** Code that can understand the intention even with stupid code ** is easier to take as a first step.

There are various ways to write the configuration file required to use CircleCI. The code should be neat and convenient as a result, though it should be devised. By having them touch from how to write a configuration file that even beginners can intuitively understand I hope it will be an opportunity to take the first step.

Commentary part

System configuration (CI/CD pipeline)

The system configuration and CI/CD pipeline this time are like this.

image.png

The overall flow is

  1. Push the modified code to GitHub and
  2. CircleCI detects that it has been pushed to the specified repository and
  3. Build and test on the machine launched on CircleCI,
  4. If the test passes, log in to EC2 with SSH from another machine and
  5. Reflect the code on GitHub to EC2 (production environment) and
  6. Reboot the container to reflect the modifications in the production environment

It is a flow.

The first stupid config file I wrote

It's finally the main subject. First, I will introduce the whole picture of the configuration file.

.circleci/config.yml


#CircleCI version specification
version: 2

jobs:
  build:
    machine:
      image: circleci/classic:edge
    steps:
    #Run Django tests
      - checkout
      - run:
          name: make env file
          command: |
            echo DEBUG=${DEBUG} > src/.env
            echo SECRET_KEY=${SECRET_KEY} >> src/.env
            echo APIKEY=${APIKEY} >> src/.env
            echo MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} >> src/.env
            echo MYSQL_DATABASE=${MYSQL_DATABASE} >> src/.env
            echo MYSQL_USER=${MYSQL_USER} >> src/.env
            echo MYSQL_PASSWORD=${MYSQL_PASSWORD} >> src/.env
            echo MYSQL_HOST=${MYSQL_HOST} >> src/.env
      - run:
          name: docker-compose up
          command: docker-compose up -d
      - run:
          name: docker ps
          command: docker ps
      - run:
          name: collectstatic
          command: docker-compose run python ./manage.py collectstatic
      - run:
          name: migrate
          command: docker-compose run python ./manage.py migrate
      - run:
          name: test
          command: docker-compose run python ./manage.py test
      - run:
          name: docker-compose down
          command: docker-compose down

#SSH to EC2 and deploy
  deploy:
    machine:
      image: circleci/classic:edge
    steps:
      - checkout
      #Call the private key registered in CircleCI.
      - add_ssh_keys
      - run: ssh ${USER_NAME}@${HOST_NAME} -p ${SSH_PORT} 'cd xxxxx && git pull origin main && docker-compose restart && docker-compose run python ./manage.py migrate'

#Only run deploy if the test is successful.
workflows:
  version: 2
  build_and_deploy:
    jobs:
      - build
      - deploy:
          requires:
            - build
          #Deploy only when pushed to the main branch.
          filters:
            branches:
              only: main

I will explain some excerpts of this configuration file.

version

version: 2

The key to use depends on the version you specify, but at that time I was using version "2".

jobs - build - machine

jobs:
  build:
    machine:
      image: circleci/classic:edge

** image: ** specifies the official machine image to use for the build. The following is implemented in the machine image specified here.

--Ubuntu version 14.04 image --Docker version 17.10.0-ce --docker-compose version 1.16.1

I'm grateful that I don't have to build from clean Linux.

→ Official document: Set CircleCI

jobs - build - steps

After build, we will go from building to testing the Docker container at ** steps **.

What I want to tell you here is that ** just write the command you want to execute **. Normally, a human hand hits a command to build a container and even executes a test command. The merit is that all such work is automated. I love this kind of automation.

This part will be the longest, so I will explain it in the comment out. It includes commands because it uses a Python framework called Django, but you don't have to worry too much about it.

    steps:
      - checkout
      - run:
          # .Environment variables set in CircleCI for the design of reading secure information from the env file.The process of writing to an env file
          #There seems to be other good ways, but I wrote each one honestly
          name: make env file
          command: |
            echo DEBUG=${DEBUG} > src/.env
            echo SECRET_KEY=${SECRET_KEY} >> src/.env
            echo APIKEY=${APIKEY} >> src/.env
            echo MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} >> src/.env
            echo MYSQL_DATABASE=${MYSQL_DATABASE} >> src/.env
            echo MYSQL_USER=${MYSQL_USER} >> src/.env
            echo MYSQL_PASSWORD=${MYSQL_PASSWORD} >> src/.env
            echo MYSQL_HOST=${MYSQL_HOST} >> src/.env
      - run:
          #Launch the container required for testing
          name: docker-compose up
          command: docker-compose up -d
      - run:
          #When troubleshooting, display the container list so that you can check the container startup status on the CircleCI UI (hobby)
          name: docker ps
          command: docker ps
      - run:
          #What you need to do with Django
          name: collectstatic
          command: docker-compose run python ./manage.py collectstatic
      - run:
          #What you need to do with Django
          name: migrate
          command: docker-compose run python ./manage.py migrate
      - run:
          #Django automated testing
          #If it fails here, it will not proceed to the next process, so the problematic code will not be reflected in the production environment.
          name: test
          command: docker-compose run python ./manage.py test
      - run:
          #Stop container(Maybe I don't think about it now?)
          name: docker-compose down
          command: docker-compose down

It is convenient to write ** name: ** so that you can see the successful and unsuccessful jobs at a glance when checking each job on the CircleCI UI. It is obvious because it is displayed like this on the UI. It feels good to be filled with green marks. image.png

deploy - machine

When the test is successful with the build in the previous section, the deploy job will run for the first time. On the CircleCI UI, the build job and the deploy job are displayed in this way (although the screenshot succeeds in deploying). image.png

In this environment, I'm running a container on AWS EC2, so I'm aiming to SSH into EC2 and hit git pull.

To do this, set up a machine for SSH login to EC2. I'm using the same machine image as when building, but it's okay if it's Linux that can even SSH. I could have used a lighter machine image.

  deploy:
    machine:
      image: circleci/classic:edge

deploy --steps --add_ssh_keys (register private key)

To log in to EC2, use the private key you created in advance. Normally, you would store the private key in a specific folder and call it when you hit the SSH command. Therefore, please note that it is handled a little differently from environment variables.

    steps:
      - checkout
      #Call the private key registered with CircleCI
      - add_ssh_keys

It will be registered in advance by the operation on the UI, but it can be called with the command ** add_ssh_keys **.

All CircleCI jobs use ssh-agent to automatically sign all registered SSH keys. However, you must use the add_ssh_keys key to actually register the keys in the container.

deploy - steps - SSH & git pull

Thank you for waiting. At this point, I logged in to ECS with SSH, git pulled it, launched the container, and performed the tasks required by the application.

      - run: ssh ${USER_NAME}@${HOST_NAME} -p ${SSH_PORT} &&
        'cd xxxxx &&
         git pull origin main &&
         docker-compose restart &&
         docker-compose run python ./manage.py migrate'

Of course, I also need the user name, host name, and SSH port information. It's more secure to pre-register environment variables with CircleCI for this information, so don't write it solid. (If you write it directly in config.yml, the entire repository will be published.)

Finally

What did you think. It's not stylish, it's because it describes each process honestly I think it was easy to get an image of the process you are trying to do.

This time I introduced the deployment part, but it is convenient enough just to automate the build & test. If you have a test where you usually type commands manually, why not try CircleCI?

In addition to Django, I also wrote an article that explains automatic build & test with Ruby on Rails from environment construction, so please have a look. → How to automatically build and test (RSpec) Ruby on Rails application in Docker environment with CircleCI

Recommended Posts

[CircleCI] I will explain the stupid configuration file (config.yml) that I wrote for the first time.
I will explain the nesting of for statements that kill beginners
I tried using Docker for the first time
I tried touching Docker for the first time
[Rails] I tried using the button_to method for the first time
Spring Boot for the first time
Spring AOP for the first time
Glassfish tuning list that I want to keep for the time being
[First Java] Make something that works with Intellij for the time being
Learning for the first time java [Introduction]
I will absolutely convert the time string!
Ideal and reality that I felt when I used Optional for the first time ~ Implementation of cache using Map ~
Walls hit by Rspec for the first time
Android Studio development for the first time (for beginners)
I wrote a test code (Junit & mockit) for the code that calls the AWS API (Java)
CircleCI configuration file that has been replaced by the operation of my own library
Learn for the first time java # 3 expressions and operators
Oreore certificate https (2020/12/19) for the first time with nginx
Learning memo when learning Java for the first time (personal learning memo)
How to study kotlin for the first time ~ Part 2 ~
How to study kotlin for the first time ~ Part 1 ~