I tried to build an API server with Go (Echo) x MySQL x Docker x Clean Architecture

Introduction

As the title says, I made an API server with Go x MySQL x Docker. in the beginning,

・ Create a simple REST API with Go ・ Set up a development environment with Docker

So, this article is recommended for those who want to write API with Go, set up a development environment with Docker, and want to adopt a clean architecture at that time. On the contrary, it may not be very helpful for those who want to study a solid clean architecture. ..

In addition, it should be noted ** Echo ** in the framework (https://echo.labstack.com/) ** Gorm ** (https://github.com/go-gorm/gorm) to access the DB using.

Click here for repository ↓

Preparation

Development environment

I built a development environment with Docker.

Dockerfile Dockerfile Official Reference is a good reference for how to write a basic Dockerfile.

Now, let's take a quick look at them in order.

First is Go.

docker/api/Dockerfile


FROM golang:1.14

#use go module
ENV GO111MODULE=on 

#Specify the directory to run the application
WORKDIR /go/src/github.com/Le0tk0k/go-rest-api

#Go to the above directory.mod and go.copy sum
COPY go.mod go.sum ./
#Cache can be used if the above file is unchanged
RUN go mod download

COPY . .
RUN go build .

RUN go get github.com/pilu/fresh

EXPOSE 8080

#Start the server with the fresh command
CMD ["fresh"]

You can now hot reload on github.com/pilu/fresh. Other than that, there is nothing special about it.

reference -Using go mod download to speed up Golang Docker buildsGo v1.11 + Docker + fresh to create a hot reload development environment and enjoy Go language life

Next is MySQL.

docker/mysql/Dockerfile


FROM mysql

EXPOSE 3306

#Copy the MySQL configuration file into the image
COPY ./docker/mysql/my.cnf /etc/mysql/conf.d/my.cnf

CMD ["mysqld"]

The MySQL configuration file looks like this: Since the authentication method at the time of connection has changed in MySQL 8.0 or later, it seems that the description on the second line is necessary. Also, the character code is changed to prevent garbled characters.

docker/mysql/my.cnf


[mysqld]
default_authentication_plugin=mysql_native_password
character-set-server=utf8mb4

[client]
default-character-set=utf8mb4

If you put the initialization SQL and script in the directory /docker-entrypoint-initdb.d/, the data can be initialized automatically when you first start the image.

So, let's create a table.

docker/mysql/db/init.sql


CREATE DATABASE IF NOT EXISTS go_rest_api;
USE go_rest_api;

CREATE TABLE IF NOT EXISTS users (
  id          INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
  name        VARCHAR(256) NOT NULL,
  age         INT NOT NULL,
  created_at  TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  updated_at  TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

reference -Create DB for development environment with Docker + MySQL

docker-compose.yml You can also refer to Compose File / Official Reference.

docker-compose.yml


version: '3'
services: 
  db:
    build: 
      #Specify the path of the directory where the Dockerfile is located
      context: .
      dockerfile: ./docker/mysql/Dockerfile
    ports: 
      #Specify public port
      - "3306:3306"
    volumes: 
      #Initialize the data at startup
      - ./docker/mysql/db:/docker-entrypoint-initdb.d
      #mysql persistence
      - ./docker/mysql/db/mysql_data:/var/lib/mysql
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: go_rest_api
      MYSQL_USER: user
      MYSQL_PASSWORD: password

  api:
    build:
      context: .
      dockerfile: ./docker/api/Dockerfile
    volumes: 
      - ./:/go/src/github.com/Le0tk0k/go-rest-api
    ports: 
      - "8080:8080"
    depends_on:
      #db starts first
      - db

Since mysql is persisted, the previous data will remain even if the container is restarted.

Also, by specifying depend_on for api, db will start first, and then api will start. However, depend_on only specifies the order of startup and does not wait until it is completed, so another measure is required. The countermeasures are introduced in the implementation section.

reference

-docker-compose + MySQL5.7 (8.0) + Initialization + Persistence

Clean architecture

Clean Architecture is the architecture proposed in The Clean Architecture.

Am I an architecture in the first place? What that? It was in a state, but after reading the following article, I think I understood about 1mm of supernatant. There are many other articles, so please check them out!

Reference (There are also articles other than clean architecture)

-I read a book on clean architecture, so I implemented an API server -Let's understand "Layered Architecture + DDD" now. (Golang) -Try building API Server with Clean Architecture -[Golang + Layered Architecture] Try to implement Web API with DDD in mind

Implementation

Directory structure

The directory structure is as follows.


├── docker
│   ├── api
│   │   └── Dockerfile
│   └── mysql
│       ├── Dockerfile
│       ├── db
│       │   ├── init.sql
│       │   └── mysql_data
│       └── my.cnf
├── docker-compose.yml
├── domain
│   └── user.go
├── infrastructure
│   ├── router.go
│   └── sqlhandler.go
├── interfaces
│   ├── controllers
│   │   ├── context.go
│   │   ├── error.go
│   │   └── user_controller.go
│   └── database
│       ├── sql_handler.go
│       └── user_repository.go
|── usecase
│   ├── user_interactor.go
│   └── user_repository.go
├── server.go
├── go.mod
├── go.sum

domain → Entities layer infrastructure → Frameworks & Drivers layer interfaces → Interface layer usecase → Use cases layer

It corresponds to.

Domain (Entities layer)

The innermost layer, which is independent of anything. We will define the User model.

For json:"-", if you write-, it will not be output.

domain/user.go


package domain

import "time"

type User struct {
	ID        int       `gorm:"primary_key" json:"id"`
	Name      string    `json:"name"`
	Age       int       `json:"age"`
	CreatedAt time.Time `json:"-"`
	UpdatedAt time.Time `json:"-"`
}

type Users []User

Interfaces / database, Infrastructure (Interface layer, Frameworks & Driver layer)

Next, we will implement the database peripherals. Since the database connection is an external connection, define it in the outermost Infrastructure layer.

infrastructure/sqlhandler.go


package infrastructure

import (
	"fmt"
	"time"

	"github.com/Le0tk0k/go-rest-api/interfaces/database"
	"github.com/jinzhu/gorm"
	_ "github.com/jinzhu/gorm/dialects/mysql"
)

type SqlHandler struct {
	Conn *gorm.DB
}

func NewMySqlDb() database.SqlHandler {

	connectionString := fmt.Sprintf(
		"%s:%s@tcp(%s:%s)/%s?charset=utf8mb4&parseTime=true&loc=Local",
		"user",
		"password",
		"db",
		"3306",
		"go_rest_api",
	)

	conn, err := open(connectionString, 30)
	if err != nil {
		panic(err)
	}

    //Check if you can connect
	err = conn.DB().Ping()
	if err != nil {
		panic(err)
	}

    //Output log details
	conn.LogMode(true)
    //Set DB engine
	conn.Set("gorm:table_options", "ENGINE=InnoDB")

	sqlHandler := new(SqlHandler)
	sqlHandler.Conn = conn

	return sqlHandler
}

//Make sure that the api container starts after confirming the startup of MySQL
func open(path string, count uint) (*gorm.DB, error) {
	db, err := gorm.Open("mysql", path)
	if err != nil {
		if count == 0 {
			return nil, fmt.Errorf("Retry count over")
		}
		time.Sleep(time.Second)
		count--
		return open(path, count)
	}
	return db, nil
}

func (handler *SqlHandler) Find(out interface{}, where ...interface{}) *gorm.DB {
	return handler.Conn.Find(out, where...)
}

func (handler *SqlHandler) Create(value interface{}) *gorm.DB {
	return handler.Conn.Create(value)
}

func (handler *SqlHandler) Save(value interface{}) *gorm.DB {
	return handler.Conn.Save(value)
}

func (handler *SqlHandler) Delete(value interface{}) *gorm.DB {
	return handler.Conn.Delete(value)
}

In opan (), the api container is set to start after waiting for the db container to start. This time, I implemented this feature in Go code, but is it better to implement it in a shell script? I don't know the best practice ... reference -How to wait until MySQL starts with docker-compose up (2 types introduced)

In addition, the infrastructure layer is only a db connection, and the actual processing is implemented in the interfaces / database layer.

interfaces/database/user_repository.go


package database

import (
	"github.com/Le0tk0k/go-rest-api/domain"
)

type UserRepository struct {
	SqlHandler
}

func (userRepository *UserRepository) FindByID(id int) (user domain.User, err error) {
	if err = userRepository.Find(&user, id).Error; err != nil {
		return
	}
	return
}

func (userRepository *UserRepository) Store(u domain.User) (user domain.User, err error) {
	if err = userRepository.Create(&u).Error; err != nil {
		return
	}
	user = u
	return
}

func (userRepository *UserRepository) Update(u domain.User) (user domain.User, err error) {
	if err = userRepository.Save(&u).Error; err != nil {
		return
	}
	user = u
	return
}

func (userRepository *UserRepository) DeleteByID(user domain.User) (err error) {
	if err = userRepository.Delete(&user).Error; err != nil {
		return
	}
	return
}

func (userRepository *UserRepository) FindAll() (users domain.Users, err error) {
	if err = userRepository.Find(&users).Error; err != nil {
		return
	}
	return
}

I am embedding a SqlHandler in the UserRepository. I'm importing the domain layer, which is the innermost layer, so the dependencies are preserved.

However, since SqlHandler is defined in the outermost Infrastructure layer, the dependencies are directed from the inside to the outside, and the rules may not be followed. However, the SqlHandler called here is not a structure defined in the Infrastructure layer, but an interface defined in the interface / database layer, which is the same layer. This is called ** DIP (Dependency Reversal Principle) **.

What this means is that in interfaces / database / user_repository.go, if you call the process defined in the Infrastructure layer, the dependency will change from inside to outside and you will not be able to protect the dependency. In order to avoid that, let's define the interaction with DB in the same layer with interface and call it. With this, although the processing is actually performed in the Infrastructure layer, the dependency is protected because the interfaces / database / user_repository.go calls the SqlHandler interface.

And in Go, when ** type T has all the methods defined in interface I, it means that interface I is implemented **, so the SqlHandler interface of interfaces / databases / sql_hander in the Infrastructure.SqlHandler structure. Defines the method defined in.

interfaces/database/sql_handler.go


package database

import "github.com/jinzhu/gorm"

type SqlHandler interface {
	Find(interface{}, ...interface{}) *gorm.DB
	Create(interface{}) *gorm.DB
	Save(interface{}) *gorm.DB
	Delete(interface{}) *gorm.DB
}

Usecase (Use Case layer)

The Usecase layer is responsible for receiving information from the interfaces / database layer and passing it to the interfaces / controller layer.

usecase/user_interactor.go


package usecase

import "github.com/Le0tk0k/go-rest-api/domain"

type UserInteractor struct {
	UserRepository UserRepository
}

func (interactor *UserInteractor) UserById(id int) (user domain.User, err error) {
	user, err = interactor.UserRepository.FindByID(id)
	return
}

func (interactor *UserInteractor) Users() (users domain.Users, err error) {
	users, err = interactor.UserRepository.FindAll()
	return
}

func (interactor *UserInteractor) Add(u domain.User) (user domain.User, err error) {
	user, err = interactor.UserRepository.Store(u)
	return
}

func (interactor *UserInteractor) Update(u domain.User) (user domain.User, err error) {
	user, err = interactor.UserRepository.Update(u)
	return
}

func (interactor *UserInteractor) DeleteById(user domain.User) (err error) {
	err = interactor.UserRepository.DeleteByID(user)
	return
}

UserInteractor calls UserRepository, but if this is also called from the Interfase layer, it will depend on the inside → outside and it will not be possible to protect the dependency, so create user_repository.go in the same layer and implement the interface * Use * DIP (Principle of Dependency Reversal) ** to protect dependencies.

usecase/user_repository.go


package usecase

import "github.com/Le0tk0k/go-rest-api/domain"

type UserRepository interface {
	FindByID(id int) (domain.User, error)
	Store(domain.User) (domain.User, error)
	Update(domain.User) (domain.User, error)
	DeleteByID(domain.User) error
	FindAll() (domain.Users, error)
}

Interfaces / controllers, Infrastructure (Interface layer, Frameworks & Driver layer)

Then we will implement the controller and router.

interfaces/controllers/user_controller.go


package controllers

import (
	"strconv"

	"github.com/Le0tk0k/go-rest-api/domain"
	"github.com/Le0tk0k/go-rest-api/interfaces/database"
	"github.com/Le0tk0k/go-rest-api/usecase"
)

type UserController struct {
	Interactor usecase.UserInteractor
}

func NewUserController(sqlHandler database.SqlHandler) *UserController {
	return &UserController{
		Interactor: usecase.UserInteractor{
			UserRepository: &database.UserRepository{
				SqlHandler: sqlHandler,
			},
		},
	}
}

func (controller *UserController) CreateUser(c Context) (err error) {
	u := domain.User{}
	c.Bind(&u)
	user, err := controller.Interactor.Add(u)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(201, user)
	return
}

func (controller *UserController) GetUsers(c Context) (err error) {
	users, err := controller.Interactor.Users()

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, users)
	return
}

func (controller *UserController) GetUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	user, err := controller.Interactor.UserById(id)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, user)
	return
}

func (controller *UserController) UpdateUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	u := domain.User{ID: id}
	c.Bind(&u)

	user, err := controller.Interactor.Update(u)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(201, user)
	return
}

func (controller *UserController) DeleteUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	user := domain.User{ID: id}

	err = controller.Interactor.DeleteById(user)
	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, user)
	return
}

Since echo uses echo.Context, define the interface of the method used this time.

interfaces/controllers/context.go


package controllers

type Context interface {
	Param(string) string
	Bind(interface{}) error
	JSON(int, interface{}) error
}

interfaces/controllers/error.go


package controllers

type Error struct {
	Message string
}

func NewError(err error) *Error {
	return &Error{
		Message: err.Error(),
	}
}

Since routing uses echo (using an external package), we will implement it in the Infrastructure layer. Note that the echo.Context type can be passed as an argument in the uesrController. Function (c) because echo.Context satisfies the context interface of interfaces / controllers / context.go. (That is, the echo.Context type has all the methods of the context interface.)

infrastructure/router.go


package infrastructure

import (
	"github.com/Le0tk0k/go-rest-api/interfaces/controllers"
	"github.com/labstack/echo"
	"github.com/labstack/echo/middleware"
)

func Init() {
	e := echo.New()

	userController := controllers.NewUserController(NewMySqlDb())

    //Output request-based log such as access log
	e.Use(middleware.Logger())
    //If you accidentally panic somewhere in your application, the server will recover so that it can return an error response without dropping it.
	e.Use(middleware.Recover())

	e.GET("/users", func(c echo.Context) error { return userController.GetUsers(c) })
	e.GET("/users/:id", func(c echo.Context) error { return userController.GetUser(c) })
	e.POST("/users", func(c echo.Context) error { return userController.CreateUser(c) })
	e.PUT("/users/:id", func(c echo.Context) error { return userController.UpdateUser(c) })
	e.DELETE("/users/:id", func(c echo.Context) error { return userController.DeleteUser(c) })

	e.Logger.Fatal(e.Start(":8080"))
}

Finally, call it from server.go.

server.go


package main

import "github.com/Le0tk0k/go-rest-api/infrastructure"

func main() {
	infrastructure.Init()
}

Execution

Now let's hit the completed API. This time I would like to use ** Postman **. (https://www.postman.com/downloads/)

First, set the Content-Type of Headers to application / json. スクリーンショット 2020-09-19 16.09.22.png

GetUsers Try to get all users. Send a GET request to / users. スクリーンショット 2020-09-19 16.14.07.png

Getuser Try to get any user. Send a GET request to / users /: id. スクリーンショット 2020-09-19 16.16.59.png

CreateUser First, let's create a User. Send a POST request to / users. スクリーンショット 2020-09-19 16.11.47.png

UpdateUser Try updating any user. Send a PUT request to / users /: id. スクリーンショット 2020-09-19 16.18.02.png

DeleteUser Try deleting any user. Send a DELETE request to / users /: id. スクリーンショット 2020-09-19 16.19.14.png

When I check it with GET, it is deleted firmly. スクリーンショット 2020-09-19 16.19.24.png

It's working well!

finally

I should have just set up a development environment with Docker and created a REST API, but it was difficult to study clean architecture, but I think it was a great opportunity to know the architecture. On the contrary, it seems that the API is complicated with a small API like this time, but I had the impression that it would be good if it was a large scale. (~~ Actually, I don't understand much ~~) I would like to study design again!

Recommended Posts

I tried to build an API server with Go (Echo) x MySQL x Docker x Clean Architecture
I tried to make an introduction to PHP + MySQL with Docker
I tried to build the environment of PlantUML Server with Docker
Build Rails (API) x MySQL x Nuxt.js environment with Docker
I tried to build an environment using Docker (beginner)
I tried to link chat with Minecraft server with Discord API
I tried to build an http2 development environment with Eclipse + Tomcat
I tried to build a Firebase application development environment with Docker in 2020
I tried to verify AdoptOpenJDK 11 (11.0.2) with Docker image
[Rails] How to build an environment with Docker
How to build an environment of [TypeScript + Vue + Express + MySQL] with Docker ~ Vue edition ~
I started MySQL 5.7 with docker-compose and tried to connect
I tried to draw animation with Blazor + canvas API
Rails6 I tried to introduce Docker to an existing application
How to build [TypeScript + Vue + Express + MySQL] environment with Docker ~ Express ~
I tried to build the environment little by little using docker
I tried to build the environment of WSL2 + Docker + VSCode
I tried BIND with Docker
I tried to create a padrino development environment with Docker
I tried to create an API to get data from a spreadsheet in Ruby (with service account)
02. I made an API to connect to MySQL (MyBatis) from Spring Boot
I tried connecting to MySQL using JDBC Template with Spring MVC
How to build [TypeScript + Vue + Express + MySQL] environment with Docker ~ MySQL edition ~
I tried to make an Android application with MVC now (Java)
I tried to check the operation of gRPC server with grpcurl
How to build Rails + Vue + MySQL environment with Docker [2020/09 latest version]
How to build [TypeScript + Vue + Express + MySQL] environment with Docker ~ Sequelize ~
I tried to build a laravel operating environment while remembering Docker
I tried to interact with Java
Update MySQL from 5.7 to 8.0 with Docker
Build an environment of "API development + API verification using Swagger UI" with Docker
I tried to make a Web API that connects to DB with Quarkus
How to build a Ruby on Rails development environment with Docker (Rails 6.x)
How to build a Ruby on Rails development environment with Docker (Rails 5.x)
After all I wanted to preview the contents of mysql with Docker ...
I installed WSL2 without using Microsoft Store and tried to build an environment where Docker can be used
I tried running Docker on Windows Server 2019
I tried to get started with WebAssembly
I tried using Scalar DL with Docker
Deploy to heroku with Docker (Rails 6, MySQL)
I made an eco server with scala
Build an authentication proxy server using Docker
I tried to build AdoptOpenjdk 11 on CentOS 7
What is Docker? I tried to summarize
I tried to build Ruby 3.0.0 from source
Implement simple CRUD with Go + MySQL + Docker
Build an environment with Docker on AWS
I tried to implement ModanShogi with Kinx
How to build Rails 6 environment with Docker
I tried to create a portfolio with AWS, Docker, CircleCI, Laravel [with reference link]
I tried to make a machine learning application with Dash (+ Docker) part3 ~ Practice ~
[First environment construction] I tried to create a Rails 6 + MySQL 8.0 + Docker environment on Windows 10.
[Java] I tried to connect using a connection pool with Servlet (tomcat) & MySQL & Java
[Error resolution] Occurs when trying to build an environment for spring with docker
I tried to make Basic authentication with Java
I tried to manage struts configuration with Coggle
I tried to manage login information with JMX
Rails6 [API mode] + MySQL5.7 environment construction with Docker
I tried to build AdoptOpenJDK 8 (Addition: Amazon Corretto 8)
How to build API with GraphQL and Rails
I tried to build Micra mackerel in 1 hour!