I tried to build an API server with Go (Echo) x MySQL x Docker x Clean Architecture

Introduction

As the title says, I made an API server with Go x MySQL x Docker. in the beginning,

・ Create a simple REST API with Go ・ Set up a development environment with Docker

So, this article is recommended for those who want to write API with Go, set up a development environment with Docker, and want to adopt a clean architecture at that time. On the contrary, it may not be very helpful for those who want to study a solid clean architecture. ..

In addition, it should be noted ** Echo ** in the framework (https://echo.labstack.com/) ** Gorm ** (https://github.com/go-gorm/gorm) to access the DB using.

Click here for repository ↓

Preparation

Development environment

I built a development environment with Docker.

Dockerfile Dockerfile Official Reference is a good reference for how to write a basic Dockerfile.

Now, let's take a quick look at them in order.

First is Go.

docker/api/Dockerfile

FROM golang:1.14

#use go module
ENV GO111MODULE=on 

#Specify the directory to run the application
WORKDIR /go/src/github.com/Le0tk0k/go-rest-api

#Go to the above directory.mod and go.copy sum
COPY go.mod go.sum ./
#Cache can be used if the above file is unchanged
RUN go mod download

COPY . .
RUN go build .

RUN go get github.com/pilu/fresh

EXPOSE 8080

#Start the server with the fresh command
CMD ["fresh"]

You can now hot reload on github.com/pilu/fresh. Other than that, there is nothing special about it.

reference -Using go mod download to speed up Golang Docker builds ・ Go v1.11 + Docker + fresh to create a hot reload development environment and enjoy Go language life

Next is MySQL.

docker/mysql/Dockerfile

FROM mysql

EXPOSE 3306

#Copy the MySQL configuration file into the image
COPY ./docker/mysql/my.cnf /etc/mysql/conf.d/my.cnf

CMD ["mysqld"]

The MySQL configuration file looks like this: Since the authentication method at the time of connection has changed in MySQL 8.0 or later, it seems that the description on the second line is necessary. Also, the character code is changed to prevent garbled characters.

docker/mysql/my.cnf

[mysqld]
default_authentication_plugin=mysql_native_password
character-set-server=utf8mb4

[client]
default-character-set=utf8mb4

If you put the initialization SQL and script in the directory /docker-entrypoint-initdb.d/, the data can be initialized automatically when you first start the image.

So, let's create a table.

docker/mysql/db/init.sql

CREATE DATABASE IF NOT EXISTS go_rest_api;
USE go_rest_api;

CREATE TABLE IF NOT EXISTS users (
  id          INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
  name        VARCHAR(256) NOT NULL,
  age         INT NOT NULL,
  created_at  TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  updated_at  TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

reference -Create DB for development environment with Docker + MySQL

docker-compose.yml You can also refer to Compose File / Official Reference.

docker-compose.yml

version: '3'
services: 
  db:
    build: 
      #Specify the path of the directory where the Dockerfile is located
      context: .
      dockerfile: ./docker/mysql/Dockerfile
    ports: 
      #Specify public port
      - "3306:3306"
    volumes: 
      #Initialize the data at startup
      - ./docker/mysql/db:/docker-entrypoint-initdb.d
      #mysql persistence
      - ./docker/mysql/db/mysql_data:/var/lib/mysql
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: go_rest_api
      MYSQL_USER: user
      MYSQL_PASSWORD: password

  api:
    build:
      context: .
      dockerfile: ./docker/api/Dockerfile
    volumes: 
      - ./:/go/src/github.com/Le0tk0k/go-rest-api
    ports: 
      - "8080:8080"
    depends_on:
      #db starts first
      - db

Since mysql is persisted, the previous data will remain even if the container is restarted.

Also, by specifying depend_on for api, db will start first, and then api will start. However, depend_on only specifies the order of startup and does not wait until it is completed, so another measure is required. The countermeasures are introduced in the implementation section.

reference

-docker-compose + MySQL5.7 (8.0) + Initialization + Persistence

Clean architecture

Clean Architecture is the architecture proposed in The Clean Architecture.

Am I an architecture in the first place? What that? It was in a state, but after reading the following article, I think I understood about 1mm of supernatant. There are many other articles, so please check them out!

Reference (There are also articles other than clean architecture)

-I read a book on clean architecture, so I implemented an API server -Let's understand "Layered Architecture + DDD" now. (Golang) -Try building API Server with Clean Architecture -[Golang + Layered Architecture] Try to implement Web API with DDD in mind

Implementation

Directory structure

The directory structure is as follows.

├── docker
│   ├── api
│   │   └── Dockerfile
│   └── mysql
│       ├── Dockerfile
│       ├── db
│       │   ├── init.sql
│       │   └── mysql_data
│       └── my.cnf
├── docker-compose.yml
├── domain
│   └── user.go
├── infrastructure
│   ├── router.go
│   └── sqlhandler.go
├── interfaces
│   ├── controllers
│   │   ├── context.go
│   │   ├── error.go
│   │   └── user_controller.go
│   └── database
│       ├── sql_handler.go
│       └── user_repository.go
|── usecase
│   ├── user_interactor.go
│   └── user_repository.go
├── server.go
├── go.mod
├── go.sum

domain → Entities layer infrastructure → Frameworks & Drivers layer interfaces → Interface layer usecase → Use cases layer

It corresponds to.

Domain (Entities layer)

The innermost layer, which is independent of anything. We will define the User model.

For json:"-", if you write-, it will not be output.

domain/user.go

package domain

import "time"

type User struct {
	ID        int       `gorm:"primary_key" json:"id"`
	Name      string    `json:"name"`
	Age       int       `json:"age"`
	CreatedAt time.Time `json:"-"`
	UpdatedAt time.Time `json:"-"`
}

type Users []User

Interfaces / database, Infrastructure (Interface layer, Frameworks & Driver layer)

Next, we will implement the database peripherals. Since the database connection is an external connection, define it in the outermost Infrastructure layer.

infrastructure/sqlhandler.go

package infrastructure

import (
	"fmt"
	"time"

	"github.com/Le0tk0k/go-rest-api/interfaces/database"
	"github.com/jinzhu/gorm"
	_ "github.com/jinzhu/gorm/dialects/mysql"
)

type SqlHandler struct {
	Conn *gorm.DB
}

func NewMySqlDb() database.SqlHandler {

	connectionString := fmt.Sprintf(
		"%s:%s@tcp(%s:%s)/%s?charset=utf8mb4&parseTime=true&loc=Local",
		"user",
		"password",
		"db",
		"3306",
		"go_rest_api",
	)

	conn, err := open(connectionString, 30)
	if err != nil {
		panic(err)
	}

    //Check if you can connect
	err = conn.DB().Ping()
	if err != nil {
		panic(err)
	}

    //Output log details
	conn.LogMode(true)
    //Set DB engine
	conn.Set("gorm:table_options", "ENGINE=InnoDB")

	sqlHandler := new(SqlHandler)
	sqlHandler.Conn = conn

	return sqlHandler
}

//Make sure that the api container starts after confirming the startup of MySQL
func open(path string, count uint) (*gorm.DB, error) {
	db, err := gorm.Open("mysql", path)
	if err != nil {
		if count == 0 {
			return nil, fmt.Errorf("Retry count over")
		}
		time.Sleep(time.Second)
		count--
		return open(path, count)
	}
	return db, nil
}

func (handler *SqlHandler) Find(out interface{}, where ...interface{}) *gorm.DB {
	return handler.Conn.Find(out, where...)
}

func (handler *SqlHandler) Create(value interface{}) *gorm.DB {
	return handler.Conn.Create(value)
}

func (handler *SqlHandler) Save(value interface{}) *gorm.DB {
	return handler.Conn.Save(value)
}

func (handler *SqlHandler) Delete(value interface{}) *gorm.DB {
	return handler.Conn.Delete(value)
}

In opan (), the api container is set to start after waiting for the db container to start. This time, I implemented this feature in Go code, but is it better to implement it in a shell script? I don't know the best practice ... reference -How to wait until MySQL starts with docker-compose up (2 types introduced)

In addition, the infrastructure layer is only a db connection, and the actual processing is implemented in the interfaces / database layer.

interfaces/database/user_repository.go

package database

import (
	"github.com/Le0tk0k/go-rest-api/domain"
)

type UserRepository struct {
	SqlHandler
}

func (userRepository *UserRepository) FindByID(id int) (user domain.User, err error) {
	if err = userRepository.Find(&user, id).Error; err != nil {
		return
	}
	return
}

func (userRepository *UserRepository) Store(u domain.User) (user domain.User, err error) {
	if err = userRepository.Create(&u).Error; err != nil {
		return
	}
	user = u
	return
}

func (userRepository *UserRepository) Update(u domain.User) (user domain.User, err error) {
	if err = userRepository.Save(&u).Error; err != nil {
		return
	}
	user = u
	return
}

func (userRepository *UserRepository) DeleteByID(user domain.User) (err error) {
	if err = userRepository.Delete(&user).Error; err != nil {
		return
	}
	return
}

func (userRepository *UserRepository) FindAll() (users domain.Users, err error) {
	if err = userRepository.Find(&users).Error; err != nil {
		return
	}
	return
}

I am embedding a SqlHandler in the UserRepository. I'm importing the domain layer, which is the innermost layer, so the dependencies are preserved.

However, since SqlHandler is defined in the outermost Infrastructure layer, the dependencies are directed from the inside to the outside, and the rules may not be followed. However, the SqlHandler called here is not a structure defined in the Infrastructure layer, but an interface defined in the interface / database layer, which is the same layer. This is called ** DIP (Dependency Reversal Principle) **.

What this means is that in interfaces / database / user_repository.go, if you call the process defined in the Infrastructure layer, the dependency will change from inside to outside and you will not be able to protect the dependency. In order to avoid that, let's define the interaction with DB in the same layer with interface and call it. With this, although the processing is actually performed in the Infrastructure layer, the dependency is protected because the interfaces / database / user_repository.go calls the SqlHandler interface.

And in Go, when ** type T has all the methods defined in interface I, it means that interface I is implemented **, so the SqlHandler interface of interfaces / databases / sql_hander in the Infrastructure.SqlHandler structure. Defines the method defined in.

interfaces/database/sql_handler.go

package database

import "github.com/jinzhu/gorm"

type SqlHandler interface {
	Find(interface{}, ...interface{}) *gorm.DB
	Create(interface{}) *gorm.DB
	Save(interface{}) *gorm.DB
	Delete(interface{}) *gorm.DB
}

Usecase (Use Case layer)

The Usecase layer is responsible for receiving information from the interfaces / database layer and passing it to the interfaces / controller layer.

usecase/user_interactor.go

package usecase

import "github.com/Le0tk0k/go-rest-api/domain"

type UserInteractor struct {
	UserRepository UserRepository
}

func (interactor *UserInteractor) UserById(id int) (user domain.User, err error) {
	user, err = interactor.UserRepository.FindByID(id)
	return
}

func (interactor *UserInteractor) Users() (users domain.Users, err error) {
	users, err = interactor.UserRepository.FindAll()
	return
}

func (interactor *UserInteractor) Add(u domain.User) (user domain.User, err error) {
	user, err = interactor.UserRepository.Store(u)
	return
}

func (interactor *UserInteractor) Update(u domain.User) (user domain.User, err error) {
	user, err = interactor.UserRepository.Update(u)
	return
}

func (interactor *UserInteractor) DeleteById(user domain.User) (err error) {
	err = interactor.UserRepository.DeleteByID(user)
	return
}

UserInteractor calls UserRepository, but if this is also called from the Interfase layer, it will depend on the inside → outside and it will not be possible to protect the dependency, so create user_repository.go in the same layer and implement the interface * Use * DIP (Principle of Dependency Reversal) ** to protect dependencies.

usecase/user_repository.go

package usecase

import "github.com/Le0tk0k/go-rest-api/domain"

type UserRepository interface {
	FindByID(id int) (domain.User, error)
	Store(domain.User) (domain.User, error)
	Update(domain.User) (domain.User, error)
	DeleteByID(domain.User) error
	FindAll() (domain.Users, error)
}

Interfaces / controllers, Infrastructure (Interface layer, Frameworks & Driver layer)

Then we will implement the controller and router.

interfaces/controllers/user_controller.go

package controllers

import (
	"strconv"

	"github.com/Le0tk0k/go-rest-api/domain"
	"github.com/Le0tk0k/go-rest-api/interfaces/database"
	"github.com/Le0tk0k/go-rest-api/usecase"
)

type UserController struct {
	Interactor usecase.UserInteractor
}

func NewUserController(sqlHandler database.SqlHandler) *UserController {
	return &UserController{
		Interactor: usecase.UserInteractor{
			UserRepository: &database.UserRepository{
				SqlHandler: sqlHandler,
			},
		},
	}
}

func (controller *UserController) CreateUser(c Context) (err error) {
	u := domain.User{}
	c.Bind(&u)
	user, err := controller.Interactor.Add(u)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(201, user)
	return
}

func (controller *UserController) GetUsers(c Context) (err error) {
	users, err := controller.Interactor.Users()

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, users)
	return
}

func (controller *UserController) GetUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	user, err := controller.Interactor.UserById(id)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, user)
	return
}

func (controller *UserController) UpdateUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	u := domain.User{ID: id}
	c.Bind(&u)

	user, err := controller.Interactor.Update(u)

	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(201, user)
	return
}

func (controller *UserController) DeleteUser(c Context) (err error) {
	id, _ := strconv.Atoi(c.Param("id"))
	user := domain.User{ID: id}

	err = controller.Interactor.DeleteById(user)
	if err != nil {
		c.JSON(500, NewError(err))
		return
	}
	c.JSON(200, user)
	return
}

Since echo uses echo.Context, define the interface of the method used this time.

interfaces/controllers/context.go

package controllers

type Context interface {
	Param(string) string
	Bind(interface{}) error
	JSON(int, interface{}) error
}

interfaces/controllers/error.go

package controllers

type Error struct {
	Message string
}

func NewError(err error) *Error {
	return &Error{
		Message: err.Error(),
	}
}

Since routing uses echo (using an external package), we will implement it in the Infrastructure layer. Note that the echo.Context type can be passed as an argument in the uesrController. Function (c) because echo.Context satisfies the context interface of interfaces / controllers / context.go. (That is, the echo.Context type has all the methods of the context interface.)

infrastructure/router.go

package infrastructure

import (
	"github.com/Le0tk0k/go-rest-api/interfaces/controllers"
	"github.com/labstack/echo"
	"github.com/labstack/echo/middleware"
)

func Init() {
	e := echo.New()

	userController := controllers.NewUserController(NewMySqlDb())

    //Output request-based log such as access log
	e.Use(middleware.Logger())
    //If you accidentally panic somewhere in your application, the server will recover so that it can return an error response without dropping it.
	e.Use(middleware.Recover())

	e.GET("/users", func(c echo.Context) error { return userController.GetUsers(c) })
	e.GET("/users/:id", func(c echo.Context) error { return userController.GetUser(c) })
	e.POST("/users", func(c echo.Context) error { return userController.CreateUser(c) })
	e.PUT("/users/:id", func(c echo.Context) error { return userController.UpdateUser(c) })
	e.DELETE("/users/:id", func(c echo.Context) error { return userController.DeleteUser(c) })

	e.Logger.Fatal(e.Start(":8080"))
}

Finally, call it from server.go.

server.go

package main

import "github.com/Le0tk0k/go-rest-api/infrastructure"

func main() {
	infrastructure.Init()
}

Execution

Now let's hit the completed API. This time I would like to use ** Postman **. (https://www.postman.com/downloads/)

First, set the Content-Type of Headers to application / json.

GetUsers Try to get all users. Send a GET request to / users.

Getuser Try to get any user. Send a GET request to / users /: id.

CreateUser First, let's create a User. Send a POST request to / users.

UpdateUser Try updating any user. Send a PUT request to / users /: id.

DeleteUser Try deleting any user. Send a DELETE request to / users /: id.

When I check it with GET, it is deleted firmly.

It's working well!

finally

I should have just set up a development environment with Docker and created a REST API, but it was difficult to study clean architecture, but I think it was a great opportunity to know the architecture. On the contrary, it seems that the API is complicated with a small API like this time, but I had the impression that it would be good if it was a large scale. (~~ Actually, I don't understand much ~~) I would like to study design again!