[DOCKER] Deploy Django + React from scratch to GKE (1) Backend development --Nginx + Django

Deploy Django + React from scratch to GKE (1) Backend development --Nginx + Django

Thing you want to do

I wanted to develop an application with Djagno + React configuration and deploy it to Google Kubernetes Engine, but I didn't think there was a cohesive tutorial, so I wrote it.

However, ** I think there are some points that are not perfect yet **, but I think that if you have some experience, you can use it immediately.

Caution

This is a record of an inexperienced hobby engineer struggling to deploy to create a portfolio. If there are any deficiencies, please point them out. ..

Aiming figure

environment

$ node --version
v12.14.1

$ npm --version
6.13.7

$ python --version
Python 3.7.4

$ docker --version
Docker version 19.03.8

OS windows10 pro

Start locally

Create a directory

#Creating a project folder
$ mkdir gke-django-tutorial
$ cd gke-django-tutorial

#Create a directory
$\gke-django-tutorial\mkdir backend
$\gke-django-tutorial\mkdir frontend

Backend development (Django edition)

The backend pod delivers the Rest API with Django-rest-framework. Let's organize the inside of the backend pod.

Role Container image
Proxy server Nginx: 1.17.4-alpine
Application Python3.7 --Django rest framework
cloud_sql_proxy gcr.io/cloudsql-docker/gce-proxy

Create a directory in the backend.

#Move to backend directory
$\gke-django-tutorial\cd backend

#Create django directory
$\gke-django-tutorial\backend\mkdir web-back

#Create Nginx directory
$\gke-django-tutorial\backend\mkdir nginx

Start a project with Django

We will create a virtual environment for Python and develop an API server with the Django rest framework. Create this in the backend \ web-back \ directory.

# web-back directory
$\gke-django-tutorial\backend\cd web-back

#Creating a virtual environment for Python
$\gke-django-tutorial\backend\web-back\python -m venv venv

#Enable virtual environment
$\gke-django-tutorial\backend\web-back\venv\Scripts\activate

#Python package installation
(venv)$\gke-django-tutorial\backend\web-back\python -m install --upgrade pip setuptools
(venv)$\gke-django-tutorial\backend\web-back\python -m install django djangorestframework python-dotenv

#Start a Django project
(venv)$\gke-django-tutorial\backend\web-back\django-admin startproject config .

By doing django-admin startprject config . under the web-back directory A Django project folder called config has been created.

Let's check if the local server starts.

(venv)$\gke-django-tutorial\backend\web-back\python manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
April 27, 2020 - 11:22:06
Django version 3.0.5, using settings 'config.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.

Since the development server has started, you can see the screen of The install worked successfully! By accessing http: // localhost: 8000 /.

settings.py

Edit config / settings.py to include the basic settings. Confidential information in settings.py should be written in the .env file so that it is not disclosed. Use the python-dotenv package to change it to use the information contained in .env.

# .Creating an env file
(venv)$\gke-django-tutorial\backend\web-back\type nul > .env
# config/settings.py

import os
from dotenv import load_dotenv  #add to

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.basename(BASE_DIR)  #add to

# .Load env
load_dotenv(os.path.join(BASE_DIR, '.env'))  #add to

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.getenv('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = os.getenv('DEBUG')

ALLOWED_HOSTS = ["*"]  #Change


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'config.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],  #Change
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'config.wsgi.application'


# Database
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}


# Password validation
# https://docs.djangoproject.com/en/3.0/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/3.0/topics/i18n/

LANGUAGE_CODE = 'ja'  #Change

TIME_ZONE = 'Asia/Tokyo'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.0/howto/static-files/

STATIC_URL = '/static/'

#Where to refer to static files in the development environment
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static')] #add to

#Where to reference static files in production
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') #add to

#Media file path
MEDIA_URL = '/media/' #add to

.env


# .env
SECRET_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
DEBUG = False

Add application

Let's create a todo application.

(venv)$\gke-django-tutorial\backend\web-back\python manage.py startapp todo

Add todo and rest_framework to ʻINSTALLED_APPS in config / settings.py`.

# config/settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # 3rd party
    'rest_framework',

    # Local
    'todo.apps.TodoConfig',
]

#add to
REST_FRAMEWORK = {
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',
    ]
}

rest_framework.permissions.AllowAny is for removing the default setting'DEFAULT_PERMISSION_CLASSES' implicitly set by django-rest-framework.

todo/models.py

Create a todo application model.

# todo/models.py
from django.db import models


class Todo(models.Model):
    title = models.CharField(max_length=200)
    body = models.TextField()

    def __str__(self):
        return self.title

Add the created model to todo / admin.py.

# todo/admin.py
from django.contrib import admin
from .models import Todo


admin.site.register(Todo)

Migrate.

(venv)$\gke-django-tutorial\backend\web-back\python manage.py makemigrations
Migrations for 'todo':
  todo\migrations\0001_initial.py
    - Create model Todo

(venv)$\gke-django-tutorial\backend\web-back\python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, todo
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying sessions.0001_initial... OK
  Applying todo.0001_initial... OK

createsuperuser

Create an admin user.

(venv)$\gke-django-tutorial\backend\web-back\python manage.py createsuperuser
username(leave blank to use '[YOUR_NAME]'): [USER_NAME]
mail address: [email protected]
Password:
Password (again):
Superuser created successfully.

Start your development server and go to http: // localhost: 8000 / admin / to see the Django admin site login screen. Enter the set user name and password to log in.

If you can log in, you can check the table of the created application Todo. Let's add a few items.

URLs

Add a route to the todo appison in config / urls.py.

# config/urls.py

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include('todo.urls'))  #add to
]

todo/urls.py

Create todo / urls.py.

(venv)$\gke-django-tutorial\backend\web-back\type nul > todo\urls.py
# todo/urls.py
from django.urls import path, include
from .views import ListTodo, DetailTodo

urlpatterns = [
    path('<int:pk>/', DetailTodo.as_view()),
    path('', ListTodo.as_view())
]

todo/selializers.py

Create a serializer to easily convert the model instance to json format.

(venv)$\gke-django-tutorial\backend\type nul > todo\serializers.py
# todo/serializers.py
from rest_framework import serializers
from .models import Todo


class TodoSerializer(serizers.ModelSerializer):
    class Meta:
        model = Todo
        fields = ('id', 'title', 'body')

ʻId at fields = ('id','title','text') is automatically added by Django if you don't specify a PrimaryKey` in model.

todo/views.py

When creating views.py with Django rest framework, it inherits ~~ APIView from rest_framework.generics.

# todo/views.py

from django.shortcuts import render
from rest_framework import generics
from .models import Todo
from .serializers import TodoSerializer


class ListTodo(generics.ListAPIView):
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer


class DetailTodo(generics.RetrieveAPIView):
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer

I haven't set up the router etc., but for the time being, I'm ready to deliver the items of the Todo application as Rest API. You can check the API view by accessing http: // localhost: 8000 / api / on the development server.

So far, we've been developing in a local environment, which is common in Django.

CORS

How Django (localhost: 8000) interacts with React (localhost: 3000) for json It is necessary to set CORS (Cross-Origin Resource Sharing).

Let's install django-cors-headers.

(venv)$\gke-django-tutorial\backend\web-back\python -m pip install django-cors-headers

Update config / settings.py.

# config/settings.py

# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # 3rd party
    'rest_framework',
    'corsheaders',

    # Local
    'todos.apps.TodosConfig',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'corsheaders.middleware.CorsMidddleware',  #add to
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

##################
# rest_framework #
##################

REST_FRAMEWORK = {
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',
    ]
}

CORS_ORIGIN_WHITELIST = (
    'http://localhost:3000',
)

local_settings.py

Assuming that config / settings.py will be used in a production environment, create config / local_settings.py and keep it for local development. By separating settings.py so that CloudSQL is used when deploying GKE and sqlite3 is used locally, it is not necessary to rewrite the setting values.

#File creation
(venv)$\gke-django-tutorial\backend\web-back\type nul > config/local_settings.py
# config/local_settings.py
from .settings import *

DEBUG = True

ALLOWED_HOSTS = ['*']

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

Start the development server using config / local_settings.py.

(venv)$\gke-django-tutorial\backend\web-back\python manage.py runserver --settings config.local_settings

Tests

Write a test.

# todos/test.py

from django.test import TestCase
from .models import Todo


class TodoModelTest(TestCase):

    @classmethod
    def setUpTestData(cls):
        Todo.objects.create(title="first todo", body="a body here")

    def test_title_content(self):
        todo = Todo.objects.get(id=1)
        excepted_object_name = f'{todo.title}'
        self.assertEqual(excepted_object_name, 'first todo')

    def test_body_content(self):
        todo = Todo.objects.get(id=1)
        excepted_object_name = f'{todo.body}'
        self.assertEqual(excepted_object_name, 'a body here')
(venv)$\gke-django-tutorial\backend\web-back\python manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
..
----------------------------------------------------------------------
Ran 2 tests in 0.007s

OK
Destroying test database for alias 'default'...

It seems to have worked.

Static file

Aggregate static files so that the css of the administrator function is reflected after deployment. The directory for aggregating static files for distribution is staticfiles /, and the static file directory added for development is static /.

#Directory for static file delivery
(venv)$\gke-django-tutorial\backend\web-back\mkdir staticfiles

#Static file development directory
(venv)$\gke-django-tutorial\backend\web-back\mkdir static

#Static file aggregation
(venv)$\gke-django-tutorial\backend\web-back\python manage.py collectstatic

You can see that admin CSS etc. are also added under the staticfiles / directory.

Add Python package

Use Cloud SQL Postgres when deploying to GKE. You need psycopig2 to use Postgres from Django. Also, use gunicorn to launch the application.

Install the required packages additionally, and put together the Python packages installed in the virtual environment in requirements.txt.

#Package installation
(venv)$\gke-django-tutorial\backend\web-back\python -m pip install wheel gunicorn psycopg2-binary

# requirements.Update txt
(venv)$\gke-django-tutorial\backend\web-back\python -m pip freeze > requirements.txt

When executed, requirements.txt will be created under backend /.

asgiref==3.2.7
Django==3.0.5
django-cors-headers==3.2.1
djangorestframework==3.11.0
gunicorn==20.0.4
psycopg2-binary==2.8.5
python-dotenv==0.13.0
pytz==2019.3
sqlparse==0.3.1

Creating a Dockerfile

Create a Dockerfile to create a container image on the Django side.

#Creating a Dockerfile
(venv)$\gke-django-tutorial\backend\web-back\type nul > Dockerfile

# .Creating docker ignore
(venv)$\gke-django-tutorial\backend\web-back\type nul > .dockerignore
# backend/web-back/Dockerfile

# set base image
FROM python:3.7

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# set work directory
WORKDIR /code

# install dependencies
COPY requirements.txt ./
RUN python3 -m pip install --upgrade pip setuptools
RUN pip install -r requirements.txt

# Copy project
COPY . ./

# Expose application port
EXPOSE 8000

Also create a .dockerignore to separate the files you don't want to put in the container.

.dockerignore


venv/
.env
Dockerfile
config/local_settings.py

Now you're ready to create a Docker image for Django.

Development of backend (Nginx edition)

Place the Nginx container as a reverse proxy server in the backend-Pod. Nginx uses the configuration file in /etc/nginx/conf.d/ to define the reverse proxy function.

Also, at the end of the backend development, I would like to start it with docker-compose, so create a file for docker-compose as well.

#File creation for Nginx
$\gke-django-tutorial\backened\nginx\type nul > Dockerfile
$\gke-django-tutorial\backened\nginx\type nul > Dockerfile.dev
$\gke-django-tutorial\backened\nginx\type nul > default.conf
$\gke-django-tutorial\backened\nginx\type nul > default.dev.conf

I set the reverse proxy so that default.conf is Nginx container: 80Django: 8000.

The location = / healthz directive is the health check path you will need after deploying to GKE. The location / static / directive is the path for delivering static files. Without this, the CSS of the administrator screen will not be applied. When deploying to GKE, static files will be delivered from Cloud Storage, so delete them.

The server directive is localhost: 8000 when deploying to GKE, and web-back: 8000 when starting with docker-compose. This is because when starting with docker-compose, it is necessary to resolve the name by service name. When deploying to GKE, it is in the same pod, so name resolution is possible with localhost: 8000.

; default.dev.conf
upstream django {
    server web-back:8000;
}

; default.For conf
; upstream django {
    ; server localhost:8000;
; }

server {

    listen 80;

    location = /healthz {
        return 200;
    }

    location / {
        proxy_pass http://django;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_redirect off;
    }

;Deleted when deploying GKE
    location /static/ {
        alias /code/staticfiles/;
    }
}

Dockerfile reflects the settings by copying the Nginx configuration file to the Nginx container.

# backend\nginx\Dockerfile.dev
FROM nginx:1.17.4-alpine

RUN rm /etc/nginx/conf.d/default.conf
COPY default.dev.conf /etc/nginx/conf.d

# backend\nginx\Dockerfile.For dev
# COPY default.conf /etc/nginx/conf.d

Development of backend (docker-compose edition)

I would like to use docker-compose to start a container with Nginx + Django configuration.

# docker-compose.Create yml
$\gke-django-tutorial\backend\type nul > docker-compose.yml
version: "3.7"

services:
  web-back:
    container_name: python-backend
    env_file: ./web-back/.env
    build: ./web-back/.
    volumes:
      - ./web-back:/code/
      - static_volume:/code/staticfiles # <-- bind the static volume
    stdin_open: true
    tty: true
    command: gunicorn --bind :8000 config.wsgi:application
    networks:
      - backend_network
    environment:
      - CHOKIDAR_USEPOLLING=true
      - DJANGO_SETTINGS_MODULE=config.local_settings
  server:
    container_name: nginx
    build:
      context: ./nginx/.
      dockerfile: Dockerfile.dev
    volumes:
      - static_volume:/code/staticfiles # <-- bind the static volume
    ports:
      - "8080:80"
    depends_on:
      - web-back
    networks:
      - backend_network
networks:
  backend_network:
    driver: bridge
volumes:
  static_volume:
# docker-compose.Start with yml
$\gke-django-tutorial\backend\docker-compose up --build

http: // localhost: 8080 Nginx container: 80Django: 8000 It is port forwarding.

Go to http: // localhost: 8080 / admin / and see if the CSS is reflected.

The environment where you can start backend development by starting with docker-compose in the local environment is ready.

⇒ Go to (2) frontend development: Nginx + React

Recommended Posts

Deploy Django + React from scratch to GKE (1) Backend development --Nginx + Django
Deploy Django + React from scratch to GKE: Table of Contents
Deploy Django + React from scratch to GKE (3) Create a GCP project
Deploy Django + React from scratch to GKE (4) Create cluster and container PUSH
From 0 to Django development environment construction to basic operation
Django memo # 1 from scratch
Django starting from scratch (part: 2)
Django starting from scratch (part: 1)
Deploy django project to heroku
Deploy the Django tutorial to IIS ①
Deploy Django (Ubuntu 14.04 LTS + Nginx + uWSGI + Supervisor)
Introducing Docker Engine to Linux From Scratch