Run a Python web application with Docker

Last time: Operating Emby with Docker This is the trajectory of my trial and error that I want to be able to use Docker soon.

Python web application with Docker

Last time, we migrated the media streaming service Emby to containers. Untitled(11).png

This time, I created a Docker image that can deploy a Python web application. Migrate WebSocket-based and Django applications to Docker. Then move the static content to the container and close the VM server. Untitled(12).png

Static content

I moved what was on the VM server to the Docker host and bound it to the distribution directory when the container started.

docker-compose.yml


    deploy:
        image: nginx
        restart: always
        ports:
            - 80:80
            - 443:443
        volumes:
            - /home/fclef/docker/server/deploy/nginx/conf:/etc/nginx/conf.d
+           - /home/fclef/docker/server/deploy/content/html:/usr/share/nginx/html

Docker operation of Python application

Since the application using Web Socket is also made of Python (bottle), we aim to create a container that can easily deploy Python applications.

Container design

I often make web applications in Python. So, rather than consolidating the entire application into an image Imagine a foundation that can run any Python application The concrete application itself passes git information at the time of container startup, and clones and expands it in the container. Untitled(9) (1).png

Dockerfile Whenever I write something in Python, I use Pipenv. Also, in the case of a Web application, it is made into a socket and delivered by nginx, so Install Pipenv and nginx at the time of the image.

Dockerfile


FROM python:3.8-slim-buster

#Installation of dependent packages
RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt-get install -y git make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libs
qlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev nginx

#Install Pyenv
RUN git clone https://github.com/yyuu/pyenv.git /.pyenv
ENV PYENV_ROOT /.pyenv
ENV PATH $PYENV_ROOT/bin:$PATH
RUN pyenv init - | bash
#Installing Pipenv
RUN pip install pipenv
ENV PIPENV_VENV_IN_PROJECT 1
ENV PIPENV_YES true

RUN mkdir /app #Application deployment directory
RUN mkdir /script #Various script placement directories

COPY up.sh /script #Startup script

#I will clone the git project to app in the startup script, so move it
WORKDIR /app
#Web application delivery is a reverse proxy via the main web server, so leave only 80 ports open.
EXPOSE 80
RUN chmod +x /script/up.sh
ENTRYPOINT [ "/script/up.sh" ] #When the container starts, the start script is up.run sh

Container start script

The git information of the application is passed as an environment variable when the container is started. For some reason, I can't clone git with ssh in my environment, so it's a prerequisite to clone with https. After cloning, parse the Pipfile in the project root directory to determine the Python version you need. Some Python libraries need to have dependent programs installed, so The script (dependencies.sh) for installing them is bound at the time of container startup and called here. Since gunicorn is used for socketing Python Web applications, gunicorn is also installed.

up.sh


#!/bin/bash

#Get git information from environment variables
gitCloneUrl=${GIT_CLONE_URL}
gitUserName=${GIT_USER_NAME}
gitPassword=${GIT_PASSWORD}
gitCloneTarget=`echo ${gitCloneUrl} | sed -r "s#^(https?://)(.*)#\1${gitUserName}:${gitPassword}@\2#g
"`

#Get the project name
projectName=`echo ${gitCloneUrl} | sed -r "s#.*/(\w+)\.git#\1#g"`
echo "■ ■ ■ PROJECT NAME <${projectName}> ■ ■ ■"

git clone ${gitCloneTarget}

#Get the python version from the Pipfile
cd ${projectName}
pythonVersion=`grep python_version Pipfile | sed -r 's/python_version = "(.+)"/\1/g'`
echo "■ ■ ■ PYTHON VERSION <${pythonVersion}> ■ ■ ■"

#If some of the Python libraries to be installed depend on other programs, install them here.
if [ -f /script/dependencies.sh ]; then
    source /script/dependencies.sh
fi

pipenv --python ${pythonVersion}
pipenv install
pipenv install gunicorn

curPath=`pwd`
export APP_ROOT=${curPath}

chmod +x /script/run.sh
service nginx start
/script/run.sh #Application launch script
while true; do sleep 1000; done

Start container

As an example, I'll show you a configuration that launches a Django application.

docker-compose.yml excerpt


django:
    image: pipenv-gunicorn
    restart: always
    environment:
        GIT_CLONE_URL: https://xxxxxxxx/user/project.git
        GIT_USER_NAME: user
        GIT_PASSWORD: password
    volumes:
        - /home/fclef/docker/server/app/dependencies.sh:/script/dependencies.sh
        - /home/fclef/docker/server/app/run.sh:/script/run.sh
        - /home/fclef/docker/server/app/app.conf:/etc/nginx/conf.d/nginx_gunicorn.co
nf
    depends_on:
        - deploy
        - gitlab

dependencies.sh


apt-get -y --no-install-recommends install libpq-dev

run.sh


cd /app/project

source .venv/bin/activate

python manage.py collectstatic --noinput
python manage.py makemigrations
python manage.py migrate

deactivate

/app/project/.venv/bin/gunicorn \
          --access-logfile /var/log/socket_success.log \
          --error-logfile /var/log/socket_error.log \
          --workers 1 \
          --bind unix:/run/socket.sock \
          config.wsgi:application

app.conf


server {
    listen       80;
    listen [::]:80;
    server_name xxxx.xxx.xxx;

    root /app;

    location /static {
        alias /app/project/static;
    }

    location / {
        include /etc/nginx/proxy_params;
        proxy_pass http://unix:/run/socket.sock;
    }

    location ~ ^/apple-touch-icon(.*)\.png$ {
        root /app/project/;
        rewrite ^/apple-touch-icon(.+)\.png$ /static/img/apple-touch-icon.png break;
    }
}

The application itself is cloned from git when the container is started, Since the data is stored in PostgreSQL in another container, No matter how many times you recreate this container, you can deploy the service in the operating state normally.

Basically, most Python applications can be run with the above image and startup method. Since the procedure from git clone to deployment is simple, I was able to create an image that is easy to reuse by going out of the application. For applications with complicated environment construction, it may be easier to maintain the image of each application.

This is the series for migrating VM servers to containers.

Recommended Posts

Run a Python web application with Docker
Launch a Python web application with Nginx + Gunicorn with Docker
Start a simple Python web server with Docker
Web application with Python + Flask ② ③
Web application with Python + Flask ④
A memo about building a Django (Python) application with Docker
Deploy a Django application with Docker
Build a web application with Django
Application development with Docker + Python + Flask
Put Docker in Windows Home and run a simple web server with Python
Daemonize a Python web app with Supervisor
Let's make a web framework with Python! (1)
Create a web service with Docker + Flask
Let's make a web framework with Python! (2)
I made a WEB application with Django
If you know Python, you can make a web application with Django
Run Python with VBA
Run prepDE.py with python3
Use python with docker
Run iperf with python
I made a GUI application with Python + PyQt5
[Python] Build a Django development environment with Docker
Steps to develop a web application in Python
Launch a web server with Python and Flask
Create a Python console application easily with Click
Extract data from a web page with Python
[GCP] Procedure for creating a web application with Cloud Functions (Python + Flask)
[Streamlit] I hate JavaScript, so I make a web application only with Python
Implement a simple application with Python full scratch without using a web framework.
Web scraping with python + JupyterLab
Run python with PyCharm (Windows)
Run a machine learning pipeline with Cloud Dataflow (Python)
Run Python with CloudFlash (arm926ej-s)
Let's run Excel with Python
(Python) Try to develop a web application using Django
Web application development with Flask
[ES Lab] I tried to develop a WEB application with Python and Flask ②
Parse and visualize JSON (Web application ⑤ with Python + Flask)
Web application creation with Django
Prepare python3 environment with Docker
Web API with Python + Falcon
Web application made with Python3.4 + Django (Part.1 Environment construction)
Build a machine learning application development environment with Python
Make a fortune with Python
Run Label with tkinter [Python]
Create a Layer for AWS Lambda Python with Docker
Run a Python file with relative import in PyCharm
Web scraping beginner with python
Create a directory with python
Run DHT22 with RasPi + Python
Streamline web search with python
I want to run a quantum computer with Python
Rails application building with Docker
Create a web application that recognizes numbers with a neural network
How to convert an array to a dictionary with Python [Application]
I made a simple book application with python + Flask ~ Introduction ~
Create a simple Python development environment with VSCode & Docker Desktop
Web application with Python3.3.1 + Bottle (1) --Change template engine to jinja2
[Python] What is a with statement?
Solve ABC163 A ~ C with Python
Operate a receipt printer with python