Communication between uWSGI and Nginx using Unix Domain Socket

This article is the 16th day article of Tokyo Gakugei University Hashiyama Laboratory Advent Calendar 2020.

Introduction

This article is about learning about Unix Domain Socket communication through the content of running an application created with flask, a Python web framework, using a uWSGI server.

What is Unix Domain Socket?

Unix Domain Socket is a function installed in POSIX OS, and unlike socket communication by TCP/UDP, it creates a ** high-speed network interface ** that is completed inside the kernel. Since it is completed inside the kernel, it cannot be connected to an external computer.

Creating an application for flask

Create a simple application that accesses with the endpoint / and returns only Hello, World.

app.py

from flask import Flask
app = Flask(__name__)

@app.route("/")
def index():
    return "Hello World"

if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True)

Then, execute the following command to start the application. At this time, the embedded server for development starts up at startup.

python app.py

http://localhost:5000/ If you access and the message Hello, World is displayed, you are successful.

Run with uWSGI

In the previous example, we used an embedded server for development. Python has uWSGI as an application server. uWSGI supports communication with Unix Domain Socket.

First, let's run the previous application using uWSGI.

pip install uWSGI

app.py

from flask import Flask
app = Flask(__name__)

@app.route("/")
def index():
    return "Hello World"

if __name__ == "__main__":
    app.run()

To start the above application with uWSGI, enter the following command.

uwsgi --http=0.0.0.0:4000 --wsgi-file=app.py --callable=app

http://localhost:4000/ If you access and the message Hello, World is displayed, you are successful.

It is also possible to start with a shorter command by creating a configuration file called uwsgi.ini.

uwsgi.ini

[uwsgi]
wsgi-file=app.py
callable=app

http=0.0.0.0:4000

uwsgi uwsgi.ini

Introducing Nginx

Now, here is the access with Unix Domina Socket, which is the main subject. The conventional startup method was access via HTTP communication (TCP socket).

In actual operation, Nginx will often be used as a reverse proxy in consideration of running multiple applications on the same server. At this time, if the communication is within the same server, it can be accessed faster than HTTP communication using Unix Domain Socket.

This time, I will use Docker for convenience.

Editing uwsgi.ini

Edit uwsgi.ini to communicate with Unix Domain Socket. Unix Domain Socket creates a socket file at a specified location on the file system. The client attempts to access the socket file. In other words, the communication partner is determined by the file path instead of the IP address and port number.

This time, create a socket file called /var/app/app.sock.

uwsgi.ini

[uwsgi]
wsgi-file=app.py
callable=app

http=0.0.0.0:4000
#Postscript
socket=/var/app/app.sock

Preparing the Docker environment

First, prepare a Dockerfile that prepares the environment for running the flask application.

Dockerfile

FROM python:3.7

ENV LANG C.UTF-8
ENV TZ Asia/Tokyo

WORKDIR /app
ADD app.py ./
ADD uwsgi.ini ./

RUN pip install flask uWSGI

CMD ["uwsgi", "uwsgi.ini"]

I also want to run nginx, so prepare docker-compose.yml.

docker-compose.yml

version: '3.7'

services:

  app:
    container_name: uds_app
    build:
      context: .
      dockerfile: ./Dockerfile
    volumes:
      - ./tmp/:/var/app/
    tty: true

  nginx:
    container_name: uds_nginx
    image: nginx
    volumes:
      - ./tmp:/var/app/
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 80:80
    tty: true

The important thing here is that the ./tmp directory on the host machine is shared with both the flask application container and the Nginx container.

1. ./tmp directory on the host machine ← → flask / var/app directory of the application container
2. ./tmp directory on the host machine ← →/var/app directory of Nginx container

This allows you to reference the socket file from the Nginx container as well.

Describe the Nginx configuration file so that access to / calls the application of flask.

nginx.conf

server {
    listen       80 default_server;
    listen       [::]:80 default_server;
    server_name  _;

    location / {
    include uwsgi_params;
    uwsgi_pass unix:/var/app/app.sock;
    }

    error_page 404 /404.html;
    location = /404.html {
    }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
    }
}

Let's start it with the docker-compose up command.

docker-compose up

http://localhost/ If you can access and confirm the character string Hello, World as before, it is successful.

in conclusion

In this article, we dealt with the fact that the reverse proxy Nginx communicates with the application created by flask using Unix Domain Socket.

I would like to convey the part that determines the communication partner according to the file path, unlike TCP communication. Also, if I can afford it, I would like to investigate the performance.