Using Flask with Nginx + gunicorn configuration [Local environment]

Summary of this article

Flask_Nginx_unicorn_diagramdrawio-Step1-Server (1).png

Article quick reference table

――When you read a similar article, why often? What do you mean? I feel-> [What is / what is ○○? ](What does # mean) --I should move-> [Try to move](# Try to move) --I want to know about the configuration file of gunicorn and Nginx-> [Unravel the configuration file and start / end commands](#Unravel the configuration file start / end commands)

Try to move

First, let's actually operate with the above configuration. Install the required packages.

brew install nginx
pip install gunicorn

Project structure and contents of each file

PROJECT
├── config
│   ├── gunicorn_settings.py #Application server configuration file
│   └── nginx.conf           #Web server configuration file
└── flask_app.py             #Main application

--Application file
This time, we don't care about the behavior of the application, so we will prepare only two routes to return the character string.

flask_app.py


from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
    return 'Index Page!'

@app.route('/health')
def health():
    return 'Health Check OK!!'

--Web server configuration file

gunicorn_settings.py


import os

bind = '127.0.0.1:' + str(os.getenv('PORT', 9876))
proc_name = 'Infrastructure-Practice-Flask'
workers = 1

--Nginx configuration file

nginx.conf


#See the configuration file section for details
worker_processes  1;

events {
	worker_connections 512;
}

http {
    server {
    	listen  9123;
    	server_name INFRA-PRACTICE-NGINX;
		charset UTF-8;

		proxy_set_header    Host    $host;

    	location / {
        	proxy_pass http://127.0.0.1:9876;
    	}

	    location /health/ {
        	proxy_pass http://127.0.0.1:9876/health;
    	}
    }
}

Start a process

Terminal


#Start the web server
nginx -c ${PROJECT_PATH}/config/nginx.conf

#Start application server
gunicorn flask_app:app -c config/gunicorn_settings.py

Confirm communication

Terminal


curl http://localhost:9123/
# -> Index Page!OK if it returns
curl http://localhost:9123/health/
# -> Health Check OK!!OK if it returns

Unravel the configuration file and start / end commands

gunicorn

setting file

In the Official Document, it is described in the optional format. When using a configuration file, use the key = value format with the configuration item name as the key (* e.g. * loglevel ='debug'). The extension of the configuration file is arbitrary. Some of the settings are shown below.

key meaning Mold        Default value
reload Automatically reread and restart when code changes bool False
accesslog Access log output destination to the server
Specify the file, "-Means standard output
e.g. accesslog = logs/gunicorn_access.log
string None
errorlog Error log output destination
Specify the file, "-Means standard output
e.g. errorlog = logs/gunicorn_err.log
string '-'
proc_name Process name displayed by the ps command
Default unless otherwise specified_proc_It becomes the name "gunicorn"
string None
limit_request_line Maximum size of request, specified by DDos measures
Limit to similar items_request_fields
int 4094
bind Connection host and port
HOST:Specified in PORT format
e.g. bind = os.getenv('HOST') + ':' + os.getenv('PORT')
string 127.0.0.1:8000
daemon Start the gunicorn process as a daemon bool False
workers Number of Worker processes handling requests
One parent process is started, and as many child processes as there are workers are set up and handled here.
Specifying 2 or more when parallelism cannot be considered will cause a problem.
int 1
worker_connections Maximum number of connections int 1000

Start / end command

【Start-up】

Terminal


gunicorn {Flask application file name}:{Flask instance variable name in the file} -c {Configuration file path}

[End] Ctrl + C in Terminal where process is running

Nginx

setting file

Looking at the official docs, it's different from gunicorn's key = value format. You can see that the Directive (instruction) is described in the context format (* curly bracket: * block enclosed in {}) inside the Module.

Module

Module name Contents to be described
core Log, process control
This is the outermost context
events Event processing
http Processing related to the Web server
Probably the most described module

Directive Syntax in http context A server context within the http context and a further location context within the server context And so on, we will write in context in context format. Here is an example of setting the http block. By looking at the example, you can understand what the configuration file at the beginning means.

--Basic settings

config


http {
    server {
        listen        80;                          #Wait for connection on port 80
        server_name   MY_SERVER;                   #The server name is MY_SERVER
        charset       UTF-8;                       #Content of response header-type to UTF-Designated as 8
        error_page    404  /404_not_found.html;    #404 when status code is 404_not_found.Internally redirect to html
    }
}

--Reverse proxy

config


http {
    server {
        # /Access to index
        location /index {
            proxy_pass          http://127.0.0.1:8080;  # 127.0.0.Reverse proxy to port 1 8080
            proxy_set_header    Host    $host;          #Of the request headers passed to the proxied server, host$Reconfigure host
        proxy_read_timeout  60s;                    #Time out if there is no response from the proxied server for 60 seconds
        }
    }
}

--Access control / IP filtering

config


config
http {
    server {
        location / {
            allow 192.168.0.1;    # 192.168.0.Allow access from 1
            deny  all;            #Deny any other network access
        }
    }
}

--BASIC authentication

config


http {
    server {
        location / {
            auth_basic             "Basic Auth closed site";  #Clarify that BASIC authentication is required
            auth_basic_user_file   conf/htpasswd;               # .htpasswd Specify the file path
        }
    }
}

--Referer check

config


http {
    server {
        #There is no Referer in the request header(none), In the header but http://Or https://Does not start with(blocked)、
        # mintak.Does not end with com(server_names)Judged as invalid when accessing from the root->$invalid_Set referer to 1
        valid_referers none blocked server_names *mintak.com
        if ($invalid_referer) {
           return 403;  #If it is determined to be invalid, return it with 403 Forbidden
        }
    }
}

Embbed Variable Something like an environment variable in an Nginx server. For more information, click here (https://nginx.org/en/docs/http/ngx_http_core_module.html#variables). Some are shown below.

variable Contents
$host Host in request header, server name if not
$remote_addr Connection client address
$remote_port Connection client port
$status Response status
$server_name Server name
$http_name Set the header field name in lowercase and underscore in the HTTP request header field and name part.
e.g. http_host、http_referer

Start / end command

【Start-up】

Terminal


nginx -c {Configuration file path}

[End]

Terminal


nginx -s stop

#If the above fails, identify the process and kill
ps ax | grep nginx
kill -9 ${PID}

[Startup confirmation]

Terminal


ps aux | grep nginx    #Here nginx:If you see master process, nginx is running.

#Command that can confirm PORT
lsof -i -P | grep nginx
#=> nginx     41845 mintak    6u  IPv4 0x595e7f947462ada5      0t0  TCP *:9123 (LISTEN)

What is ○○?

You can achieve the purpose by preparing the files up to this point and hitting the command. However, there are many difficult words and concepts for beginners like myself. From here, I will describe my own interpretation of such keywords.

Why is it such a configuration

According to an expert, the Web server and application server were not separated in the past. This configuration became a problem when we faced the fact that the load on the server increased due to the increase in the number of accesses to services that are open to the public due to the spread of the Internet, and the overall performance deteriorated. did. Therefore, aiming for stable operation by distributing the load, the flow is to divide the server according to the role as described later. This idea is commonplace in modern times, and it seems that Web services are generally made up of a three-layer structure of Web server <-> application server <-> DB server. I think that the basic idea of ** separating processes (servers) for each role ** is similar to the design concept for coding (one clear role for each class / function).

But it works with python flask_app.run

This is because the framework called Flask is a framework that can be started while concurrently serving as a Web server / application server. It is useful when it is difficult to set up the Web server and the application server separately, such as ** for simple verification and operation check during development **. When starting with this command, the log "WARNING: This is a development server. Do not uses it in a production development.`" appears [^ 1], but it is for development only, and with this configuration, mass access It means that there is a big risk of a flat tire in the process, so ** stop in production **.

The main role of the web server and application server

I will summarize each in one word. See the Reference link page for details.

--Web server: Supports requests that do not change regardless of who accesses them (static processing). The TOP page is easy to hit. --Application server: Supports applications (dynamic processing) whose response changes according to access. Member My Page, etc. (Display contents differ depending on the member).

Why is a reverse proxy reverse?

If there is a reverse, there is a forward **. What we usually call a proxy is a forward proxy.

What is the difference between a forward proxy and a reverse proxy?

They are still proxies to each other. Processing between the client-> Internet is a forward proxy (server), and processing between the Internet-> target Web server is a reverse proxy (server). [^ 2]

Flask_Nginx_unicorn_diagramdrawio-Step1-Porxy.png

You can connect it directly to the target server! You might think, but you can benefit from biting a proxy server (see below).

Use of forward proxy

Use Details
Labor saving By temporarily caching the data of the site that was accessed once, it is possible to shorten the time until data acquisition as a flow to acquire data from the cache without accessing the Web for the second and subsequent accesses.
Content filtering Block access to specific sites here.
~~It's mostly because of this guy that you can't see the naughty site on your company PC~~

Use of reverse proxy

Use Details
Enhanced security Specific access method(http://、https://Such)ではない場合や特定のIPアドレス以外からのアクセスを遮断するSuchの機構を持たせることが出来る。
load distribution(Load balancer) Once cushioned, it successfully distributes access to multiple application servers.
Speeding up Static exploitation. It is possible to reduce the load on the server and return a faster response by holding the reference content such as images and html files in the cache.

What is WSGI?

Abbreviation for Web Server Gateway Interface. ~~ Commonly known as Uesugi-kun. ~~ Django and Flask are frameworks that support WSGI, and gunicorn and uWEGI are servers that support WSGI, so I think there is no problem with recognition. The background of its appearance is as follows. [^ 3]

・ Framework A-chan> I'm cool, so I'm crazy except α-kun, a cool web server ・ Framework B-chan> I like the solid β-kun ♡

In the past, the web servers that can be used for each Python web framework were limited (and vice versa). Actually, I wanted B-chan to use α-kun, but here the marriage-like relationship with A-chan is a drag. It was ~~ Uesugi-kun and ~~ WEGI who appeared to solve this problem. WEGI says, "Let's freely choose each other for the server and framework ~~ love ~~ Let's connect", how to interact in the relationship of application framework <-> Web server I set the rule. Thanks to WSGI and WSGI-compatible servers and frameworks, we are now free to choose the combination of server and framework. I'm happy.

References -Gunicorn Official Document -Nginx document (original) -Nginx document (Japanese translation) -Summary of Nginx settings (Qiita) -Difference between Web server and AP server (Think IT)

[^ 1]: If you haven't set FLASK_ENV = development. [^ 2]: This time, Nginx is both a proxy server and a web server. [^ 3]: I think this is the only way to explain this stupid thing. .. ..

Recommended Posts

Using Flask with Nginx + gunicorn configuration [Local environment]
[With image diagram] Nginx + gunicorn + Flask converted to Docker [Part 2]
[With image diagram] Nginx + gunicorn + Flask converted to Docker [Part 1]
Using Chainer with CentOS7 [Environment construction]
Web application created with Python + Flask (using VScode) # 1-Virtual environment construction-
Create a Python3.4 + Nginx + uWSGI + Flask Web application execution environment with haste using pyenv on Ubuntu 12.04
Competitive programming with python Local environment settings
code-server Local environment (3) Try using VSCode Plugin
Deploy flask app with mod_wsgi (using pipenv)
Vue.js + Flask environment construction memorandum ~ with Anaconda3 ~
Get a local DynamoDB environment with Docker
Create Python + uWSGI + Nginx environment with Docker
Nginx setting summary (Flask + Docker + Gunicorn edition)
Build NGINX + NGINX Unit + MySQL environment with Docker
Flask + Gunicorn + Nginx + Supervisor Until it works
Create a web application execution environment of Python3.4 + Nginx + uWSGI + Flask with haste using venv on Ubuntu 14.04 LTS
Python local development environment construction template [Flask / Django / Jupyter with Docker + VS Code]
Build Python + django + nginx + MySQL environment using docekr
Build Django + NGINX + PostgreSQL development environment with Docker
Make Django's environment Docker (Docker + Django + Gunicorn + nginx) Part 2
Try using conda virtual environment with Jupyter Notebook
Create Nginx + uWSGI + Python (Django) environment with docker
Send msgpack with ajax to flask (werkzeug) environment
Make Django's environment Docker (Docker + Django + Gunicorn + nginx) Part 3
Virtual environment construction with Docker + Flask (Python) + Jupyter notebook
Create an animated GIF local server with Python + Flask
Full-scale server made with Nginx + uWSGI + Flask + Ubuntu (installation)
Output log to console with Flask + Nginx on Docker
[Python] I tried running a local server using flask
Full-scale server made with Nginx + uWSGI + Flask + Ubuntu (implementation)
Create a django environment with docker-compose (MariaDB + Nginx + uWSGI)