I made a spare2 cheaper algorithm for uWSGI

The application server uWSGI, which can handle not only Python but also Ruby, php, and Perl, has a mode called cheaper. This is a function that dynamically adjusts the number of workers according to the load.

By default, there were spare, backlog, and busyness control algorithms. However, I didn't like any of them, so I created an algorithm called spare2 and merged it into the original family. Since we've included the master branch (development), we've also provided a stable version of Plugins for uWSGI 2.0.

I will explain the spare2 algorithm added this time and why it was difficult to adopt other existing algorithms. (See Documentation for more information on other algorithms.)

spare2

The explanation is based on the sample configuration file.

[uwsgi]
...
processes=10
cheaper-algo=spare2

cheaper=2
cheaper-initial=2
cheaper-step=2
cheaper-idle=30

process = 10 is the maximum number of worker processes. cheaper = 2 is the minimum number of processes and will be adjusted during this time. cheaper-initial = 2 is the number of workers immediately after startup.

The spare2 algorithm attempts to maintain the number of idle workers specified by cheaper. For example, if the number of idle workers becomes one, 2-1 = 1 worker will be started.

cheaper-step = 2 is the maximum number of processes that can be started at one time. When cheaper is large, you can adjust cheaper-step so that it doesn't start too many processes at once.

If the number of idle processes is greater than cheaper for cheaper-idle = 30 seconds, one worker process is stopped. By setting this value longer, you can prevent workers from stopping and restarting in detail.

spare

[uwsgi]
...
processes=10
cheaper-algo=spare

cheaper=2
cheaper-initial=2
cheaper-step=2
cheaper-overload=5

The spare algorithm counts when there are no processes in the idle state (= overload) and when there are two or more processes in the idle state (= idle). If you count up one, the other will be reset, but if you have only one idle process, neither will be counted.

When the overload count reaches cheaper-overload = 5, it adds cheaper-step = 2 worker processes. When the idle count reaches cheaper-overload = 5, stop one worker process.

This algorithm requires a small cheaper-overload and a large cheaper-step to quickly increase the number of workers, such as when a server is added under load distribution. Still, the initial response time tends to be significantly worse because new workers are not launched until all worker processes are busy.

Also, if you set cheaper-overload = 1, the number of workers tends to increase or decrease little by little. For example, if you have about 40 workers, it feels overkill to stop a worker with just two idle processes.

backlog

It is similar to spare, but it makes an overload judgment by taking advantage of the fact that Linux can monitor the number of connections accumulated in the TCP backlog.

It's out of the question because it inherits the weaknesses of spare and isn't available when nginx and uWSGI are connected on unix sockets.

busyness

Busyness complicates spare.

Let cheaper-overload be the time frame, and let busyness be the percentage of workers who were idle during that time. Set the minimum and maximum values of this busyness, and if it falls below the minimum value, the number of workers will decrease, and if it exceeds the maximum value, the number of workers will increase.

Other options are quite complicated, such as the option to make it difficult to reduce the number of workers, and the option to monitor the backlog and add workers without waiting for the time frame, but this also increases the number of workers smoothly at the beginning. It is difficult to handle the inrush load.

Recommended Posts

I made a spare2 cheaper algorithm for uWSGI
I made a dash docset for Holoviews
I made a library for actuarial science
I made a python dictionary file for Neocomplete
〇✕ I made a game
I made a useful tool for Digital Ocean
I made a downloader for word distributed expression
I made a peeping prevention product for telework.
I made a user management tool for Let's Chat
I made a window for Log output with Tkinter
I made a cleaning tool for Google Container Registry
I made a VM that runs OpenCV for Python
[Python] I made a classifier for irises [Machine learning]
I made a python text
Made a command for FizzBuzz
I made a discord bot
[VSCode] I made a user snippet for Python print f-string
I made a resource monitor for Raspberry Pi with a spreadsheet
I made a learning kit for word2vec / doc2vec / GloVe / fastText
I made a face diagnosis AI for a female professional golfer ③
I touched PyAutoIt for a moment
I made a Line-bot using Python!
I made a CUI-based translation script (2)
I made a wikipedia gacha bot
I made a fortune with Python.
I made a CUI-based translation script
I made a daemon with Python
Python> I made a test code for my own external file
I made a client / server CLI tool for WebSocket (like Netcat for WebSocket)
I made a lot of files for RDP connection with Python
I made a development environment for Django 3.0 with Docker, Docker-compose, Poetry
I made a scaffolding tool for the Python web framework Bottle
I made a Python wrapper library for docomo image recognition API.
I made a Docker container to use JUMAN ++, KNP, python (for pyKNP).
I made a new AWS S3 bucket
I made a payroll program in Python!
I touched "Orator" so I made a note
[Updated Ver1.3.1] I made a data preprocessing library DataLiner for machine learning.
I made a character counter with Python
I made an alternative module for japandas.DataReader
Beginner: I made a launcher using dictionary
I made a conversation partner like Siri
I made a script to display emoji
I made a Hex map with Python
I made a life game with Numpy
I made a stamp generator with GAN
I made a browser automatic stamping tool.
After studying Python3, I made a Slackbot
I made a roguelike game with Python
I made a simple blackjack with Python
I made a configuration file with Python
I made a WEB application with Django
A textbook for beginners made by Python beginners
I made a neuron simulator with Python
[For beginners] I made a motion sensor with Raspberry Pi and notified LINE!
I made a stamp substitute bot with line
I made a competitive programming glossary with Python
I made a weather forecast bot-like with Python.
I made a GUI application with Python + PyQt5
I made a Twitter fujoshi blocker with Python ①
Procedure for creating a LineBot made with Python