Find a guideline for the number of processes / threads to set in the application server

Based on CPU processing power to run the application server It seems that it is good to set an appropriate number of processes / threads, I thought it would be nice if I could find a simple guideline for the actual setting, so I measured it.

Measurement environment

For the server configuration, prepare the following containers with Docker. (The number of CPU cores on each server is 1) --Nginx (Web server) --uwsgi (application server) --MySQL (DB server) At the time of measurement, start the Python application linked with the DB server with uwsgi It is a flow to access uwsgi via Proxy of Nginx of the front server.

Tools used for measurement confirmation --Locust (load test tool) --vmstat command (check system load status)

Measurement contents

This measurement confirmation is focused on the processing capacity of the application server. In the following pattern, I changed the number of processes / threads in the uwsgi setting and measured it. --Process: 1, Thread: 1 --Process: 1, Thread: 2 --Process: 2, Thread: 1 --Process: 2, Thread: 2 --Process: 3, Thread: 1

Make simultaneous requests with 15 clients until a total of 300 requests Consider the request processing status and system load status every 3 seconds at that time as measurement results. Check the request processing status from the result of Locust Check the system load status with the vmstat command for uwsgi (application server).

Item description of Locust and vmstat

Locust

item Description
reqs Current number of requests
Avg Average processing time for one request(ms)
Min Minimum processing time for 1 request(ms)
Max Maximum processing time for 1 request(ms)
Median Median processing time for one request
req/s How many requests can be processed per second

vmstat

[procs]

item Description
r Number of processes waiting to run(Road average)
b Number of non-interruptable sleep processes

[system]

item Description
in Number of interrupts per second
cs Number of context switches per second

[cpu]

item Description
us User time used for execution(User side)
sy System time used to execute kernel code(Kernel side)
id CPU idle rate
wa IO waiting CPU usage

Measurement result

Process: 1, Thread: 1

The load average is low, the processing time for one request, and req / s are not bad overall. In actual operation, it seems that CPU resources can be spared. You may try a slightly higher performance setting.

System load status (vmstat)

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 481572  67852 610612    0    0    23    20  271  526  8  2 90  0  0
 0  0      0 481560  67852 610644    0    0     0     0  226  447  9  1 90  0  0
 1  0      0 480256  67860 610936    0    0     0    31 1207 2606 81 11  9  0  0
 1  0      0 478552  67884 611288    0    0     0    45 1263 2815 89  8  3  0  0
 1  0      0 477824  67908 611696    0    0     0    49 1302 2915 84 13  3  0  0
 2  0      0 477420  67924 612060    0    0     0    24 1351 2995 86 11  3  0  0
 1  0      0 477080  67932 612440    0    0     0     9 1333 2942 85 13  3  0  0
 1  0      0 476280  67964 612720    0    0     0    40 1269 2803 87 10  3  0  0
 1  0      0 476124  67996 613084    0    0     0    44 1361 2989 83 13  3  0  0
 1  0      0 475596  68012 613512    0    0     0    27 1358 2996 85 12  3  0  0
 1  0      0 473144  68020 613888    0    0     0    12 1327 2934 83 14  3  0  0
 2  0      0 472648  68044 614192    0    0     0    39 1322 2958 86 11  3  0  0
 1  0      0 471964  68068 614596    0    0     0  1263 1299 2874 86 11  3  0  0
 1  0      0 471656  68084 614956    0    0     0    24 1349 2973 83 14  4  0  0
 1  0      0 471116  68092 615280    0    0     0    79 1351 2992 85 11  3  0  0
 1  0      0 470464  68116 615716    0    0     0    61 1356 3008 88  9  3  0  0
 0  0      0 470464  68116 615804    0    0     0     0  473 1026 25  3 72  0  0
 0  0      0 470480  68124 615804    0    0     0     7  108  194  0  0 100  0  0
 0  0      0 470604  68124 615804    0    0     0     0  102  192  0  0 100  0  0

Request processing status (Locust)

 Name                # reqs      # fails     Avg     Min     Max  |  Median   req/s
-----------------------------------------------------------------------------------------------------------
 GET /admin/             22     0(0.00%)     982     714    1111  |    1000    6.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/             69     0(0.00%)     950     714    1111  |     970    7.43
-----------------------------------------------------------------------------------------------------------
 GET /admin/            115     0(0.00%)     944     714    1111  |     930    7.80
-----------------------------------------------------------------------------------------------------------
 GET /admin/            162     0(0.00%)     937     714    1111  |     920    7.70
-----------------------------------------------------------------------------------------------------------
 GET /admin/            210     0(0.00%)     925     714    1111  |     920    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            256     0(0.00%)     925     714    1111  |     920    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            289     0(0.00%)     920     714    1111  |     910    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300     0(0.00%)     918     714    1111  |     910    7.80
-----------------------------------------------------------------------------------------------------------

1 Request processing time according to the completion rate of the request

 Name                # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300    910    930    960    980   1000   1000   1100   1100   1111
-----------------------------------------------------------------------------------------------------------

For process: 1, thread: 2

Although the load average is low, there are many cs of 3000 units and the overhead is high. The processing time of one request is getting longer in the second half and the performance is getting worse.

System load status (vmstat)

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 464032  68324 616496    0    0    22    19  264  512  8  2 90  0  0
 0  0      0 464196  68324 616508    0    0     0     0  117  203  0  0 100  0  0
 1  0      0 461796  68332 616712    0    0     0     8  967 2085 64  9 27  0  0
 1  0      0 461472  68356 617068    0    0     0    31 1353 3020 86 11  3  0  0
 1  0      0 461016  68364 617492    0    0     0     9 1373 3081 87  9  3  0  0
 1  0      0 460632  68388 617856    0    0     0    33 1375 3108 86 11  3  0  0
 1  0      0 459980  68412 618228    0    0     0    45 1370 3089 88  9  3  0  0
 1  0      0 459608  68436 618536    0    0     0    40 1386 3097 87 10  3  0  0
 1  0      0 459112  68436 618960    0    0     0     0 1387 3095 89  8  3  0  0
 1  0      0 458796  68460 619228    0    0     0    41 1239 2708 84 13  2  0  0
 1  0      0 458320  68476 619608    0    0     0    24 1372 3081 87 10  3  0  0
 2  0      0 457944  68484 619964    0    0     0     9 1380 3074 84 13  3  0  0
 1  0      0 457348  68508 620384    0    0     0  1301 1362 3048 85 11  3  0  0
 1  0      0 456820  68532 620696    0    0     0    41 1379 3093 87 10  3  0  0
 1  0      0 456236  68548 621056    0    0     0    24 1362 2987 87 10  3  0  0
 1  0      0 455956  68556 621484    0    0     0     9 1381 3109 86 11  4  0  0
 0  0      0 455704  68580 621668    0    0     0    33  871 1909 50  5 45  0  0
 0  0      0 455704  68580 621668    0    0     0    19  101  186  0  0 100  0  0
 0  0      0 455720  68588 621668    0    0     0     7  103  195  0  0 100  0  0

Request processing status (Locust)

 Name                # reqs      # fails     Avg     Min     Max  |  Median   req/s
-----------------------------------------------------------------------------------------------------------
 GET /admin/             24     0(0.00%)     928     693    1042  |     940    6.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/             71     0(0.00%)     902     693    1042  |     910    7.57
-----------------------------------------------------------------------------------------------------------
 GET /admin/            119     0(0.00%)     898     693    1042  |     900    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            164     0(0.00%)     922     693    1210  |     900    7.70
-----------------------------------------------------------------------------------------------------------
 GET /admin/            211     0(0.00%)     921     693    1210  |     910    7.60
-----------------------------------------------------------------------------------------------------------
 GET /admin/            258     0(0.00%)     916     693    1210  |     910    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            290     0(0.00%)     915     693    1210  |     910    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300     0(0.00%)     914     693    1210  |     910    7.80
-----------------------------------------------------------------------------------------------------------

1 Request processing time according to the completion rate of the request
 Name                # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300    910    920    930    940    980   1100   1200   1200   1210
-----------------------------------------------------------------------------------------------------------

Process: 2, Thread: 1

Although the road average is always about 2, it can be maintained at a certain point without continuing to climb. Req / s is high because cs is low and the processing time for one request is short. The performance is the best in this verification.

System load status (vmstat)

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 457296  66832 598732    0    0    25    19  268  520  8  2 90  0  0
 0  0      0 457424  66832 598744    0    0     0    49  130  244  0  0 100  0  0
 2  0      0 449928  66840 598956    0    0     0    21 1271 2193 64 15 21  0  0
 2  0      0 448688  66872 599300    0    0     0    41 1172 2251 88 12  0  0  0
 2  0      0 447952  66888 599700    0    0     0    24 1211 2433 86 13  0  0  0
 1  0      0 447032  66896 600044    0    0     0    41 1165 2203 86 14  0  0  0
 2  0      0 446544  66920 600432    0    0     0    45 1215 2415 84 16  0  0  0
 2  0      0 446016  66944 600804    0    0     0    36 1158 2196 87 13  0  0  0
 2  0      0 445520  66960 601192    0    0     0    55 1202 2383 87 12  0  0  0
 2  0      0 444868  66968 601544    0    0     0     9 1151 2240 87 13  0  0  0
 1  0      0 444404  66992 601912    0    0     0    59 1204 2386 87 13  0  0  0
 2  0      0 443876  67016 602288    0    0     0    37 1192 2353 86 13  0  0  0
 2  0      0 443224  67032 602668    0    0     0    35 1209 2372 82 18  0  0  0
 2  0      0 441448  67040 603028    0    0     0  1348 1174 2254 84 16  0  0  0
 2  0      0 439688  67064 603404    0    0     0    40 1192 2358 81 19  0  0  0
 2  0      0 438664  67088 603792    0    0     0    35 1223 2380 86 13  0  0  0
 0  0      0 438580  67088 603900    0    0     0     0  387  740 19  4 77  0  0
 0  0      0 438612  67096 603900    0    0     0    55   97  204  0  0 100  0  0
 0  0      0 438612  67096 603900    0    0     0     0  101  211  0  0 100  0  0

Request processing status (Locust)

 Name                # reqs      # fails     Avg     Min     Max  |  Median   req/s
-----------------------------------------------------------------------------------------------------------
 GET /admin/             24     0(0.00%)     912     763    1067  |     880    0.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/             71     0(0.00%)     888     718    1125  |     870    6.86
-----------------------------------------------------------------------------------------------------------
 GET /admin/            120     0(0.00%)     875     718    1125  |     860    8.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/            168     0(0.00%)     874     718    1125  |     860    8.10
-----------------------------------------------------------------------------------------------------------
 GET /admin/            216     0(0.00%)     872     718    1125  |     860    8.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/            264     0(0.00%)     871     698    1125  |     860    8.10
-----------------------------------------------------------------------------------------------------------
 GET /admin/            297     0(0.00%)     870     698    1125  |     860    8.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300     0(0.00%)     870     698    1125  |     860    7.90
-----------------------------------------------------------------------------------------------------------

1 Request processing time according to the completion rate of the request(ms)
 Name                # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300    860    910    930    940    980    990   1000   1100   1125
-----------------------------------------------------------------------------------------------------------

For process: 2, thread: 2

Since the setting exceeds the limit of the specifications, the overhead becomes large. The load average is also high and the processing time per request is getting longer and longer. The performance is the worst in this verification.

System load status (vmstat)

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 445772  66372 580060    0    0    26    19  263  506  8  2 90  0  0
 0  0      0 445780  66372 580068    0    0     0     0  108  223  0  0 100  0  0
 3  0      0 445664  66372 580244    0    0     0    40  620 1348 41  9 50  0  0
 2  0      0 444780  66372 580592    0    0     0     0 1130 2288 87 13  0  0  0
 4  0      0 444404  66372 580948    0    0     0    32 1084 2242 89 11  0  0  0
 4  0      0 443996  66372 581320    0    0     0    33 1110 2263 84 16  0  0  0
 3  0      0 443488  66372 581692    0    0     0     9 1122 2334 84 16  0  0  0
 4  0      0 443124  66372 582036    0    0     0    28 1068 2174 87 13  0  0  0
 5  0      0 442768  66372 582404    0    0     0   113 1083 2285 85 15  0  0  0
 3  0      0 442228  66372 582776    0    0     0    40 1112 2348 89 11  0  0  0
 4  0      0 441764  66372 583140    0    0     0     0 1075 2315 88 12  0  0  0
 5  0      0 441272  66372 583512    0    0     0    33 1130 2335 84 16  0  0  0
 5  0      0 440900  66372 583852    0    0     0    33 1094 2242 85 15  0  0  0
 5  0      0 440472  66372 584152    0    0     0  1387  979 2029 91  9  0  0  0
 4  0      0 440028  66372 584512    0    0     0     0 1103 2273 85 15  0  0  0
 5  0      0 439564  66372 584828    0    0     0    47 1077 2291 85 15  0  0  0
 3  0      0 439288  66372 585188    0    0     0    37 1055 2134 87 13  0  0  0
 0  0      0 439908  66372 585208    0    0     0     0  257  478  8  1 91  0  0
 0  0      0 439948  66372 585208    0    0     0     7  101  209  0  0 100  0  0
 0  0      0 439948  66372 585208    0    0     0     0  101  209  0  0 100  0  0

Request processing status (Locust)

 Name                # reqs      # fails     Avg     Min     Max  |  Median   req/s
-----------------------------------------------------------------------------------------------------------
 GET /admin/             23     0(0.00%)     966     718    1225  |    1000    1.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/             71     0(0.00%)     934     684    1225  |     930    6.86
-----------------------------------------------------------------------------------------------------------
 GET /admin/            117     0(0.00%)     925     623    1225  |     930    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            165     0(0.00%)     920     623    1225  |     930    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            211     0(0.00%)     918     623    1225  |     930    8.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/            253     0(0.00%)     956     623    1481  |     950    7.50
-----------------------------------------------------------------------------------------------------------
 GET /admin/            298     0(0.00%)     965     623    1481  |     950    7.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300     0(0.00%)     965     623    1481  |     950    7.10
-----------------------------------------------------------------------------------------------------------

1 Request processing time according to the completion rate of the request(ms)
 Name                # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300    960   1000   1100   1100   1200   1200   1400   1500   1481
-----------------------------------------------------------------------------------------------------------

Process: 3, Thread: 1

The load average is high, and the processing time per request is gradually increasing. Setting the number of processes x3 around one core is a tough impression.

System load status (vmstat)

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 483180  67360 605444    0    0    24    20  271  524  8  2 90  0  0
 0  0      0 483480  67360 605464    0    0     0     0  158  282  0  0 100  0  0
 1  0      0 483520  67368 605520    0    0     0   204  281  571 12  2 86  0  0
 3  0      0 482308  67392 605792    0    0     0   345 1095 2047 82 11  7  0  0
 3  0      0 481108  67408 606172    0    0     0    28 1163 2176 86 14  0  0  0
 3  0      0 476772  67416 606540    0    0     0     9 1110 2053 88 12  0  0  0
 3  0      0 475340  67440 606912    0    0     0    36 1123 2116 85 15  0  0  0
 2  0      0 473732  67464 607272    0    0     0    72 1108 2105 86 14  0  0  0
 3  0      0 472524  67480 607616    0    0     0    27 1154 2202 83 17  0  0  0
 2  0      0 471256  67488 607992    0    0     0    11 1140 2188 85 15  0  0  0
 3  0      0 469984  67512 608296    0    0     0    36 1088 2055 88 12  0  0  0
 3  0      0 469584  67536 608708    0    0     0    33 1166 2223 88 12  0  0  0
 2  0      0 467008  67552 609080    0    0     0    60 1134 2201 85 15  0  0  0
 3  0      0 463308  67560 609432    0    0     0   544 1137 2044 87 13  0  0  0
 3  0      0 462624  67584 609804    0    0     0    40 1163 2146 86 14  0  0  0
 2  0      0 462136  67608 610168    0    0     0   824 1137 2158 83 17  0  0  0
 4  0      0 461604  67608 610532    0    0     0     0 1132 2195 88 12  0  0  0
 0  0      0 461728  67632 610608    0    0     0    35  459  859 22  5 73  0  0
 0  0      0 461672  67632 610608    0    0     0    37  138  253  0  0 100  0  0
 0  0      0 461672  67632 610608    0    0     0     0  122  240  0  1 99  0  0

Request processing status (Locust)

 Name                # reqs      # fails     Avg     Min     Max  |  Median   req/s
-----------------------------------------------------------------------------------------------------------
 GET /admin/             23     0(0.00%)    1040     736    1335  |    1100    3.00
-----------------------------------------------------------------------------------------------------------
 GET /admin/             70     0(0.00%)     961     736    1335  |     940    7.14
-----------------------------------------------------------------------------------------------------------
 GET /admin/            116     0(0.00%)     950     736    1335  |     930    7.80
-----------------------------------------------------------------------------------------------------------
 GET /admin/            162     0(0.00%)     946     736    1335  |     940    7.80
-----------------------------------------------------------------------------------------------------------
 GET /admin/            210     0(0.00%)     935     736    1335  |     920    7.80
-----------------------------------------------------------------------------------------------------------
 GET /admin/            256     0(0.00%)     939     736    1335  |     930    7.80
-----------------------------------------------------------------------------------------------------------
 GET /admin/            287     0(0.00%)     932     736    1335  |     920    7.90
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300     0(0.00%)     935     736    1335  |     920    7.90
-----------------------------------------------------------------------------------------------------------

1 Request processing time according to the completion rate of the request(ms)
 Name                # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
-----------------------------------------------------------------------------------------------------------
 GET /admin/            300    920    960    980    990   1100   1100   1100   1200   1335
-----------------------------------------------------------------------------------------------------------

Summary

Web application performance is also on the Nginx (Web server) side and I / O load on the DB server side. Since it is involved, it is difficult to conclude that this setting is unconditional, but as far as this measurement is concerned, it is on the application server side. I got the impression that Process: n, Thread: 1 performed better than Process: 1, Thread: n. For the time being, as a guide, the number of processes is x2 and the number of threads is 1, while watching the situation. I think it's better to find the optimal setting.

Recommended Posts

Find a guideline for the number of processes / threads to set in the application server
Set the number of elements in a NumPy one-dimensional array to a power of 2 (0 padded)
How to find the optimal number of clusters in k-means
[Python] Programming to find the number of a in a character string that repeats a specified number of times.
Find all patterns to extract a specific number from the set
Create a bot that posts the number of people positive for the new coronavirus in Tokyo to Slack
How to count the number of elements in Django and output to a template
I want to set a life cycle in the task definition of ECS
How to get the number of digits in Python
How to identify the element with the smallest number of characters in a Python list?
I want to set up a mock server for python-flask in seconds using swagger-codegen.
Find out the maximum number of characters in multi-line text stored in a data frame
Define a task to set the fabric env in YAML
Maya | Find out the number of polygons in the selected object
Get the number of specific elements in a python list
Python --Find out number of groups in the regex expression
Get the number of occurrences for each element in the list
Find the eigenvalues of a real symmetric matrix in Python
A server that returns the number of people in front of the camera with bottle.py and OpenCV
[For beginners] I want to explain the number of learning times in an easy-to-understand manner.
I tried to find the trend of the number of ships in Tokyo Bay from satellite images.
[Python] A program to find the number of apples and oranges that can be harvested
How to set the output resolution for each keyframe in Blender
How to determine the existence of a selenium element in Python
Try to get a list of breaking news threads in Python.
[Python] Let's reduce the number of elements in the result in set operations
How to check the memory size of a variable in Python
Put the process to sleep for a certain period of time (seconds) or more in Python
[Introduction to Python] How to use the in operator in a for statement?
How to check the memory size of a dictionary in Python
How to find the memory address of a Pandas dataframe value
How to get the vertex coordinates of a feature in ArcPy
Find the rank of a matrix in the XOR world (rank of a matrix on F2)
Create a function to get the contents of the database in Go
The story of remounting the application server
[Python] A simple function to find the center coordinates of a circle
Get the number of readers of a treatise on Mendeley in Python
The story of creating a store search BOT (AI LINE BOT) for Go To EAT in Chiba Prefecture (1)
Set up a dummy SMTP server in Python and check the operation of sending from Action Mailer
[Question] In sk-learn random forest regression, an error occurs when the number of parallels is set to -1.
[Completed version] Try to find out the number of residents in the town from the address list with Python
A story about trying to improve the testing process of a system written in C language for 20 years
A story about creating a program that will increase the number of Instagram followers from 0 to 700 in a week
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_3
[Introduction to Udemy Python3 + Application] 47. Process the dictionary with a for statement
[Python] A program that calculates the number of socks to be paired
Generate a list packed with the number of days in the current month.
[Python] How to put any number of standard inputs in a list
Check the in-memory bytes of a floating point number float in Python
[Linux] Command to get a list of commands executed in the past
I want to sort a list in the order of other lists
How to put a line number at the beginning of a CSV file
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_1
I made a program to check the size of a file in Python
I tried to display the altitude value of DTM in a graph
How to play a video while watching the number of frames (Mac)
Set up a server that processes multiple connections at the same time
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_2
Change the list in a for statement
The story of creating a store search BOT (AI LINE BOT) for Go To EAT in Chiba Prefecture (2) [Overview]
I wanted to know the number of lines in multiple files, so I tried to get it with a command