The story that Apache dealt with because it was down at AH00144

At midnight, I was informed that the production system was down. image.png The BtoB service website could not be opened. I shared the information because I urgently dealt with the case that I had to solve it by the business hour of the next day by struggling alone.

environment

A website that runs on Ubuntu 16.04 on Amazon EC2.

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

$ apache2 -v
Server version: Apache/2.4.18 (Ubuntu)

Error log

When I logged in to the server, Apache was dead with an error. When you see the string ** Fatal error **, you'll instantly get sick.

$ tail /var/log/apache2/error.log
[Tue Aug 25 20:51:54.215281 2020] [mpm_prefork:emerg] [pid 930](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:54.215369 2020] [mpm_prefork:emerg] [pid 929](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:54.216849 2020] [mpm_prefork:emerg] [pid 933](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:54.217182 2020] [mpm_prefork:emerg] [pid 931](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:54.218350 2020] [mpm_prefork:emerg] [pid 928](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:54.218757 2020] [mpm_prefork:emerg] [pid 1336](43)Identifier removed: AH00144: couldn't grab the accept mutex
[Tue Aug 25 20:51:55.040404 2020] [core:alert] [pid 1287] AH00050: Child 928 returned a Fatal error... Apache is exiting!
[Tue Aug 25 20:58:05.444208 2020] [mpm_prefork:emerg] [pid 1333](22)Invalid argument: AH00146: couldn't release the accept mutex

ʻAH00144: couldn't grab the accept mutex` is output for the number of processes.

For the time being, when I googled with ** AH00144 **, https://hirose31.hatenablog.jp/entry/2020/03/10/190010 I found an excellent blog! Even in this environment, due to various reasons, Apache was started as a non-system user (uid is 1000 or more). The reproduction method also matches perfectly.

Coping

The procedure is simple.

/etc/apache2/apache2.conf


#Mutex file:${APACHE_LOCK_DIR} default

I just uncommented and rebooted. You may want to create a file like /etc/apache2/conf-available/mutex-file.conf.

Why are you curious about this time? However, I would like to see the situation for a while.

Recommended Posts

The story that Apache dealt with because it was down at AH00144
The story that Japanese output was confused with Django
The story that XGBoost was finally installed
The story that fits in with pip installation
When I crawl the webapi that appears during rendering, it was played with CORS
A story that I was addicted to at np.where
The story that the return value of tape.gradient () was None
The story that my pull request was incorporated into Scipy
The story that the new drawing library "HiPlot" was pretty good
The story that Python stopped working with VS Code (Windows 10)
The story that the private key is set to 600 with chmod
[Apache] The story of prefork
The story that the Homebrew environment was blown away when Anaconda was installed
The story that the guard was confined when the laboratory was converted to IoT
The story of making a module that skips mail with python
Even if I converted jpg to png, I managed to do it because the transparency was not tampered with