{"id":4617,"date":"2019-02-11T17:57:41","date_gmt":"2019-02-11T19:57:41","guid":{"rendered":"https:\/\/blog.clusterweb.com.br\/?p=4617"},"modified":"2019-02-11T17:57:41","modified_gmt":"2019-02-11T19:57:41","slug":"apache-no-space-left-on-device-couldnt-create-accept-lock","status":"publish","type":"post","link":"https:\/\/blog.clusterweb.com.br\/?p=4617","title":{"rendered":"Apache: No space left on device: Couldn\u2019t create accept lock"},"content":{"rendered":"

The service \u201chttpd\u201d appears to be down.<\/h1>\n

OK just to give an update on this thread, so hopefully it helps someone else in the future.<\/p>\n

Apologies in advance to any Level 18 Fire Breathing +5 SysAdmins; this is noob stuff\u00a0\":)\"<\/p>\n

The cPanel team examined my server and determined that Apache had ran out of Semaphores.<\/p>\n

Each time HTTPD crashed it didn’t release the semaphores and eventually my server ran out (I had 128). The cPanel team kindly cleared down the “locked” semaphores as follows:<\/p>\n

<\/p>\n

\n
Code:<\/div>\n
[19:48:55 mail root@9019765 ~\/cptechsjt]cPs# for i in `ipcs | grep admin | awk {'print $2'}`; do ipcrm -s $i; done\r\n[19:49:06 mail root@9019765 ~\/cptechsjt]cPs# \/scripts\/restartsrv_httpd ===<\/pre>\n<\/div>\n
And the support analyst advised to read this article:<\/p>\n
Apache: No space left on device: Couldn’t create accept lock – major.io<\/a><\/p>\n
After reading it, I googled for corroboratory advice and found this article:<\/p>\n
Semaphore limits and many Apache instances on Linux | End Point<\/a><\/p>\n
So I decide to increase the number of semaphores from 128 to 256 as follows:<\/p>\n
How many semaphores do I have?<\/p>\n
\n
Code:<\/div>\n
# cat \/proc\/sys\/kernel\/sem\r\n250 32000 32 128\r\n<\/pre>\n<\/div>\n
Answer = 128<\/p>\n
Add this line to \/etc\/sysctl.conf:<\/p>\n
\n
Code:<\/div>\n
kernel.sem = 500 64000 64 256\r\n<\/pre>\n<\/div>\n
To make the change take immediate effect, issue this command:<\/p>\n
\n
Code:<\/div>\n
# sysctl -p\r\n<\/pre>\n<\/div>\n
Alternatively perform a graceful reboot of your server, (If you can.)<\/p>\n
So how many semaphores do I have now?<\/p>\n
\n
Code:<\/div>\n
# cat \/proc\/sys\/kernel\/sem\r\n500     64000   64      256\r\n<\/pre>\n<\/div>\n
You can also check the number of “running” semaphores with this command:<\/p>\n
\n
Code:<\/div>\n
# ipcs -s\r\n<\/pre>\n<\/div>\n
I also noticed that one website (wordpress) was creating a disproportionate number of semaphores compared to the other websites (also wordpress) on my server, so it must be a rogue plugin. I will investigate further.<\/p>\n
Thank you to great team at cPanel for helping me out, its genuinely appreciated.<\/p>\n
Best Regards<\/p>\n
Apache: No space left on device: Couldn\u2019t create accept lock<\/h1>\n
This error completely stumped me a couple of weeks ago. Apparently someone was adjusting the Apache configuration, then they checked their syntax and attempted to restart Apache. It went down without a problem, but it refused to start properly, and didn\u2019t bind to any ports.<\/p>\n
Within the Apache error logs, this message appeared over and over:<\/p>\n
[emerg] (28)No space left on device: Couldn't create accept lock<\/pre>\n
Apache is basically saying \u201cI want to start, but I need to write some things down before I can start, and I have nowhere to write them!\u201d If this happens to you, check these items in order:<\/p>\n
1. Check your disk space<\/strong><\/p>\n
This comes first because it\u2019s the easiest to check, and sometimes the quickest to fix. If you\u2019re out of disk space, then you need to fix that problem. \ud83d\ude42<\/p>\n
2. Review filesystem quotas<\/strong><\/p>\n
If your filesystem uses quotas, you might be reaching a quota limit rather than a disk space limit. Use\u00a0repquota \/<\/code>\u00a0to review your quotas on the root partition. If you\u2019re at the limit, raise your quota or clear up some disk space. Apache logs are usually the culprit in these situations.<\/p>\n
3. Clear out your active semaphores<\/strong><\/p>\n
Semaphores? What the heck is a semaphore? Well, it\u2019s actually an\u00a0apparatus for conveying information by means of visual signals<\/a>. But, when it comes to programming,\u00a0semaphores are used for communicating between the active processes of a certain application<\/a>. In the case of Apache, they\u2019re used to communicate between the parent and child processes. If Apache can\u2019t write these things down, then it can\u2019t communicate properly with all of the processes it starts.<\/p>\n
I\u2019d assume if you\u2019re reading this article, Apache has stopped running. Run this command as root:<\/p>\n
# ipcs -s<\/pre>\n
If you see a list of semaphores, Apache has not cleaned up after itself, and some semaphores are stuck. Clear them out with this command:<\/p>\n
# for i in `ipcs -s | awk '\/httpd\/ {print $2}'`; do (ipcrm -s $i); done<\/pre>\n
Now, in almost all cases, Apache should start properly. If it doesn\u2019t, you may just be completely out of available semaphores. You may want to increase your available semaphores, and you\u2019ll need to tickle your kernel to do so. Add this to \/etc\/sysctl.conf:<\/p>\n
kernel.msgmni = 1024\r\nkernel.sem = 250 256000 32 1024<\/pre>\n
And then run\u00a0sysctl -p<\/code>\u00a0to pick up the new changes.<\/p>\n","protected":false},"excerpt":{"rendered":"
The service \u201chttpd\u201d appears to be down. OK just to give an update on this thread, so hopefully it helps someone else in the future. Apologies in advance to any Level 18 Fire Breathing +5 SysAdmins; this is noob stuff\u00a0 The cPanel team examined my server and determined that Apache had ran out of Semaphores. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[455,730,1,830,725,42,51,495,548],"tags":[1289,131,1287,1288,1286,1285,1290,361,923,1284],"class_list":["post-4617","post","type-post","status-publish","format-standard","hentry","category-apache2","category-clusterweb","category-viazap","category-debian","category-hospedagem","category-leitura-recomendada","category-linux-linuxrs","category-profissional-de-ti","category-ubuntu-2","tag-accept","tag-apache","tag-couldnt","tag-create","tag-device","tag-left","tag-lock","tag-no","tag-on","tag-space"],"_links":{"self":[{"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/posts\/4617","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4617"}],"version-history":[{"count":1,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/posts\/4617\/revisions"}],"predecessor-version":[{"id":4618,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=\/wp\/v2\/posts\/4617\/revisions\/4618"}],"wp:attachment":[{"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4617"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4617"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.clusterweb.com.br\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4617"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}