|
I'm running a WordPress site using FastCGI and every week or so the number of fcgi processes goes from the usual 7 to something approaching 30 and everything ends up being killed by WebFaction because the memory usage has exceeded the maximum 1GB. The .htaccess file has the following at the top: Here's an normal ps listing: And here's an example of when it has gone wrong: I thought the SetHandler php56-fcgi6 was supposed to limit the number of FastCGI processes to 6? Does anyone have any suggestions as to how we can further investigate this, and find out what is causing the explosion in processes? Thanks in advance! asked 17 Jul '18, 16:57 pd2018 |
|
The thread limit set should limit the number of threads started, unless there is a forking bug or something in your PHP trying to execute a command in shell which is failing. With FCGI you should be using a task queue to manage external threads. It is hard to see if this is the case from the data above. Ideally you would want to use a debugger like strace while the problem is happening to get more info from the stuck threads. First identify the PID of the process you want, Once you know the PID you can run strace, replace PID with the number from ps, This will provide verbose information from which you may be able to figure out what is causing the failure. answered 17 Jul '18, 21:02 johns |
|
Thank you for the suggestion, I have incorporated strace into my script that monitors memory usage and this will hopefully give me some more clues. In the meantime, I have noticed that we have the following crontab entry, I don't know when or why this was added, or indeed if it is part of some standard WebFaction configuration: Is this still needed these days, and what does/did it do? Thanks! answered 21 Jul '18, 09:32 pd2018 phpstack was an old custom apache and php that was often used on some CentOS 5 servers when customers needed something specific. You should be able to safely delete that cron job after verifying none of your websites is still using that application.
(21 Jul '18, 11:54)
aaront ♦♦
|
|
Update: I'm now logging memory usage every five minutes, and if there are more than the normal number of php56.fcgi processes then I do an strace on all of them to see what they are doing. The normal number of php56.fcgi processes seems to be 7 - one of these is parent to the other six, e.g. as follows: I can't get info about the parent process (15286), though. I assume this is hidden from me by WebFaction's shared server? Five minutes later, another seven processes have appeared, so the total number is now 14. At the end of each line above is what the strace says the process is doing - "busy" means it seems to me like it's doing WordPress stuff. Five minutes later, the number of processes has grown to 21 - notice this is a multiple of 7 - and they are all in either wait4() or accept(): Five minutes later, 7 of the processes have disappeared (it is the most recent 7 that have gone): And another five minutes later we are back to the original 7 processes: This pattern repeats itself several times a day, and the total number of fcgi process is always a multiple of 7. Does this give any clue as to what could be going wrong? Does anyone know exactly how these fcgi processes are created and managed? In another incident later that day, we managed to get up to a total of 70 fcgi processes, which exceeded the 1GB memory limit: At this point all 70 fcgi processes were in wait4() or accept(), with the majority being in accept() Here most are in wait4() or accept(), a couple seem to be doing WordPress stuff. Any ideas what could explain this behaviour? Is it even worth using FastCGI anyway? Are other people using it? Should we just disable FastCGI and go back to whatever the default PHP setup is? Thanks in advance, suggestions welcome! answered 06 Sep '18, 00:53 pd2018 |
|
Hi, just wondering if anyone has any advice on how I can further investigate this? Does anyone know how the FastCGI mechanism is controlled, and what could be going wrong in this case? I can't seem to find anything useful in the WebFaction documentation about this. How can I debug this? Another option is to disable FastCGI and go back to the default config. We switched to FastCGI about five years ago because we were finding that we'd regularly run of out memory (and get a warning from WebFaction) due to too many CGI processes being created (as I understood one CGI process is created for each incoming request). The idea of FastCGI was to limit the number of processes being created. Nowadays we have a much higher 1GB memory limit though, so maybe disabling FastCGI now is a practical option. A final option is just to ignore the fact that we exceeded our memory limit every week or so, and allow WebFaction to kill the processes as and when. I think it would be far better to understand exactly what is going on, though. answered 26 Sep '18, 08:21 pd2018 |
