WebFaction
Community site: login faq

There is a ulimit on servers that set to 200 for 'process per user'. Problem is that linux counts threads as process, so even in 200MB plan i can't run many django sites ( on one account 'normal way'. Support recommend me to start other process on different ssh user. This looks as an very ugly way.

Why this limit set to such low level? If you give me premission to launch more threads with different user, why not just make me happy and set ulimit to 4096?

asked 17 Aug '11, 06:08

zeus
313
accept rate: 0%


Hi,

The limit is set to 200 because, for the entire machine, there are only 65,000 processes available. That includes all of the system processes and processes we run so you don't have to, such as MySQL, PostgreSQL, iptables, etc. If we upped the limit to 4096 ~16 would be able to use the server one of them being root. This is, of course, not functional. Running 4096 processes would be ~20 secondary accounts + your main account and probably over your 200MB limit.

Running the process as a secondary user that is tied to your account is a fairly good solution since it still allows you to run processes if you are above your process limit on your main account and gives you the opportunity to use the control panel as you normally would.

permanent link

answered 17 Aug '11, 14:20

klynton
1.6k1210
accept rate: 41%

  1. I ran multiple django sites on mod_wsgi
  2. 'max user processes' - limit both threads and process counts for user, that means i can't serve more than 200 requests simultaneously across all sites (even if it uses low ram and cpu).
  3. Threads in linux doesn't use 65,000 PID pool, max number of threads in system is 540672 (cat /proc/sys/kernel/threads-max) that's 'enough for everyone'.
  4. I ran multiple django sites on mod_wsgi with multithreading instead multiprocessing (low memory usage and no useless pid usage).
  5. We use fabric scripts to deploy our project. Using different ssh username for process owner will give us a huge problem in deployment (we can't use suid too ().
  6. ok, i can undersntad why not 4096, but why not 1000? 500? And if i CAN do it to but should use more actions - why not just allow me to use it in more simple way?
(17 Aug '11, 15:48) zeus
  1. See 2.

  2. After refreshing my memory on http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading it looks like you are using the pre-fork method instead of the worker method (otherwise it wouldn't limit your ability to server mover than 200 concurrent requests). This is more of a configuration issue rather than the limit being too strict.

  3. The number in threads-max can be edited by just running an echo command so that's not particularly relevant.

  4. I'm not sure this is the most efficient way to serve the files. Opening 1 thread per request is, partially, one of the reasons that we have the limit in place. The limit protects all the users on the server from another user accidentally or purposefully "fork bombing" the server.

  5. In the fabric scripts I run I set a variable env.username = username for the machine I'm deploying to, that should work for your situation as well.

  6. As I mentioned in 4 this limit also helps to protect agains fork bombing the server and causing downtime for all customers on the machine. You are able to do it in multiple SSH users because then they are still contained. 1000 threads spawning at the same time can easily take the server offline, depending on the size of the threads and the amount of memory used by each. In the past we had the limit set higher and it caused numerous "semaphore" problems. Setting the limit to 200 has fixed those problems.

(18 Aug '11, 14:58) klynton
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×5
×2

question asked: 17 Aug '11, 06:08

question was seen: 3,670 times

last updated: 18 Aug '11, 14:59

                              
WEBFACTION
REACH US
SUPPORT
LEGAL
© COPYRIGHT 2003-2020 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM