WebFaction
Community site: login faq

I wrote a simple python script that generates N random search queries against my site, and fires them off using urllib2.urlopen()/get() in N different threads. I run this script from an external box not on Webfaction.

While I am running the script, I am monitoring the memory usage of my httpd processes, and only one of them increases in memory usage:

$ while true; do clear; ps -u $USER -o rss,command |grep httpd | grep -v grep | awk '{print $1}'; sleep 1; done
1808
3432
3416
3416
3416
3416
21104

(Right after an apache restart, that last process is also ~3500.)

My wsgi config looks like this:

WSGIDaemonProcess django processes=5 python-path=/home/(...)/webapps/django:/home/(...)/webapps/django/lib/python2.6 threads=1

I have tried tweaking processes. The only effect is that the number of httpd.worker processes increases, but it still looks like only one is doing any work.

I have also tried tweaking ThreadsPerChild. This increases the number of simultaneous connections, but they all have to wait for that single process to finish, and the total running time of the script does not change. (Let M=ThreadsPerChild, N=number of threads in my script and T=execution time of my script. If M==N, all threads finish simultaneously after T seconds. If M == 1, threads finish one after another in T/N second intervals. If 1<M<N, threads finish in groups of M. T is constant for all three scenarios.)

asked 25 Nov '10, 17:05

knatten
113
accept rate: 0%

edited 25 Nov '10, 17:05

It is normal for the main apache process to increase in memory and the worker threads to stay relatively level.

(25 Nov '10, 20:16) johns

I discovered I had overlooked a ServerLimit 1 that was default in the httpd.conf.

(30 Nov '10, 11:27) knatten
Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×909
×225
×55

question asked: 25 Nov '10, 17:05

question was seen: 3,832 times

last updated: 30 Nov '10, 11:27

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM