WebFaction
Community site: login faq

Hello (again). I have read all the posts (here in community and in stackoverflow) about memory Django's consumption. I just want to ask one thing:

Every time I hit refresh in my Django Admin page I get +1MB of RSS memory. Is this OK and healthy?

Here is my first ps output after an Apache restart:

ps -u $USER -o rss,pid,command

RSS PID COMMAND 10212 5638 /usr/bin/memcached -d -m 64 -s /home/$USER/memcached.sock -P /home/$USER/memcached.pid 50540 7734 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 51704 7735 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 2620 7736 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 2532 10045 -bash 124680 12794 java -jar -Xmx16M -Xmn4M -Xss256K start.jar --daemon 3176 21117 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 1608 24392 ps -u $USER -o rss,pid,command

Now if I hit F5 in the admin page all the processes stay the same except #7734 which increases by +1MB:

51724 7734 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start

P.S.: My http.conf file is with the defaults. Nothing changed in there and all recommendations about reducing memory consumption have been applied (as per webfaction's documentation).

EDIT: Today I am going to do a little memory debugging with some tools I have read about and post here the results and (maybe?) the solution! Hope everything will work...

asked 17 May '16, 12:18

nik's gravatar image

nik
13019
accept rate: 16%

edited 17 May '16, 13:12


If your httpd memory usage is going up every time you reload your admin page, then there is probably a memory leak in one of your Django project's apps. Debugging with Pympler will help you identify it.

You can also use the max-requests parameter in your WSGIDaemonProcess directive to configure Apache to recycle your worker processes after they've serviced N requests - this will keep their memory usage from growing endlessly.

Hope that helps!

permanent link

answered 18 May '16, 00:39

seanf's gravatar image

seanf
12.1k21635
accept rate: 37%

OK.

Problem #1: Pympler runs on top of django-debug-toolbar which must have Debub = False. Solved by running my app locally.

Problem #2: After running Pympler locally, the Memory panel was not very helpful! On the right hand side it gives me the Megabytes that was reserved extra (i.e 80MB +8MB) and when I click on it, it gives me a summary like the below:

Class Number of instances Show sparklines Total size myApp1.models.MyModel1 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 9, 8, 7, 6, 5, 4 9.34 KB myApp1.models.MyModel2 1, 2, 3, 4, 5, 6, 5, 4, 3, 2, 1, 0, 1, 2, 3, 4, 5 78.73 KB myApp1.models.MyModel3 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 80.20 KB django.contrib.auth.models.User 1 2.92 KB django.template.base.Template 1, 2, 3, 4, 5, 6, 7, 6, 7, 8, 9, 10, 11, 10, 11, 10, 11, 12, 13, 14, 15, 14, 15, 14, 15, 14, 15, 14, 15, 14, 15, 14, 13, 12, 11, 12, 13, 14, 15, 16, 15, 16, 15, 14, 13, 12, 13, 14, 15, 16, 17, 16, 15, 16, 15, 14, 13, 12, 13, 12, 13, 14, 15, 14, 15, 14, 15, 14, 13, 12, 11, 12, 13, 14, 13, 14, 15, 16, 15, 14, 13, 12 123.03 KB django.template.context.Context 1, 2, 1 3.40 MB

What useful info can I extract from this summary?

Apart from the fact that the total size of the above sizes does not much the size that in shown in parenthesis on the right hand side.

I really want to use the maximum-requests directive but the memory leak will still be there and this will not be a good practice IMO. Should I ask this in stackoverflow?

permanent link

answered 18 May '16, 11:21

nik's gravatar image

nik
13019
accept rate: 16%

Unfortunately, I was not able to figure out a solution to my problem, using pympler. In my above post (not the first one) I was wrong because I had set DEBUG = True (I was running my app locally, which is wrong) and the memory should increase. So, reject my previous post.

After I had set my app to run locally with DEBUG = False then the pympler panel inside the django-debug-toolbar panel did not work. Always getting a KeyError < django.template.base.Template > error and didn't know what to do.

I think I will stay with this memory consumption and use maximum-requests inside WSGIDaemonProcess.

If I have any other news, I will post right here.

Best regards to everyone

permanent link

answered 18 May '16, 15:02

nik's gravatar image

nik
13019
accept rate: 16%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×898
×82
×25

question asked: 17 May '16, 12:18

question was seen: 1,465 times

last updated: 18 May '16, 15:02

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2016 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM