Another one question about memory consumption and Django

Hello (again). I have read all the posts (here in community and in stackoverflow) about memory Django's consumption. I just want to ask one thing:

Every time I hit refresh in my Django Admin page I get +1MB of RSS memory. Is this OK and healthy?

Here is my first ps output after an Apache restart:

ps -u $USER -o rss,pid,command

RSS PID COMMAND 10212 5638 /usr/bin/memcached -d -m 64 -s /home/$USER/memcached.sock -P /home/$USER/memcached.pid 50540 7734 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 51704 7735 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 2620 7736 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 2532 10045 -bash 124680 12794 java -jar -Xmx16M -Xmn4M -Xss256K start.jar --daemon 3176 21117 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start 1608 24392 ps -u $USER -o rss,pid,command

Now if I hit F5 in the admin page all the processes stay the same except #7734 which increases by +1MB:

51724 7734 /home/$USER/webapps/my_app/apache2/bin/httpd.worker -f /home/$USER/webapps/my_app/apache2/conf/httpd.conf -k start

P.S.: My http.conf file is with the defaults. Nothing changed in there and all recommendations about reducing memory consumption have been applied (as per webfaction's documentation).

EDIT: Today I am going to do a little memory debugging with some tools I have read about and post here the results and (maybe?) the solution! Hope everything will work...

django memory apache2

asked 17 May '16, 12:18

nik
130●1●9
accept rate: 16%

edited 17 May '16, 13:12

3 Answers:

active answers oldest answers newest answers popular answers

If your httpd memory usage is going up every time you reload your admin page, then there is probably a memory leak in one of your Django project's apps. Debugging with Pympler will help you identify it.

You can also use the max-requests parameter in your WSGIDaemonProcess directive to configure Apache to recycle your worker processes after they've serviced N requests - this will keep their memory usage from growing endlessly.

Hope that helps!

permanent link

answered 18 May '16, 00:39

seanf
12.1k●2●16●35
accept rate: 37%

OK.

Problem #1: Pympler runs on top of django-debug-toolbar which must have Debub = False. Solved by running my app locally.

Problem #2: After running Pympler locally, the Memory panel was not very helpful! On the right hand side it gives me the Megabytes that was reserved extra (i.e 80MB +8MB) and when I click on it, it gives me a summary like the below:

Class Number of instances Show sparklines Total size myApp1.models.MyModel1 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 9, 8, 7, 6, 5, 4 9.34 KB myApp1.models.MyModel2 1, 2, 3, 4, 5, 6, 5, 4, 3, 2, 1, 0, 1, 2, 3, 4, 5 78.73 KB myApp1.models.MyModel3 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 80.20 KB django.contrib.auth.models.User 1 2.92 KB django.template.base.Template 1, 2, 3, 4, 5, 6, 7, 6, 7, 8, 9, 10, 11, 10, 11, 10, 11, 12, 13, 14, 15, 14, 15, 14, 15, 14, 15, 14, 15, 14, 15, 14, 13, 12, 11, 12, 13, 14, 15, 16, 15, 16, 15, 14, 13, 12, 13, 14, 15, 16, 17, 16, 15, 16, 15, 14, 13, 12, 13, 12, 13, 14, 15, 14, 15, 14, 15, 14, 13, 12, 11, 12, 13, 14, 13, 14, 15, 16, 15, 14, 13, 12 123.03 KB django.template.context.Context 1, 2, 1 3.40 MB

What useful info can I extract from this summary?

Apart from the fact that the total size of the above sizes does not much the size that in shown in parenthesis on the right hand side.

I really want to use the maximum-requests directive but the memory leak will still be there and this will not be a good practice IMO. Should I ask this in stackoverflow?

permanent link

answered 18 May '16, 11:21

nik
130●1●9
accept rate: 16%

Unfortunately, I was not able to figure out a solution to my problem, using pympler. In my above post (not the first one) I was wrong because I had set DEBUG = True (I was running my app locally, which is wrong) and the memory should increase. So, reject my previous post.

After I had set my app to run locally with DEBUG = False then the pympler panel inside the django-debug-toolbar panel did not work. Always getting a KeyError < django.template.base.Template > error and didn't know what to do.

I think I will stay with this memory consumption and use maximum-requests inside WSGIDaemonProcess.

If I have any other news, I will post right here.

Best regards to everyone

permanent link

answered 18 May '16, 15:02

nik
130●1●9
accept rate: 16%

Your answer

toggle preview

community wiki:

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

*italic* or _italic_
**bold** or __bold__
link:[text](http://url.com/ "title")
image?![alt text](/path/img.jpg "title")
numbered list: 1. Foo 2. Bar
to add a line break simply add two spaces to where you would like the new line to be.
basic HTML tags are also supported

learn more about Markdown

Question tags:

django ×898
memory ×82
apache2 ×25

question asked: 17 May '16, 12:18

question was seen: 1,465 times

last updated: 18 May '16, 15:02

Another one question about memory consumption and Django

Follow this question

Related questions