WebFaction
Community site: login faq
0
1

I've recently run into a problem with my Django project and memory usage here at WebFaction.

Here is the two processes running in memory for this project:

  30396  4-20:20:00 13486 
  30404  4-20:20:00 13487

After the view is run one of the processes will increase substantially:

  69720  4-20:20:22 13486 
  30404  4-20:20:22 13487

As you can see the first process more than doubled in memory usage! As this function will be used often I need to figure out what is happening. I believe I've narrowed it down to the following view (it's a 3 step process of upload an image, add details, crop a thumbnail).

Here is the view below. It gets a photo object, loads the image from the file, gets the box coordinates that the user has submitted and then creates a 200,200 sized image. This newly created image is written back to the disk with a .thumbnail in the filename and the photo object is saved.

@login_required
def upload3(request, photo_pk):
    photo = get_object_or_404(Photo, pk=photo_pk, user=request.user)
    if request.method == "POST":
        form = upload3Form(request.POST)
        if form.is_valid():
            im = Image.open(photo.image.path)
            try:
                box =(form.cleaned_data['x1'],form.cleaned_data['y1'],form.cleaned_data['x2'],form.cleaned_data['y2'])
            except:
                box = ('0','0','1000','1000')
            cropped = im.crop(box)
            cropped.thumbnail((200,200),Image.ANTIALIAS)
            result = os.path.splitext(photo.image.path)
            cropped.save(result[0] + '.thumbnail' + result[1])
            photo.status = 3
            photo.save()

I have be able to duplicate that this view is when the memory spike happens and not in the other views. It's important that I figure out what is happening because of how much memory this eats up on my account (and then causes WebFaction to have to kill the process). I've also asked this question on StackOverflow with no bites yet.

Any ideas of what I may be doing wrong would be greatly appreciated.

asked 28 Feb '12, 10:07

KennyFern
111
accept rate: 0%


I'm not sure this should be necessary, but have you tried destroying the photo object after you save it, eg "del photo"?

Failing that, maybe you can use something like Pympler to get a better idea of exactly what parts of that view are using the most memory.

permanent link

answered 28 Feb '12, 16:53

seanf
12.2k42136
accept rate: 37%

No I haven't tried that. I think I looked into it a few weeks ago and felt from what I was reading that PIL doesn't require you to and would do necessary clean-up automatically. And if it were the case of just the new image in memory I would expect a much smaller jump in memory usage, not a doubling of memory.

I will look into Pympler and see if I can't find anything.

(29 Feb '12, 13:08) KennyFern
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×910
×12

question asked: 28 Feb '12, 10:07

question was seen: 3,380 times

last updated: 29 Feb '12, 13:08

                              
WEBFACTION
REACH US
SUPPORT
LEGAL
© COPYRIGHT 2003-2021 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM