WebFaction
Community site: login faq

I have a forum (http://forum.hudsonbelfast.com) written in Django/Python that essentially consists of the following model setup:

  • Post
  • Reply
  • User
  • Feedback ('starring' posts)
  • Private Messages

Reply is specifically structured like this:

class PostReply(models.Model):
    user = models.ForeignKey(User, related_name='replyuser')
    post = models.ForeignKey(Post, related_name='replypost')
    created = models.DateTimeField(auto_now_add=True)
    content = models.TextField()
    ip_address = models.CharField(max_length=255)

The views are fairly simple, viewing posts, viewing profiles, registration/authentication flows, etc. However it is beginning to hit a few limits with regards to memory. I've investigated a bit with mytop, and nothing really stands out as being terribly inefficient. The only thing I have noticed is that the Reply objects are close to 100,000 now, but I would imagine a DB shouldn't have a huge problem with this. So a few questions:

  • Is there a way to have mytop log things so that I can see 24 hours worth of data and investigate exactly the queries that are causing problems?
  • Does anyone have any suggestions for good ways to cache this setup?

To clarify, I understand how to cache when it comes to Django, however I'm finding it tricky to imagine a good way to cache my views and maintain a real-time feel to the forum (which tends to have quite a fast turnover in new posts). Caching views involving posts or replies seems like it would inhibit that real-time action. Ultimately I could start archiving old posts, however this would be a last resort.

asked 27 Sep '12, 06:30

matthewj
112
accept rate: 0%

edited 27 Sep '12, 18:00

Hi,

Have you tried adding an index to your database table? This by itself should help a lot.

(27 Sep '12, 07:30) todork

I tried creating one on (id, post_id) in the model (that I've just now added to my original post), but it didn't really help a noticeable amount.

(27 Sep '12, 17:58) matthewj

If you need to figure out where Django is using memory, then use a memory profiler. Here's one: Profiling Django object size and memory usage with Pympler

permanent link

answered 28 Sep '12, 15:55

seanf
12.2k41836
accept rate: 37%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×909
×337
×28
×8

question asked: 27 Sep '12, 06:30

question was seen: 3,253 times

last updated: 28 Sep '12, 15:55

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM