login community faq

I have a forum (http://forum.hudsonbelfast.com) written in Django/Python that essentially consists of the following model setup:

  • Post
  • Reply
  • User
  • Feedback ('starring' posts)
  • Private Messages

Reply is specifically structured like this:

1
2
3
4
5
6
class PostReply(models.Model):
    user = models.ForeignKey(User, related_name='replyuser')
    post = models.ForeignKey(Post, related_name='replypost')
    created = models.DateTimeField(auto_now_add=True)
    content = models.TextField()
    ip_address = models.CharField(max_length=255)

The views are fairly simple, viewing posts, viewing profiles, registration/authentication flows, etc. However it is beginning to hit a few limits with regards to memory. I've investigated a bit with mytop, and nothing really stands out as being terribly inefficient. The only thing I have noticed is that the Reply objects are close to 100,000 now, but I would imagine a DB shouldn't have a huge problem with this. So a few questions:

  • Is there a way to have mytop log things so that I can see 24 hours worth of data and investigate exactly the queries that are causing problems?
  • Does anyone have any suggestions for good ways to cache this setup?

To clarify, I understand how to cache when it comes to Django, however I'm finding it tricky to imagine a good way to cache my views and maintain a real-time feel to the forum (which tends to have quite a fast turnover in new posts). Caching views involving posts or replies seems like it would inhibit that real-time action. Ultimately I could start archiving old posts, however this would be a last resort.

asked Sep 27 '12 at 06:30

matthewj's gravatar image

matthewj
11

edited Sep 27 '12 at 18:00

Hi,

Have you tried adding an index to your database table? This by itself should help a lot.

(Sep 27 '12 at 07:30) todork todork's gravatar image

I tried creating one on (id, post_id) in the model (that I've just now added to my original post), but it didn't really help a noticeable amount.

(Sep 27 '12 at 17:58) matthewj matthewj's gravatar image

If you need to figure out where Django is using memory, then use a memory profiler. Here's one: Profiling Django object size and memory usage with Pympler

answered Sep 28 '12 at 15:55

seanf's gravatar image

seanf ♦♦
56441220

Your answer
If you have an answer to the above question, then use the form below. Otherwise, use the appropriate 'add new comment' button above to post your feedback.
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Tags:

×643
×229
×22
×7

Asked: Sep 27 '12 at 06:30

Seen: 347 times

Last updated: Sep 28 '12 at 15:55

Plans & prices    Sign up    Why WebFaction?    Contact us    Affiliate program    Support    Legal    Jobs    Blog    Control panel login
Powered by OSQA
© Copyright 2003-2012 Swarma Limited - WebFaction is a service of Swarma Limited