WebFaction
Community site: login faq

Our Rails 3.0.10/passenger 3.0.9 app running on nginx 1.0.6 has been crashing every 4-6 hours with the message:

2011/11/05 04:23:03 [info] 22888#0: *13834319 client closed prematurely connection, so upstream connection is closed too while sending request to upstream, client: 204.93.223.151, server: site.com, request: "HEAD / HTTP/1.0", upstream: "http://127.0.0.1:59548/", host: "www.site.com"

We are seeing periodic non-fatal notices in our error logs as follows:

2011/11/05 03:40:14 [info] 22888#0: *13816710 client 58.61.164.140 closed keepalive connection

We have tried various different nginx configurations, particurly concerning keepalive, tcp_nopush, and gzip to try to prevent this error, but it seems to be getting worse, not better.

Any help you can offer is greatly appreciated; second night in a row I've been woken up by outage alerts.

Many thanks, Alex We've tried a bunch

asked 05 Nov '11, 06:04

alexfirebelly
1111
accept rate: 0%


I don't think this is really a problem with Nginx. The "client closed prematurely connection" message means that a client, for example a browser terminated the request, so Nginx stopped waiting for a response from nginx that runs Rails on the back-end.

Is your application unresponsive or do you get a 502 error while visiting it, after that message shows up in your logs and you report it as crashing?

permanent link

answered 05 Nov '11, 07:03

iliasr ♦♦
2.1k14
accept rate: 35%

The "crash" results in a 502 being passed from nginx; the rails app is stopped as our memory use has dropped from ~130MB to 5.

The error usually occurs when a client is fetching an image crop generated using ImageMagick.

We've also noted that despite caching at the application level and through nginx, crops are generated more often than expected. Image heavy views are causing a significant increase in the resource usage of our stack.

Hope this is helpful; further assistance is very much appreciated.

-Alex

permanent link

answered 05 Nov '11, 11:18

alexfirebelly
1111
accept rate: 0%

I took a look at your server logs and your processes were killed for using too much RAM, 11 times in the last 3 days, at the rate you specify at your initial question. The time stamp on one of log entries is also close enough to the timestamps on your question.

So your application is probably passing your RAM limit while generating those images and our monitor kills it, which eventually generates this error message

You should raise a support ticket the next time you see such behaviour so we could give it a more closer look.

(05 Nov '11, 11:33) iliasr ♦♦

iliasr, thanks so much--we suspected the memory leak in the ImageMagick implementation as being the culprit, but were hoping someone else had experience resolving this with proper caching. The app crashed again ~90 minutes ago, so I'll go ahead and open up that ticket.

Best, Alex

permanent link

answered 05 Nov '11, 11:49

alexfirebelly
1111
accept rate: 0%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×186
×108
×25

question asked: 05 Nov '11, 06:04

question was seen: 12,746 times

last updated: 05 Nov '11, 11:49

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM