WebFaction
Community site: login faq

I periodically need to launch a process to do several update tasks. Things start with an rsync from an external server. At that point I can either detect the change (file timestamps) in a cron script, or I can have the external server do a simple wget http://example.com/do_update. Depending on how much has changed, the tasks will take 3-15 minutes. When done, the webserver (Django under Apache) needs to do a cache-clear and restart.

If this was happening with great regularity I would do it with cron and be done with it. But sometimes it might happen 4 or 5 times in one day, and then not happen again for a few weeks. It feels unclean to be polling the status of a file with cron when I could just as easily trigger things with a GET.

I know I can have things launched by cron run for an arbitrary amount of time, but I have a feeling there is some sort of timeout that will trip me up if I go via the server.

Thoughts?

asked 23 Jul '13, 18:40

hedronist
27212
accept rate: 0%


The only timeout that might be an issue is if you make an HTTP request that waits for the task to be completed before returning a response. In that case you will hit the frontend proxy timeout of 60 seconds. If this is what you are trying to do then the only option will be to run your own web server instance and connect to it directly[1].

Another option might be to have the progress recorded somewhere, which is then accessible via another request (or via the same URL but triggered with GET parameters). Essentially short-polling. This way, none of your requests will be waiting for long-running tasks to complete at any given time.

Hope this helps, good luck!

[1] If you create a "Custom app (listening on port)" then we can open the port for that application for you externally, so you can connect directly to your app. Just open a support ticket in your control panel and we can do this for you.

permanent link

answered 23 Jul '13, 19:26

waynek
4254
accept rate: 27%

Sigh. I just realized that since I'm already doing the rsync with ssh and shared keys, all I have to do is rsh example.com do_update. Clearly, all I needed was a good night's sleep and a gallon of coffee.

(24 Jul '13, 11:10) hedronist
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×37
×17
×1

question asked: 23 Jul '13, 18:40

question was seen: 1,349 times

last updated: 24 Jul '13, 11:10

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM