WebFaction
Community site: login faq

I'm having a problem running a python script (iceCalendar.py) from cron. the python script uses the BeautifulSoup module which i've installed onto py2.7 using easy_install-2.7. here are the contents of my crontab:

PATH=/home/myName/lib/python2.7:$PATH

HOME=/home/myName/pythonscripts

26 11 * * 0-5 /usr/local/bin/python2.7 iceCalendar.py

when the cronjob runs, i get an error ImportError No module named BeautifulSoup. However, when i run this from the command from the ssh it runs without a glitch...

$ python2.7 iceCalendar.py

I've added the top PATH= line to my crontab per a recommendation from another post but that's not actually doing anything it seems... to further debug this, i've added the following to the top line of my script so that it shows me the path the script sees but even though i've added the PATH line to my crontab /home/myName/lib/python2.7, that directory is not listed when the following print from my script ran by cron.

import sys

print sys.path

Please help! Thanks in advance!!

asked 05 Sep '11, 11:55

dgre420
3135
accept rate: 0%


PATH does not tell Python where to find libraries - PYTHONPATH does.

Try this for your cron job:

26 11 * * 0-5 PYTHONPATH=/home/myName/lib/python2.7 /usr/local/bin/python2.7 iceCalendar.py

Hope that helps!

permanent link

answered 05 Sep '11, 13:24

seanf
12.2k41836
accept rate: 37%

thanks for the clarification on PYTHONPATH and code snipet. It definitely served to add myName/lib/python2.7 to the path but somehow BeautifulSoup is still giving an ImportError. This still only happening when iceCalendar.py is ran by cron. I've checked easy-install.ph and i do find the BeautifulSoup egg listed there as well as other third party eggs like pysftp. i'll test those eggs to see if they work in cron. they do work running them from ssh. Not sure what else i can try.

(05 Sep '11, 15:19) dgre420

after i tested the pysftp module in cron, and i was unable to import that module, i checked out the path in a manual run in ssh to find that the egg must be listed as a path. so adding the following line fixed it:

PYTHONPATH=/home/myName/lib/python2.7/BeautifulSoup-3.2-py2.7.egg

I'm new to using eggs. I wonder if anyone knows a way to have the manual python run PATH into cron so as to avoid having to list each egg manually, somehow where:

$ python

import sys

print sys.path

sys.path could be imported into cron.

permanent link

answered 05 Sep '11, 15:42

dgre420
3135
accept rate: 0%

You can do this in a Python script:

from site import addsitedir
addsitedir('/home/myName/lib/python2.7')

That said, I just did some testing on your server, and as far as I can tell, you shouldn't need to do any of this.

For example. I did this:

mkdir -p ~/lib/python2.7
easy_install-2.7 BeautifulSoup

I could then run the following with no errors:

/usr/local/bin/python2.7 -c "import BeautifulSoup; print BeautifulSsoup"

And the following cron job did not log any errors:

* * * * * /usr/local/bin/python2.7 -c "import BeautifulSoup; print BeautifulSoup" > /dev/null 2>$HOME/cron.log

It sounds to me like something in your script is nuking your Python search path when it is run via cron, but I'm not sure what would cause that.

(05 Sep '11, 16:01) seanf

thanks for the heads up on the research and the additional help. it is very strange. the cron runs correctly after having the egg added manually to the python search path inside the crontab itself, and also, more strange is that the script fails upon reading the very first line if the beautifulsoup egg is not explicitly added to the crontab's python search path, i.e. line #1 is import BeautifulSoup. So not sure what else inside the script could be messing with the python search pad :(

do you think that me setting the HOME of my crontab to:

HOME=/home/myName/pythonscripts could be causing the issue with the cronjob's python search paths?

(05 Sep '11, 23:29) dgre420

It is possible HOME is a environment variable that linux uses for many things. The way to check would be to set it to /home/you/ and create a new environment variable for your scripts directory.

(05 Sep '11, 23:45) johns

johns, thanks so much. you're totally right. first, i had:

HOME=/home/myName/pythonscripts

at the top of the crontab. and this did not work:

          • /usr/local/bin/python2.7 -c "import BeautifulSoup; print BeautifulSoup"

Then, i printed sys.path using:

          • /usr/local/bin/python2.7 -c "import sys; print sys.path"

and it did not print the third party eggs that i've installed on the server. then, i removed the line from cron and the eggs finally appeared on cron's print out and therefore, the import worked just fine.

i'm new to linux and hence to cron. i'm confused as to how have a script use it's own directory as the working directory besides hardcoding it inside the python script. for example, i have a script called fileImport.py that lives inside:

/home/myName/pythonscripts

the script downloads a file, and i expect it to download into that same directory. but instead, when i run the cron, it downloads into:

/home/myName

Does anyone know a way to change the script's working directory from within cron without changing cron's HOME environment variable? I tried this:

          • HOME=/home/myName/pythonscripts /usr/local/bin/python2.7 -c "import os; print os.getcwd()"

i still get:

/home/myName

thanks!!

(11 Sep '11, 21:55) dgre420
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×337
×81
×50
×6

question asked: 05 Sep '11, 11:55

question was seen: 10,759 times

last updated: 11 Sep '11, 22:21

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM