Script to update to latest version

[squigish]

I've written a script to upgrade calibre to the latest version, and then restart it. I scheduled it to run every Saturday morning using crontab on a Linux server. The script works fine when I run it manually, but fails when crontab runs it.

Note: I do not have root or sudoer privileges on this server. I am installing calibre inside my home directory. I also don't have physical access to the server, I just log in using VNC or SSH. My VNC display is :2.0

the script:

#/bin/sh

killall calibre
rm /u1/myusername/opt/* -rf
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/u1/myusername/opt')"

env DISPLAY=:2.0 XAUTHORITY=/u1/myusername/.Xauthority /u1/myusername/opt/calibre/calibre &

When I run the script from the command line over an SSH session, I get the following output:

Automatically installing to: /u1/myusername/opt
Will download and install calibre-0.8.35-i686.tar.bz2
                    Downloading calibre-0.8.35-i686.tar.bz2
100% [======================================================================]
                                                                                Downloaded 49210770 bytes
Checking downloaded file integrity...
tar: Record size = 8 blocks
Extracting files to /u1/myusername/opt/calibre ...
Extracting application files...
Creating symlinks...
        Symlinking /u1/myusername/opt/calibre/fetch-ebook-metadata to /usr/bin/fetch-ebook-metadata
Traceback (most recent call last):
  File "site.py", line 58, in main
  File "site-packages/calibre/linux.py", line 559, in main
  File "site-packages/calibre/linux.py", line 129, in __init__
OSError: [Errno 13] Permission denied
Run "calibre" to start calibre

And when I log in to VNC, calibre is running.

Every Saturday, I get the following email from crontab

Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "<string>", line 270, in main
 File "<string>", line 182, in prints
TypeError: encode() argument 1 must be string, not None
env: /u1/myusername/opt/calibre/calibre: No such file or directory

and calibre is not running, and doesn't download the news or email it to me.

I suspect that the problem may have to do the permission denied error I get when running the script manually, because I don't have root privileges. Is there an option to install calibre without symlinking or doing anything else that requires root? I admit I don't really understand all of the python code in the script, I just copied it from the calibre website.

Thanks a bunch!

[kovidgoyal]

That permission denied error is not going to prevent calibre from running. Setup your cron script to log its stoudout and stderr somewhere and see what the real problem is.

[squigish]

I redirected both stderr and stdout to files. stdout was empty, and stderr contained precisely the text from the email I received from crontab, in my initial post.

I should add that when crontab runs the script, it successfully deletes everything in /u1/myusername/opt/, but does not download and install the new version. The reason it doesn't start is that the program has been deleted, and not reinstalled.

[kovidgoyal]

The symlinking happens after the download is extracted, as it must, since you cannot symlink to a file that doesn't exist

[squigish]

OK, so it seems my intuition was wrong about what's causing the problem, but I still have a problem.

When my script is run from an ssh terminal, it deletes the current version of calibre, downloads and installs a new one, and the re-launches calibre.

When my script is run from a cron job, it deletes the current version of calibre, and fails to download and install a new version. Any ideas why?

[kovidgoyal]

Something in your cron environment is different from the ssh environment. What that might be, I have no idea.

[chaley]

Cron has a very limited path. I have had problems running scripts that use programs in places other than /bin and /usr/bin. You might be having the same problem. In your script, set the path to the same as you have when logged in.

[squigish]

Changing the path didn't help. Output is unchanged. As far as I can tell, the only programs executed by the script are python (in /usr/bin) and calibre (entire path specified explicitly).

New version of script with path changes, in case I screwed it up:

#/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games ; export PATH

killall calibre
rm /u1/myusername/opt/* -rf
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/u1/myusername/opt')"

env DISPLAY=:2.0 XAUTHORITY=/u1/mbbush/.Xauthority /u1/myusername/opt/calibre/calibre &

Anybody have any other ideas?

[jsambrook]

I've been having the same problem with crontab, the nub being:

File "<string>", line 356, in main
File "<string>", line 192, in prints
TypeError: encode() argument 1 must be string, not None

If after:

enc = getattr(f, 'encoding', 'utf-8')

I add:

if enc == None:
enc = "utf-8"

then all is well.

Presumeably this means that the stdout/stdout.buffer is defining an 'encoding' attribute, but it's set to None?

[oblongau]

Similar to jsambrook (#9) I get:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 356, in main
  File "<string>", line 192, in prints
TypeError: encode() argument 1 must be string, not None

I have the file /home/bob/Documents/scripts/calibre-update.sh which is:

#!/bin/sh
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/opt')"

Using

sudo crontab -e

I create (changing the crontab times as needed, to test):

50 16 * * 0 /home/bob/Documents/scripts/calibre-update.sh > /home/bob/Documents/scripts/error.log 2>&1 &

(That error.log is where the error I mentioned at the beginning appears.)

I don't understand what jsambrook means by:

If after:

enc = getattr(f, 'encoding', 'utf-8')

I add:

if enc == None:
enc = "utf-8"

and if that (whatever it is) is a fix.

If I add "sudo" to before "python..." in the script and run it from a terminal where the script lives,

./calibre-update.sh

then I am prompted for my password and it runs fine.

What am I doing wrong with my apparently simple method?

Any help appreciated.

[oblongau]

Ok, I got this working well enough. I used user level crontab:

52 11 * * 0 DISPLAY=:0 gnome-terminal -e /home/bob/Documents/scripts/calibre-update.sh

Adjust path and times accordingly.

The script contains:

#!/bin/sh
sudo python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/opt')"

When it runs, a terminal will appear, with a prompt for password. If you want to be insecure and expose the password, and avoid the prompt, you can use:

echo "your-system-password"|sudo -S python ...etc...

[MelBr]

Interesting.. but you still have to download 100+MB files. I'm on 4G a lot (when not on campus) so data can be too expensive.

For a while now I've been just updating source code and building the app myself. It's much, much faster to download.

Not sure if there's some delta updates library for python that Kovid could use.…