Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-14-2012, 06:14 PM   #1
squigish
Junior Member
squigish began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jan 2012
Device: Kindle 3, Kindle Touch
Script to update to latest version

I've written a script to upgrade calibre to the latest version, and then restart it. I scheduled it to run every Saturday morning using crontab on a Linux server. The script works fine when I run it manually, but fails when crontab runs it.

Note: I do not have root or sudoer privileges on this server. I am installing calibre inside my home directory. I also don't have physical access to the server, I just log in using VNC or SSH. My VNC display is :2.0

the script:
Code:
#/bin/sh

killall calibre
rm /u1/myusername/opt/* -rf
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/u1/myusername/opt')"

env DISPLAY=:2.0 XAUTHORITY=/u1/myusername/.Xauthority /u1/myusername/opt/calibre/calibre &
When I run the script from the command line over an SSH session, I get the following output:

Code:
Automatically installing to: /u1/myusername/opt
Will download and install calibre-0.8.35-i686.tar.bz2
                    Downloading calibre-0.8.35-i686.tar.bz2
100% [======================================================================]
                                                                                Downloaded 49210770 bytes
Checking downloaded file integrity...
tar: Record size = 8 blocks
Extracting files to /u1/myusername/opt/calibre ...
Extracting application files...
Creating symlinks...
        Symlinking /u1/myusername/opt/calibre/fetch-ebook-metadata to /usr/bin/fetch-ebook-metadata
Traceback (most recent call last):
  File "site.py", line 58, in main
  File "site-packages/calibre/linux.py", line 559, in main
  File "site-packages/calibre/linux.py", line 129, in __init__
OSError: [Errno 13] Permission denied
Run "calibre" to start calibre
And when I log in to VNC, calibre is running.

Every Saturday, I get the following email from crontab
Code:
Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "<string>", line 270, in main
 File "<string>", line 182, in prints
TypeError: encode() argument 1 must be string, not None
env: /u1/myusername/opt/calibre/calibre: No such file or directory
and calibre is not running, and doesn't download the news or email it to me.

I suspect that the problem may have to do the permission denied error I get when running the script manually, because I don't have root privileges. Is there an option to install calibre without symlinking or doing anything else that requires root? I admit I don't really understand all of the python code in the script, I just copied it from the calibre website.

Thanks a bunch!
squigish is offline   Reply With Quote
Old 01-14-2012, 09:42 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That permission denied error is not going to prevent calibre from running. Setup your cron script to log its stoudout and stderr somewhere and see what the real problem is.
kovidgoyal is offline   Reply With Quote
Advert
Old 01-14-2012, 10:33 PM   #3
squigish
Junior Member
squigish began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jan 2012
Device: Kindle 3, Kindle Touch
I redirected both stderr and stdout to files. stdout was empty, and stderr contained precisely the text from the email I received from crontab, in my initial post.

I should add that when crontab runs the script, it successfully deletes everything in /u1/myusername/opt/, but does not download and install the new version. The reason it doesn't start is that the program has been deleted, and not reinstalled.
squigish is offline   Reply With Quote
Old 01-14-2012, 10:37 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The symlinking happens after the download is extracted, as it must, since you cannot symlink to a file that doesn't exist
kovidgoyal is offline   Reply With Quote
Old 01-14-2012, 10:47 PM   #5
squigish
Junior Member
squigish began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jan 2012
Device: Kindle 3, Kindle Touch
OK, so it seems my intuition was wrong about what's causing the problem, but I still have a problem.

When my script is run from an ssh terminal, it deletes the current version of calibre, downloads and installs a new one, and the re-launches calibre.

When my script is run from a cron job, it deletes the current version of calibre, and fails to download and install a new version. Any ideas why?
squigish is offline   Reply With Quote
Advert
Old 01-14-2012, 10:51 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Something in your cron environment is different from the ssh environment. What that might be, I have no idea.
kovidgoyal is offline   Reply With Quote
Old 01-15-2012, 02:33 AM   #7
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Cron has a very limited path. I have had problems running scripts that use programs in places other than /bin and /usr/bin. You might be having the same problem. In your script, set the path to the same as you have when logged in.
chaley is offline   Reply With Quote
Old 01-16-2012, 10:14 PM   #8
squigish
Junior Member
squigish began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jan 2012
Device: Kindle 3, Kindle Touch
Changing the path didn't help. Output is unchanged. As far as I can tell, the only programs executed by the script are python (in /usr/bin) and calibre (entire path specified explicitly).

New version of script with path changes, in case I screwed it up:

Code:
#/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games ; export PATH

killall calibre
rm /u1/myusername/opt/* -rf
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/u1/myusername/opt')"

env DISPLAY=:2.0 XAUTHORITY=/u1/mbbush/.Xauthority /u1/myusername/opt/calibre/calibre &
Anybody have any other ideas?
squigish is offline   Reply With Quote
Old 06-28-2013, 08:21 PM   #9
jsambrook
Junior Member
jsambrook began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Aug 2010
Device: Asus DR-9[05]0
I've been having the same problem with crontab, the nub being:

File "<string>", line 356, in main
File "<string>", line 192, in prints
TypeError: encode() argument 1 must be string, not None

If after:

enc = getattr(f, 'encoding', 'utf-8')

I add:

if enc == None:
enc = "utf-8"

then all is well.

Presumeably this means that the stdout/stdout.buffer is defining an 'encoding' attribute, but it's set to None?
jsambrook is offline   Reply With Quote
Old 09-22-2013, 03:33 AM   #10
oblongau
Member
oblongau began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Sep 2013
Location: Australia
Device: Calibre; Galaxy SII
Similar to jsambrook (#9) I get:

Code:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 356, in main
  File "<string>", line 192, in prints
TypeError: encode() argument 1 must be string, not None
I have the file /home/bob/Documents/scripts/calibre-update.sh which is:
Code:
#!/bin/sh
python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/opt')"
Using
Code:
sudo crontab -e
I create (changing the crontab times as needed, to test):
Code:
50 16 * * 0 /home/bob/Documents/scripts/calibre-update.sh > /home/bob/Documents/scripts/error.log 2>&1 &
(That error.log is where the error I mentioned at the beginning appears.)

I don't understand what jsambrook means by:

Code:
If after:

enc = getattr(f, 'encoding', 'utf-8')

I add:

if enc == None:
enc = "utf-8"
and if that (whatever it is) is a fix.

If I add "sudo" to before "python..." in the script and run it from a terminal where the script lives,
Quote:
./calibre-update.sh
then I am prompted for my password and it runs fine.

What am I doing wrong with my apparently simple method?

Any help appreciated.
oblongau is offline   Reply With Quote
Old 09-29-2013, 10:00 PM   #11
oblongau
Member
oblongau began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Sep 2013
Location: Australia
Device: Calibre; Galaxy SII
Ok, I got this working well enough. I used user level crontab:

Code:
52 11 * * 0 DISPLAY=:0 gnome-terminal -e /home/bob/Documents/scripts/calibre-update.sh
Adjust path and times accordingly.

The script contains:

Code:
#!/bin/sh
sudo python -c "import sys; py3 = sys.version_info[0] > 2; u = __import__('urllib.request' if py3 else 'urllib', fromlist=1); exec(u.urlopen('http://status.calibre-ebook.com/linux_installer').read()); main(install_dir='/opt')"
When it runs, a terminal will appear, with a prompt for password. If you want to be insecure and expose the password, and avoid the prompt, you can use:

Code:
echo "your-system-password"|sudo -S python ...etc...
oblongau is offline   Reply With Quote
Old 09-29-2013, 10:37 PM   #12
MelBr
Zealot
MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.MelBr ought to be getting tired of karma fortunes by now.
 
Posts: 105
Karma: 414068
Join Date: Feb 2013
Device: iPad Pro, Kobo Aura One
Interesting.. but you still have to download 100+MB files. I'm on 4G a lot (when not on campus) so data can be too expensive.

For a while now I've been just updating source code and building the app myself. It's much, much faster to download.

Not sure if there's some delta updates library for python that Kovid could use.…
MelBr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How do I update Calibre to the latest version? nsivakr Calibre 18 02-22-2017 11:07 AM
Metadata update not working since latest update... Rickkins Library Management 4 10-04-2011 03:12 AM
Reader Library Software Not Working Correctly After Update to Latest Version sipig Sony Reader 1 06-02-2010 08:32 AM
script, versions, bbconfig, update 2.0.2 wz2b Kindle Developer's Corner 4 03-20-2009 12:13 PM
PRS-500 What file to change to re-update reader with latest update? pitolee Sony Reader Dev Corner 2 05-24-2007 11:56 PM


All times are GMT -4. The time now is 08:17 PM.


MobileRead.com is a privately owned, operated and funded community.