07-22-2014, 05:12 PM | #1 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2013
Location: Canada
Device: Galaxy Tab 2 10.1, iPad 2
|
ebook-convert doesn't add converted version to library?
I decided to write a quick bash script to convert my ebook formats into epub which Marvin can read (bloody wife is using the Android so I can't use moonreader). When I converted a few files, they weren't automatically added to the Calibre library. Is that intended behaviour, and if so what's the best way to work around it?
As an aside, should I run ebook-polish on newly converted files (e.g. remove unused CSS)? Code:
#! /bin/bash fullfilename=${0##*/} filename="${fullfilename%.*}" logfile="/var/log/scripts/$filename.log" convertpath="/tmp/ebookconvert" echo "Script output will be logged to $logfile" #exec > "$logfile" 2>&1 /opt/scripts/scriptheader.sh "eBook Convert" test -e "$convertpath" || mkdir "$convertpath" if ! [[ -f "/usr/bin/ebook-convert" ]]; then echo "ebook-convert not found. Aborting." exit 1 fi if [[ -z $1 ]]; then echo "No argument supplied. Please enter the path you which to convert." exit 1 elif ! [[ -d "$1" ]]; then echo "Argument passed is not a directory." exit 1 fi echo "Scanning $1 for ebooks files..." # List the types of files in the directory echo "eBook file types detected:" find "$1" -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u # Find the media types that will be converted. # Useful for verifying after the fact. echo "Files will be copied to '$convertpath' before conversion." find "$1" -type f -name "*.mobi" -o -type f -name "*.lit" -o -type f -name "*.pdf" > "$convertpath/ebooklist.txt" echo "Converting MOBI Files." find "$1" -type f -name "*.mobi" | while read line; do # Check to see if we already have the format. if ! [[ -f "${line%.mobi}.epub" ]]; then /usr/bin/ebook-convert "$line" "${line%.mobi}.epub"; if [ $? -ne 0 ]; then echo "Error converting $line."; fi else echo "ePub for $line exists already, no conversion needed."; fi done |
07-22-2014, 05:29 PM | #2 |
Wizard
Posts: 1,785
Karma: 13412766
Join Date: Nov 2010
Device: Kobo Clara HD, iPad Pro 10", iPhone 15 Pro
|
IME it does not. I think that it's just the tool that the GUI application calls to perform the translation step, but the GUI is responsible for all of the "bookkeeping" related to the db.
I personally like it that way as I don't use any of the features of Calibre beyond that one tool -- in fact, I wish I could download the backend CLI tools separate from the GUI. |
07-22-2014, 10:09 PM | #3 |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
You can use
Code:
calibredb add |
07-22-2014, 11:44 PM | #4 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2013
Location: Canada
Device: Galaxy Tab 2 10.1, iPad 2
|
I had tried that already actually. If it's a net new book that will work, but if it's a conversion of an existing book format it didn't work. There's an option to add a book format with the calibredb program, however you need to know the ID of the book, which you only can get via the list command.
|
07-23-2014, 12:04 AM | #5 |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Are you converting the books from the calibre directory? If so, you can extract the id from the folder location. Or parse the metadata.opf
It is also possible to use calibre-debug to run calibre's code to trigger a library convert, but that is above my paygrade -- I don't know how. |
07-23-2014, 12:08 AM | #6 |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Just curious -- is there a reason you use find and while read line, instead of "for file in *.mobi"
And instead of Code:
fullfilename=${0##*/} filename="${fullfilename%.*}" Code:
filename="$(basename $0)" Code:
test -e "$convertpath" || mkdir "$convertpath" Code:
mkdir -p "$convertpath" |
07-23-2014, 12:10 AM | #7 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
You're doing it wrong
You want to start with calibredb list to get a list of all books and their ids, calibredb list -f formats Then iterate over the books, converting them, sending the converted output to a temp file ad add the temp file like this calibredb add_format book_id tmpfile |
07-23-2014, 12:11 AM | #8 |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Code:
#! /bin/bash filename="$(basename $0)" logfile="/var/log/scripts/$filename.log" convertpath="/tmp/ebookconvert" echo "Script output will be logged to $logfile" #exec > "$logfile" 2>&1 /opt/scripts/scriptheader.sh "eBook Convert" mkdir -p "$convertpath" if ! [[ -f "/usr/bin/ebook-convert" ]]; then echo "ebook-convert not found. Aborting." exit 1 fi if [[ -z $1 ]]; then echo "No argument supplied. Please enter the path you which to convert." exit 1 elif ! [[ -d "$1" ]]; then echo "Argument passed is not a directory." exit 1 fi # Entering the directory cd $1 echo "Scanning $1 for ebooks files..." # List the types of files in the directory, with frequency echo "eBook file types detected:" find . -type f | sed -rn 's|.*/[^/]+\.([^/.]+)$|\1|p' | sort | uniq -c # Find the media types that will be converted. # Useful for verifying after the fact. echo "Files will be copied to '$convertpath' before conversion." find . -type f -name "*.mobi" -o -type f -name "*.lit" -o -type f -name "*.pdf" > "$convertpath/ebooklist.txt" echo "Converting MOBI Files." # IFS change for file names with spaces in them SAVEIFS=$IFS IFS=$(echo -en "\n\b") for file in *.mobi; do basefile=$(basename "$file") # Check to see if we already have the format. if ! [[ -f "$basefile.epub" ]]; then /usr/bin/ebook-convert "$file" "$basefile.epub"; if [ $? -ne 0 ]; then echo "Error converting $file."; fi /usr/bin/calibredb add $file else echo "ePub for $file exists already, no conversion needed."; fi done # restore default IFS IFS=$SAVEIFS |
07-23-2014, 12:42 AM | #9 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2013
Location: Canada
Device: Galaxy Tab 2 10.1, iPad 2
|
|
07-23-2014, 12:58 AM | #10 | |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Quote:
A general recommendation seems to be, use as few commands to get a job done. "for" can do quite a few interesting things on its own. It took me a while to catch on to basename. Happy scripting! ... trying to figure out how to parse calibredb list output... |
|
07-23-2014, 01:47 AM | #11 | |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Quote:
... something like this... Code:
#!/bin/bash # Search for which books we want. Find books that aren't in EPUB yet. calibre-search='not formats:"=EPUB"' regex="(\d+) \[([^\]]*)\]" # Search calibredb, iterate over each result. calibredb list -s "$calibre-search" -f formats | while read line; do # Extract info from lines into bash_rematch [[ "$line" =~ "$regex" ]] ebook-convert "${BASH_REMATCH[2]}" "/tmp/converted.epub" calibredb add_format "${BASH_REMATCH[1]}" "/tmp/converted.epub" done |
|
07-23-2014, 02:01 AM | #12 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Better, but it wont handle books that have multiple formats, for which there will be multiple entries inside the [] brackets.
And now that I think about it, there is actually a bug in calibredb list. It should not be separating multiple path by commas, as commas can occur in filenames. Now fixed: https://github.com/kovidgoyal/calibr...4e95f15101ec6c |
07-23-2014, 03:35 AM | #13 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Since I was here in any case, I decided to add a --for-machine option to calibredb list that makes it easier to parse the output.
And here's a one liner (in python because I hate sh) that demonstrates how to get a list of all ids and formats for ever id in the library Code:
calibredb list --for-machine -f formats | python2 -c "import sys; print '\n\n'.join('id:%s formats:%r' % (r.partition(chr(29))[0], r.partition(chr(29))[-1].split(chr(28))) for r in sys.stdin.read().split(chr(30)) if r.strip())" Code:
--for-machine Generate output that is more suitable for machine parsing. Book entries are separates using the ASCII Record Separator character (30), individual fields in an entry using the Group Separator (29) and individual filenames using the File Separator (28) |
07-23-2014, 08:28 AM | #14 |
Ex-Helpdesk Junkie
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
Thanks! I will look into using this...
|
07-23-2014, 02:32 PM | #15 | |||
Member
Posts: 21
Karma: 10
Join Date: Sep 2013
Location: Canada
Device: Galaxy Tab 2 10.1, iPad 2
|
Quote:
Quote:
Quote:
|
|||
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
ebook-meta command doesn't add cover to ePub and mobi | EbokJunkie | Conversion | 8 | 04-28-2014 03:13 AM |
Version 8.41 doesn't add book | killa | Calibre | 0 | 02-28-2012 11:53 AM |
ebook-convert vs windows version | zhoujianfu | Calibre | 12 | 05-21-2010 01:08 PM |
Calibredb add after ebook-convert fails | spullar | Calibre | 5 | 03-04-2010 01:14 PM |
Why doesn't eBook library 3.0 work? | mgmueller | Sony Reader | 30 | 09-20-2009 12:38 PM |