Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Conversion

Notices

Reply
 
Thread Tools Search this Thread
Old 07-08-2011, 11:00 AM   #1
jmeb
Junior Member
jmeb began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2011
Device: Kindle DX
Batch Converting Ebooks with Calibre and Python

Mornin' all

As part of my venturing into Python I've created a python script that runs on my server to batch convert between different ebook formats using calibre's ebook-convert.

The script looks up all files matching one extension (e.g. .epub) and checks for the same filename with another extension (e.g. .mobi). It there isn't a matching file, it will run ebook-convert to create it.

Some things it includes:
* Handles filenames with spaces
* Takes a couple of different command line arguments
- "--have {epub, mobi, lit, html}": tells it what ebook format you have to convert from
- "--want {epub, mobi, lit, html}": tells it what ebook format you want to convert to
- "-s {/PATH/TO/DIR}": source directory for the ebooks you have
-" -d {/PATH/TO/DIR}": destination directory to check for existing ebooks, and drop conversions into.
* If no commands are supplied it will try to convert epub to mobi in the current directory.

For example:
* "python mbmk.py" : will create an .mobi for every .epub without a .mobi counterpart
* "python mbmk.py --have epub --want html -s ../epubs -d ../htmls" will grab a list of epubs from the ../epub directory, and create an html version in the ../htmls directory if needed.
* "python mbmk.py --have mobi --want lit -s ../mobis" creates .lit versions of mobi files in the ../mobi directory
Its written for a linux box, but should work fine on Mac OSX. Windows I doubt....

Thanks for all the work on Calibre. If anyone has suggestions on improving the script I'd be happy to have them.

The script can be found here: http://pastebin.com/XdPZUdpg
jmeb is offline   Reply With Quote
Old 07-08-2011, 11:59 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Just to note: You can run your script without the need for a separate python, by just doing:

calibre-debug -e filename.py
kovidgoyal is offline   Reply With Quote
Advert
Old 08-07-2012, 10:27 PM   #3
chadberg
Junior Member
chadberg began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Aug 2011
Device: iPad
Great! Now to modify it to automatically read in any supported format, and output my two preferred options....

Thank you!
chadberg is offline   Reply With Quote
Old 11-18-2013, 09:44 AM   #4
naxman
Junior Member
naxman began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Nov 2013
Device: ipad
I an using Windows Vista - I love Calibre, I a still looking for a way to convert my many mobi files to epub as I have moved from a Kindle to an iPad. I want to do this within folders without having to add them all to my Calibre library as the library becomes too large to easily deal with and I won't need some of the documents for some time/ I would like to convert them in the background if possible. Is there a way to do this?
naxman is offline   Reply With Quote
Old 11-18-2013, 09:49 AM   #5
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by naxman View Post
I an using Windows Vista - I love Calibre, I a still looking for a way to convert my many mobi files to epub as I have moved from a Kindle to an iPad. I want to do this within folders without having to add them all to my Calibre library as the library becomes too large to easily deal with and I won't need some of the documents for some time/ I would like to convert them in the background if possible. Is there a way to do this?
Too large to deal with?
Tags and Virtual Libraries are the basic tools of Calibre that should male ANY size Library manageable . At least you don't need to write un-flawed python code to do the job
\
theducks is online now   Reply With Quote
Advert
Old 11-18-2013, 10:22 AM   #6
naxman
Junior Member
naxman began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Nov 2013
Device: ipad
I see I can do it I suppose but I would lose my file structure as everything is imported into the Calibre library, which was really what I was trying to avoid.
naxman is offline   Reply With Quote
Old 11-18-2013, 11:25 AM   #7
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by naxman View Post
I see I can do it I suppose but I would lose my file structure as everything is imported into the Calibre library, which was really what I was trying to avoid.
There is no reason to keep needing YOUR file structure.
Calibre can search/select and DELIVER by far more refined criteria than any STRUCTURE based method can supply

Tags, Author, Publisher,Title(words),Series (sometimes GIGO),Published Date and any other field you might conceive (and populate)
Calibre is a Database. You get to databasey type things that can not be done with a file system only search.
theducks is online now   Reply With Quote
Old 11-18-2013, 12:12 PM   #8
fidvo
Addict
fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.fidvo ought to be getting tired of karma fortunes by now.
 
Posts: 296
Karma: 1599870
Join Date: Jun 2012
Device: none
Quote:
Originally Posted by theducks View Post
There is no reason to keep needing YOUR file structure.
Calibre can search/select and DELIVER by far more refined criteria than any STRUCTURE based method can supply

Tags, Author, Publisher,Title(words),Series (sometimes GIGO),Published Date and any other field you might conceive (and populate)
Calibre is a Database. You get to databasey type things that can not be done with a file system only search.
Although I agree with you, there are a lot of people who are too stubborn to give up their hierarchical file system for a database-driven system. I know, because I was that way myself for the longest time, so I can certainly understand where naxman is coming from.

However, there is a way to compromise. You can use Calibre to convert and still keep your hierarchical file system. It takes some work up front, but you only have to do this once.

The key is to set up a save-to-disk template to re-create your file structure. This may involve setting up custom columns to store the information needed about the folder names, etc.

Once you have everything set up correctly, then to convert a book you would import it into Calibre, populate the columns, convert, and save to disk. If you don't want to maintain a library, delete the book from Calibre afterward.

I ran Calibre like this for a year before I decided to experiment with its library management features, and instantly got hooked.
fidvo is offline   Reply With Quote
Old 05-11-2016, 12:35 AM   #9
Sasuraigar
Junior Member
Sasuraigar began at the beginning.
 
Posts: 1
Karma: 10
Join Date: May 2016
Device: Kindle
My roommate and I modified the original code in the pastebin to be shorter, and to account for spaces in directories and filenames. Enjoy.


Code:
import os
import subprocess
import sys
import datetime
import argparse

def abs_file_paths(directory):
	for dirpath,_,filenames in os.walk(directory):
		for f in filenames:
			yield os.path.abspath(os.path.join(dirpath, f))



def main():
	#list of available formats to convert
	formats = [ 'epub', 'mobi', 'html', 'lit', 'rtf', 'txt', 'odt', 'pdf' ]
	
	#parse command line arguments
	ap = argparse.ArgumentParser(description='Batch convert between ebook formats.')
	ap.add_argument('--have', action='store', default='epub', choices=formats)
	ap.add_argument('--want', action='store', default='mobi', choices=formats)
	ap.add_argument('-s', '--src', default=os.getcwd())
	args = ap.parse_args()

	#Switch dirs to absolute paths
	src = os.path.abspath(args.src)

	#add period and string-ify things to be safe
	have = "." + str(args.have)
	want = "." + str(args.want)

	files = abs_file_paths(src)
	for f in files:
		if f.endswith(have):
			outfile = f[:-len(have)] + want
			if not os.path.exists(outfile):
				print("converting \ninfile=%s\noutfile=%s" % (f, outfile))
				subprocess.call(['ebook-convert', f, outfile])
			

if __name__ == '__main__':
	main()

Last edited by PeterT; 04-19-2018 at 07:35 PM. Reason: Added code block to preserve spaces
Sasuraigar is offline   Reply With Quote
Old 04-19-2018, 05:33 PM   #10
Pete1983
Junior Member
Pete1983 began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Apr 2018
Device: kindle oasis 1
Hi,
I know this is a bit of an old thread, but its just what I was looking for... though my python knowledge is zero (and my bash knowledge is only nearly zero)
so i am hoping a helpful person on here can give me a few pointers...
what needs substituting in this script to tailor it to my system? (directory/input/output format)
I am looking to run this automatically to find epubs in my calibre library with no mobi and create it...
do i need to do anything to add the format to calibre so i can access it from the web interface? - I am running a headless linux server.

many thanks...
Pete1983 is offline   Reply With Quote
Old 04-19-2018, 06:54 PM   #11
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Calibre can do it
Search
Code:
not formats:mobi
Select the answers and bulk convert to the desired format


AZW3 is a lot better than Mobi unless you have a <=K2
theducks is online now   Reply With Quote
Old 04-19-2018, 08:13 PM   #12
Pete1983
Junior Member
Pete1983 began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Apr 2018
Device: kindle oasis 1
That would work if I were running the GUI but I'm only using the calibre-server, and I'm trying to automate the process.

EDIT: I have cobbled something dirty together in bash... not as elegant but it seems to be doing the job

Last edited by Pete1983; 04-20-2018 at 07:03 AM.
Pete1983 is offline   Reply With Quote
Old 09-06-2018, 03:34 AM   #13
sdfg
Member
sdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the rough
 
Posts: 14
Karma: 7398
Join Date: Feb 2015
Device: none
This is very useful. The problem I have is I use a scrolling ebook reader, and having forced page breaks because of multiple html pages (epub) is a pain in the bum. I've played around with a few ideas (like merging html pages in the editor, but that doesn't take into account order), and the best option I can see is to convert to HTMLZ, and then back to EPUB. I'd love to use the bulk converter for this, and I've tried, but there's no option that I can see to select the input format to convert. So I can bulk convert to HTMLZ, which is fine, but then it automagically will only convert the original epub back into an epub, which isn't fine. So this script is perfect for me (I'll just add a switch to force the conversion even if the 'want' format is already present, and probably clean up the htmlz after, as well). I've got a question though, would this mess up the calibre db in any way? I imagine it keeps size information and things somewhere, I don't want anything to get corrupt because the file size on the epub has changed.

(Also let us select which input format to convert in bulk convert, it would save me having to do this outside calibre :-) )
sdfg is offline   Reply With Quote
Old 09-06-2018, 08:00 AM   #14
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Preferences->Behavior->Input format order

or simply select all books after converting to htmlz and delete the epub format by right clicking the remove books button and choosing delete specific format.

And merging files in the editor does preserve order.
kovidgoyal is offline   Reply With Quote
Old 10-19-2018, 05:39 PM   #15
sdfg
Member
sdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the roughsdfg is a jewel in the rough
 
Posts: 14
Karma: 7398
Join Date: Feb 2015
Device: none
Quote:
Originally Posted by kovidgoyal View Post
Preferences->Behavior->Input format order

or simply select all books after converting to htmlz and delete the epub format by right clicking the remove books button and choosing delete specific format.
Thanks for that. I've changed the Input format order, but it seems to be ignored when converting to a format that already exists. For example,

Input format order: HTMLZ at the top
Preferred output format: HTMLZ

Add EPUB
(Automatically get converted to HTMLZ, as expected)
Convert (either individually or in bulk) to EPUB.

The original EPUB is converted to EPUB, NOT the HTMLZ.

Last edited by sdfg; 10-19-2018 at 05:44 PM.
sdfg is offline   Reply With Quote
Reply

Tags
batch, convert, python

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Python script: batch create mobi's from epubs (if they don't exist) jmeb Conversion 0 05-20-2011 01:14 PM
Classic Batch download of B&N eBooks? mgmueller Barnes & Noble NOOK 5 02-08-2010 12:01 PM
Batch Convert Ebooks? April Workshop 7 12-09-2009 03:07 AM
Mobigen Mass Batch conversion of HTML-Single-File ebooks to .mobi ebooks cklammer Kindle Formats 9 11-20-2009 03:00 AM
Crash while batch converting epubs under windows AprilHare Calibre 3 04-26-2009 03:46 AM


All times are GMT -4. The time now is 08:54 AM.


MobileRead.com is a privately owned, operated and funded community.