Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 05-19-2009, 01:31 PM   #1
averyml
Member
averyml began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Nov 2007
Device: N800
Scripting with epub-meta

In the dark days before Calibre, I used Tellico to manage my ebook collection. Tellico is great for some things, but it wasn't really meant to be an ebook manager, and I would like to convert my massive epub collection over to Calibre. The problem is that I have spent many hours carefully organizing my collection, and the thought of redoing everything in Calibre is a rather daunting task. I have exported my Tellico collection as an XML file, and it has occurred to me that I might be able to use some kind of bash script together with epub-meta to write the Book title, author, and description to the actual file, which Calibre can then read. The problem is I know little about bash scripting, even less about xml, and I'm not sure if there's something else I would need to tie them all together. Does anyone by chance know of a script out there already capable of doing this, or something similar? If not, is there at least someone who wouldn't mind giving me a general idea of where to start?
averyml is offline   Reply With Quote
Old 05-20-2009, 02:20 AM   #2
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 630
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Ok, I definitely am not the best person to answer...

But off the top of my head, you will need to know which file(s) make up the book database. You will also need to know the file structure (how the data is listed) in each of the affected files. Since XML is generally human-readable, you might be able to simply append the necessary data to the existing file(s) and save.

Somehow, I doubt it is going to be this easy.
Sabardeyn is offline   Reply With Quote
 
Advertisement
Old 05-20-2009, 07:55 AM   #3
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,473
Karma: 1053245
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
Bash scripting doesn't have the best support for xml parsing...

Depending on who the data is written to the file it could be either very difficult or very easy.

Say the data looks like this:

Code:
<book>
  <path>/home/you/books/author/title.ext</path>
  <title>Test Title</title>
  <author>Test Author</title>
</book>
<book>
  <path>/home/you/books/author/title2.ext</path>
  <title>Test Title</title>
  <author>Test Author</title>
</book>
All you would need to do is iterate over each book node, get the path, title, and author values, then create the command line arguments for ebook-meta and call it. If you use python you could actually the components directly from your script. If you go this route you can find everything you need in calibre.ebooks.metadata.cli.
user_none is offline   Reply With Quote
Old 05-20-2009, 11:37 AM   #4
averyml
Member
averyml began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Nov 2007
Device: N800
That is, in fact, pretty much how the file is arranged...here's an example:

Code:
<entry id="21" >
   <title>The Shadow of the Lion</title>
   <authors>
    <author>Lackey, Mercedes</author>
    <author>Flint, Eric</author>
    <author>Freer, Dave</author>
   </authors>
   <binding>Paperback</binding>
   <publisher>Baen</publisher>
   <pub_year>2003</pub_year>
   <isbn>0-7434-7147-4</isbn>
   <pages>936</pages>
   <languages>
    <language>English</language>
   </languages>
   <keywords>
    <keyword>Historical fiction</keyword>
    <keyword>Fantasy fiction</keyword>
    <keyword>16th century</keyword>
    <keyword>Science fiction</keyword>
   </keywords>
   <cover>2721667105d9edee31ffa87663fa0863.jpeg</cover>
<comments>Description of the book</comments>
   <file>file:///share/fiction/Lackey,%20Mercedes/The%20Shadow%20of%20the%20Lion.txt</file>
Things like the multiple authors in this record could make things a little more complicated, but I can always just use the first author. Looks like it's time to learn python...thanks for the advice! I was mostly concerned about spending a few weeks learning how to do something, only to find out I'm completely on the wrong track (like, say, trying to do it in Bash, which I now know doesn't have great support for xml parsing).
averyml is offline   Reply With Quote
Old 05-20-2009, 12:17 PM   #5
rogue_ronin
Banned
rogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-booksrogue_ronin has learned how to read e-books
 
Posts: 475
Karma: 796
Join Date: Sep 2008
Location: Honolulu
Device: Nokia 770 (fbreader)
A scriptable text editor could do the job, too, if you're already familiar with one.

m a r
rogue_ronin is offline   Reply With Quote
Old 05-20-2009, 12:42 PM   #6
gwynevans
Wizard
gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.gwynevans ought to be getting tired of karma fortunes by now.
 
gwynevans's Avatar
 
Posts: 1,343
Karma: 1065246
Join Date: Nov 2007
Location: UK
Device: Sony 505 (retired), iPad2, iPhone 3GS & Nexus 7 3G
For the XML in Python, I've found BeautifulSoup to be easy to get started with.

('>>>' is the python command prompt when it's run interactively)
e.g.
Code:
>>> from BeautifulSoup import BeautifulStoneSoup
>>> file = open('example.xml')  # your example above with a </entry> appended
>>> soup = BeautifulStoneSoup(file)
>>> soup.title.string
u'The Shadow of the Lion'
>>> soup.findAll('author')
[<author>Lackey, Mercedes</author>, <author>Flint, Eric</author>, <author>Freer, Dave</author>]
>>> soup.findAll('author')[0].string
u'Lackey, Mercedes'
>>> keywords = soup.keywords
>>> keywords.findAll('keyword')
[<keyword>Historical fiction</keyword>, <keyword>Fantasy fiction</keyword>, <keyword>16th century</keyword>, <keyword>Science fiction</keyword>]
>>> [k.string for k in keywords.findAll('keyword')]
[u'Historical fiction', u'Fantasy fiction', u'16th century', u'Science fiction']
>>>
gwynevans is offline   Reply With Quote
Old 05-20-2009, 01:15 PM   #7
Format C:
Guru
Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.Format C: ought to be getting tired of karma fortunes by now.
 
Posts: 753
Karma: 1496807
Join Date: Jul 2008
Location: The Third World
Device: iLiad + PRS-505 + Kindle 3
Beautiful Soup is fantastic!
I'd couple it with SQLite to enter the data directly in calibre's metadata.db file.

In simple steps:
1. Import all the files in calibre (without metadata, just the title, maybe in the filename)
2. Export all those data in CSV from SQLite
3. Enrich the CSV with python/beautifulSoup
4. Load the data in the database with SQLite
Format C: is offline   Reply With Quote
Old 05-20-2009, 01:33 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,463
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I would recommend using the calibredb command to add books with their metadata to the calibre database.

Use

calibredb add
calibredb set_metadata
calibredb add_format
kovidgoyal is offline   Reply With Quote
Old 05-20-2009, 06:45 PM   #9
averyml
Member
averyml began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Nov 2007
Device: N800
Thanks everyone for your help! I've actually gotten surprisingly far given the fact that this morning I knew absolutely nothing about python. Beautiful soup has made it really easy, but now that I've gotten as far as writing the data to the file, I ran into a little bit of an issue...When I tried to write the author to the file using epub-meta, the result was "Author: Lackey & Mercedes". I was completely confused as to how this was happening until I tried to do it manually and realized it does the same thing. Apparently it takes the input "Lackey, Mercedes" as specifying two different authors. Is there any way to change this, or should i take each author in the format "lastname, firstname" and convert it to "firstname lastname" before writing the metadata?
averyml is offline   Reply With Quote
Old 05-20-2009, 09:05 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,463
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
use firstname lastname
kovidgoyal is offline   Reply With Quote
Old 05-27-2009, 09:07 PM   #11
averyml
Member
averyml began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Nov 2007
Device: N800
Thank you all for your help! After a few hours studying I have the following (which I know, is horrible, but it works):

Code:
#import os for executing commands
import os
#import BeautifulSoup
from BeautifulSoup import BeautifulStoneSoup
#define file to use
file = open('/home/michelle/ebookcollection.xml')
#soup it
soup = BeautifulStoneSoup(file)
#read all entries into an array books
for book in soup.fetch('entry'):
	#set book title
	bookTitle = book.title.string
	#set book file
	bookFile = book.file.string
	bookFile = bookFile.replace('%20', ' ')
	bookFile = bookFile.replace('%27','\'')
	#determine if book is part of series and set requisite values
	bookSeries=""
	bookSeriesIndex=""
	if book.series is not None:
		bookSeries=book.series.string
	if book.series_num is not None:
		bookSeriesIndex=book.series_num.string
	#if  book has description, set that as well
	bookComment = ""
	if book.comments is not None:
		bookComment=book.comments.string
	bookAuthors = ""   
	bookTags = "Unread, Tellico"
	for a in book.fetch('author'):
		thisAuthor = a.string
		if thisAuthor.find(",") > -1:
			firstName = thisAuthor[(thisAuthor.find(",") + 2):]
			lastName = thisAuthor[:thisAuthor.find(",")]
			thisAuthor = firstName + " " + lastName
		bookAuthors = bookAuthors + thisAuthor +", "
	bookAuthors = bookAuthors.strip(", ")
	print "Title: " + bookTitle
	print "Authors: " + bookAuthors
	print "File: " + bookFile
	print "Series: " + bookSeries
	print "Series Number: " + bookSeriesIndex
	print "Description: " + bookComment
	print "Tags: " + bookTags
	#write it out
	command = "epub-meta -t \"" + bookTitle + "\" -a \"" + bookAuthors + "\" --comment=\"" + bookComment + "\" --series=\"" + bookSeries + "\" --series-index=\"" + bookSeriesIndex +"\" --tags=\"" + bookTags + "\" \"" + bookFile + "\""
	os.system(command)
There are only two things I haven't been able to figure out how to write to the metadata yet: the ISBN number, and the cover. epub-meta seems to be able to read the ISBN number and extract the cover, but not set either of them. The cover isn't a big deal, but I'd really like to be able to set the ISBN number. I suspect that this may be part of reason kovid suggested using calibredb to add them directly, but, as I understand it, I'd have to use the xml file to create temporary opf files for each book to use for the set_metadata part, and I'm afraid that goes well beyond my python coding abilities...is there a simpler way of doing this?
averyml is offline   Reply With Quote
Old 05-27-2009, 10:57 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,463
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Just wait a couple of weeks calibre 0.6 will have an ebook-meta utility that will allow you to set the isbn and the cover
kovidgoyal is offline   Reply With Quote
Old 05-27-2009, 10:58 PM   #13
averyml
Member
averyml began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Nov 2007
Device: N800
perfect! thank you!
averyml is offline   Reply With Quote
Old 05-28-2009, 12:11 AM   #14
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by kovidgoyal View Post
Just wait a couple of weeks calibre 0.6 will have an ebook-meta utility that will allow you to set the isbn and the cover
You do realize we're already excited about the 0.6 beta, right? Why are you still whetting our appetite?
ilovejedd is offline   Reply With Quote
Old 05-28-2009, 12:49 AM   #15
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 630
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Oh my... weeks.
Sabardeyn is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ePub meta data brudigia ePub 4 07-26-2010 01:58 PM
Any command line to edit epub meta data? bthoven Calibre 1 03-25-2010 08:26 AM
Any command line to edit epub meta data? bthoven ePub 2 03-25-2010 05:15 AM
set meta data with ebook-meta and ebook-convert krischik Calibre 6 01-19-2010 12:40 PM
epub-meta tags JeffElkins Calibre 2 10-17-2008 05:04 PM


All times are GMT -4. The time now is 11:14 AM.


MobileRead.com is a privately owned, operated and funded community.