Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 08-14-2008, 06:30 PM   #1
himitsu
Junior Member
himitsu began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2008
Device: PRS-505
<Command Line> Add multiple books in multiple formats

I have a fairly extensive ebook Library. Some of my books are in multiple formats. I'd like to be able to from the commandline add books to the calibre database like. "AddBook2DB TomSawyer.lrf TomSawyer.zip TomSayer.mobi"
Is there a good way to do this?

-Joe
himitsu is offline   Reply With Quote
Old 08-14-2008, 06:43 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,344
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibredb --help
kovidgoyal is offline   Reply With Quote
Old 09-25-2010, 03:29 PM   #3
_Em
Junior Member
_Em began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2010
Device: iPod
Sorry to revive an old thread, but I'd like to keep the context of what's already been said.

According to calibredb --help, you can either add duplicates (and have to clean it up using merge from the gui later), or add by unique title, skipping files of the same title in a new format. There doesn't appear to be a way to add new formats to an existing book from the command line (no command line merge function either). Am I missing something, or has someone hacked up a script that will accomplish this?

Do I have to rip apart a few plugins to create a command line script to do this? I have found adding by command line to be orders of magnitude faster than using the GUI (either via interface or by dragging books over the window -- my preferred method).

What I'd really like to do is write a script that traverses a directory structure, verifies that my to-be-imported books have the correct naming format to be imported correctly, and then imports all of those books, correctly merging books with the same author and title. After this, the script would then trigger mekk's ISBN detection script, and after that would trigger the gui-style download metadata script.

Right now, this does not seem possible without scripting the GUI itself (which seems a bit backward, as the GUI is generated by scripts in the first place).

Thoughts? Ideas?
_Em is offline   Reply With Quote
Old 09-25-2010, 03:42 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,344
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibredb add_format -h
kovidgoyal is offline   Reply With Quote
Old 09-25-2010, 03:43 PM   #5
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by _Em View Post
Sorry to revive an old thread, but I'd like to keep the context of what's already been said.

According to calibredb --help, you can either add duplicates (and have to clean it up using merge from the gui later), or add by unique title, skipping files of the same title in a new format. There doesn't appear to be a way to add new formats to an existing book from the command line (no command line merge function either). Am I missing something, or has someone hacked up a script that will accomplish this?

Do I have to rip apart a few plugins to create a command line script to do this? I have found adding by command line to be orders of magnitude faster than using the GUI (either via interface or by dragging books over the window -- my preferred method).

What I'd really like to do is write a script that traverses a directory structure, verifies that my to-be-imported books have the correct naming format to be imported correctly, and then imports all of those books, correctly merging books with the same author and title. After this, the script would then trigger mekk's ISBN detection script, and after that would trigger the gui-style download metadata script.

Right now, this does not seem possible without scripting the GUI itself (which seems a bit backward, as the GUI is generated by scripts in the first place).

Thoughts? Ideas?
I did the code for Merge and for AutoSort/AutoMerge (put new formats into existing records where author/title fuzzy match). On the issue of command line merge, how do you plan to identify the merge records? The GUI merge takes the selection made by the user for source and destination records, finds the book ids to access the records in the library and goes from there. If you have the book ids, you can use the code from the command line (calibre.library.database2 import LibraryDatabase2) with calibre-debug -c or -e. Give me the record ids and I can give you the command line code.

As to "verifies that my to-be-imported books have the correct naming format to be imported correctly," if you can code this, it would be useful for the GUI, too. I'm not sure how you tell a title, from a series, from an author.
Starson17 is offline   Reply With Quote
Old 09-25-2010, 04:23 PM   #6
_Em
Junior Member
_Em began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2010
Device: iPod
Kovidgoyal, add_format won't work unless I already have an ID. I suppose I could re-write everything from scratch and do a loop (lookup to see if book already exists, grab ID if it does and add_format, otherwise just run add), but since all the code is already written and used by the GUI, that seems a bit too much effort.

Thanks, Starson17; that points me exactly where I want to go. I was actually hoping you'd respond to this

I plan to identify the merge records by parsing the following tree:
Code:
Check for Title
If exists,
  Check for author
  If exists,
    Check for format
    If exists, end
    else
      get book ID
      merge books
else
  add book
I was also thinking of pre-processing with mekk's ISBN code, so the first step could be to check for ISBN and merge if it's found.


As for the "correct naming format", all this test would do is verify that the RE I use properly captures all the expected fields. If everything ends up in the Title field, I know the RE failed and the book file name needs a look before that book gets imported. I could also apply normalization (eg. JRR Tolkien, J.R.R. Tolkien, J. R. R. Tolkien, and Bob Smith, Bob R. Smith, Bob Smith, Jr.) and ensure things like Jr, Esq, Sir, Mr, etc. get handled appropriately by author_sort.

Next time I have a moment, I'll try to write some of these generically in Python so I can use them on the command line but also have available for someone to port to the GUI who is more heavily involved (I don't have much spare time, which is why I'm trying to avoid duplicating effort).
_Em is offline   Reply With Quote
Old 09-25-2010, 04:30 PM   #7
_Em
Junior Member
_Em began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2010
Device: iPod
Quote:
Originally Posted by Starson17 View Post
On the issue of command line merge, how do you plan to identify the merge records? The GUI merge takes the selection made by the user for source and destination records, finds the book ids to access the records in the library and goes from there. If you have the book ids, you can use the code from the command line (calibre.library.database2 import LibraryDatabase2) with calibre-debug -c or -e. Give me the record ids and I can give you the command line code.
I just realized I didn't completely answer this. I plan to identify the merge records as the result of an in-db author/title fuzzy match, similar to what you've done in the gui. For a more standard tool, I was thinking something along the lines of calibredb merge targetid currid1 currid2 etc. This could then be passed the ID sets from a parsed calibredb list -s, using the logic from my previous post.
_Em is offline   Reply With Quote
Old 09-25-2010, 06:39 PM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by _Em View Post
I plan to identify the merge records by parsing the following tree:
Code:
Check for Title
If exists,
  Check for author
  If exists,
    Check for format
    If exists, end
    else
      get book ID
      merge books
else
  add book
The code find_identical_books at line 561 of database2.py basically does this, but in reverse. It starts with the author, instead of title, then does a fuzzy match on the book titles by that author (removes spaces, punctuation, etc.) to find "identical" books.

That code is used in add.py (line 159) to do the autosort/automerge work.

I was a bit confused by your comment that there was "no command line merge function" as I couldn't see many situations where you'd know the books to be merged outside of the GUI (i.e. how you would know the ID, other than by writing code to search for books to be merged.) I can see where you'd like the autosort/automerge of new formats into existing records, however.
Starson17 is offline   Reply With Quote
Old 09-26-2010, 12:07 AM   #9
_Em
Junior Member
_Em began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2010
Device: iPod
Quote:
Originally Posted by Starson17 View Post
I was a bit confused by your comment that there was "no command line merge function" as I couldn't see many situations where you'd know the books to be merged outside of the GUI (i.e. how you would know the ID, other than by writing code to search for books to be merged.) I can see where you'd like the autosort/automerge of new formats into existing records, however.
Two methods: calibre_report_duplicates by mekk, and calibredb list --sort-by (series|author|title|author_sort|size)

both return the ID, which then gets piped into the merge. You can also parse it so that you select your base ID as the top result from hits that contains ISBN, then publisher, and then book type in order of preference.
_Em is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Content Can I add multiple books to a collection mike_bike_kite Amazon Kindle 4 06-22-2010 09:20 PM
Bug: entries with multiple formats trigger multiple conversions flinx1 Calibre 12 05-21-2010 07:23 AM
How can I add multiple formats (without converting) and have Calibre see them? strannik Calibre 4 02-03-2010 06:49 PM
Multiple formats - What gets sent edbro Calibre 2 01-20-2010 09:38 PM
Duplicate books - multiple formats mranlett Calibre 5 09-26-2009 08:02 AM


All times are GMT -4. The time now is 09:46 AM.


MobileRead.com is a privately owned, operated and funded community.