Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 12-07-2012, 12:31 PM   #1
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Question External Index comparison?

Hola to All!

Perhaps I have missed it, but I am looking for a plugin (or other?) that will me to compare the current/active library of books against an "external" list of books.

The reason is of course to eliminate duplicates. My collection is simply too large (80K titles) to keep within Calibre and is maintained on the web. Calibre is of course used for cleaning up titles, authors, conversion, editing, adding covers, metadata, etc., but are not saved locally.

I do maintain a separate spreadsheet of all titles, but using it to compare against new entries is cumbersome, to say the least!

I would like to have an external CSV file (or plain text, or whatever) with all my current tittles that Calibre can check against once I have cleaned up a new set of titles in the active library.

Thoughts, ideas?

Thanks!
MontyJ is offline   Reply With Quote
Old 12-07-2012, 12:32 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,787
Karma: 4998511
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
use the calibre create catalog feature to generate a catalog of the calibre library in csv format. Then use your favorite spreadsheet program to compare them.
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 12-07-2012, 12:50 PM   #3
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Kovid,

Thanks for the idea! That should save me an extra step that I am now doing when I get to the compare process. I was hoping to 'automate' it even further, but will see if I can use the csv catalog approach.

Again, thanks!
MontyJ is offline   Reply With Quote
Old 12-07-2012, 02:03 PM   #4
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
I just went through the process using the CSV catalog. While it does work and save me one extra step, I would still like something more 'automated'. Yes, I realize there may be nothing out there right now, but here is an idea a plugin developer might consider:

The "Find Duplicates" plugin works great for comparing titiles within a given library; and it works great. Take that idea but let it compare an external CSV list against the current library! And of course, in my case it would need to take large lists!

Here is how I have to do it now, using the CSV catalog approach suggested by Kovid-

1. When the active library title/author lists are cleaned up, I export them using the catalog featur.

2. I open my spreadsheet containing my large list of current titles I already have.

3. I have to go through a complicated ONE-AT-A-TIME comparison of the current library list vs the existing title archive list. This is extremely tedious for say a 300 title library comparison. I chose accuracy over speed on purpose, but boy, it is slow!

4. Once I weed out the duplicates, say 50 of the 300 are dupes, I have to then generate the dupe list, and then go back into Calibre and individually delete 50 titles out of the 300 in the library; again this could be 'automated' much better!

So there is the process as I have it now. It does work, but it is a whole bunch of tedious work. Hopefully others may have a similar need and a plugin similar to the "Find Duplicates" plugin can be worthwhile for a developer to come up with....I am ready to donate!

MontyJ

[Edit] Just wanted to add that my large master index has a very simple format: 2 Columns: Author and Title. Absolutely no other data! The author is a SINGLE name, not multiple. Titles are also simple, no series, no collection or volume info, etc.

Last edited by MontyJ; 12-07-2012 at 02:11 PM.
MontyJ is offline   Reply With Quote
Old 12-07-2012, 02:22 PM   #5
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@MontyJ - just use the "Import List" plugin which has the ability to import from a CSV file. Do that import into a second library, then use Find Duplicates plugin with its ability to compare libraries...
kiwidude is offline   Reply With Quote
Old 12-07-2012, 03:27 PM   #6
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Kiwidude, Hey, thanks! Will give it a shot
MontyJ is offline   Reply With Quote
Old 12-07-2012, 04:56 PM   #7
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Kiwidude,

Cannot get the plugin to get beyond Step 1. (If I need to take this to a different forum, let me know!)

My CSV list populates properly (2 columns, 22,000 records for this test), but the "Next" button is greyed out and will not advance to whatever "Step 2" is!

I have tried importing the CSV list with an empty active library as well as one with 72 books in it, same result.

Did I miss something?
MontyJ is offline   Reply With Quote
Old 12-07-2012, 07:47 PM   #8
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Did you click on the Preview button to show the results on the right hand side, after setting which column number is which on the title and author settings on the left...
kiwidude is offline   Reply With Quote
Old 12-07-2012, 08:07 PM   #9
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Yea, I have the preview listing in the contents panel, and the right hand panel shows the same contents, fully populated with the 2 columns of title/author.

Tried removing the plugin, reinstalling, even rebooting and of course restarting Calibre after reinstalling the plugin; no change...
MontyJ is offline   Reply With Quote
Old 12-08-2012, 07:23 AM   #10
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@MontyJ - can you post a screenshot? Clicking the Preview button so that you can see some rows from the file displayed on the right side should be all that is required to enable the Next button of the wizard.
kiwidude is offline   Reply With Quote
Old 12-08-2012, 09:52 AM   #11
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Here it is.
Attached Thumbnails
Click image for larger version

Name:	Import List.JPG
Views:	49
Size:	161.2 KB
ID:	97432  
MontyJ is offline   Reply With Quote
Old 12-08-2012, 10:49 AM   #12
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Very interesting. Looks like you have done everything correctly, so there must be a bug in there somewhere. Any chance you could PM or email me the file? Just zip and upload it somewhere...
kiwidude is offline   Reply With Quote
Old 12-08-2012, 02:16 PM   #13
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
kiwidude,

Here is the csv file; it was small enough to attach, and I didn't see a way to PM you with an attachment. If this somehow buggers the file, pm me an email address you want me to send the file to.

Montyj

Somehow the .csv or .7z file isn't attaching!
Ok, got a .zip to attach ok. This is the file I am importing, I just gave it a different name than in the original screen grab.
Attached Files
File Type: zip Test.zip (242.1 KB, 48 views)

Last edited by MontyJ; 12-08-2012 at 02:21 PM.
MontyJ is offline   Reply With Quote
Old 12-08-2012, 02:27 PM   #14
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Kiwidude,

Just for a lark, I took the first 23 records from this file and made another .csv file with only those records; I am able to get to Step 2 using those 23 records!

So, it appears as tho there is a bad character/symbol somewhere in this file. I will look at my export settings from OOO Calc and see if I use quotes or some other setting if I can negate the error.

MJ

[Edit] I did an export from my ODS spreadsheet file with quotes (" ") surrounding all text in the two fields, and using a TAB as delimiter. I do have commas n the title fields, so I thought that may have been causing the hiccup somewhere further down in the .csv file. But even with the fields quoted and using a TAB as delimiter, it still hangs...

[Edit] I split the .csv file roughly in two; authors A-J, and then the second half K-Z. The first half works ok and lets me go on to Step 2, but the second half still will not. So if it is not something to do with the number of records, then it must be a bogus character in the K-Z file.

Last edited by MontyJ; 12-08-2012 at 03:03 PM.
MontyJ is offline   Reply With Quote
Old 12-08-2012, 03:56 PM   #15
MontyJ
Enthusiast
MontyJ began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Jul 2012
Device: Kindle
Kiwidude,

Ok, found a couple blank records in the k-z csv file. Once I got those cleaned up, all is well and I can get on to Step 2 with the entire a-z csv test file. When I checked the original ODS spreadsheet file, there are NO BLANKS. That means there is a bug in the conversion output to CSV...

A question: Your earlier post seemed to indicate I can then save this list in a separate library to "compare" against when I add 'new' titles to the active archive?

I did not see that option when I got to step 2, as it looked like it was making a compare of this csv list with what was in the active library? Have I got it backwards??

Thanks for the help!

MontyJ

Last edited by MontyJ; 12-08-2012 at 04:12 PM.
MontyJ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Parsing Index Steven630 Recipes 0 07-06-2012 04:53 AM
Index: Making a linked index in epub virtual_ink ePub 21 10-19-2011 11:23 PM
Index of plugins kiwidude Plugins 0 01-27-2011 07:19 PM
DR1000 How can I force a re-index Spelly iRex 4 09-11-2010 09:30 AM
PRS-600 Index corrupted? CapnKirk Sony Reader 0 07-10-2010 06:56 PM


All times are GMT -4. The time now is 08:43 PM.


MobileRead.com is a privately owned, operated and funded community.