12-07-2012, 12:31 PM | #1 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
External Index comparison?
Hola to All!
Perhaps I have missed it, but I am looking for a plugin (or other?) that will me to compare the current/active library of books against an "external" list of books. The reason is of course to eliminate duplicates. My collection is simply too large (80K titles) to keep within Calibre and is maintained on the web. Calibre is of course used for cleaning up titles, authors, conversion, editing, adding covers, metadata, etc., but are not saved locally. I do maintain a separate spreadsheet of all titles, but using it to compare against new entries is cumbersome, to say the least! I would like to have an external CSV file (or plain text, or whatever) with all my current tittles that Calibre can check against once I have cleaned up a new set of titles in the active library. Thoughts, ideas? Thanks! |
12-07-2012, 12:32 PM | #2 |
creator of calibre
Posts: 43,858
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
use the calibre create catalog feature to generate a catalog of the calibre library in csv format. Then use your favorite spreadsheet program to compare them.
|
Advert | |
|
12-07-2012, 12:50 PM | #3 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Kovid,
Thanks for the idea! That should save me an extra step that I am now doing when I get to the compare process. I was hoping to 'automate' it even further, but will see if I can use the csv catalog approach. Again, thanks! |
12-07-2012, 02:03 PM | #4 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
I just went through the process using the CSV catalog. While it does work and save me one extra step, I would still like something more 'automated'. Yes, I realize there may be nothing out there right now, but here is an idea a plugin developer might consider:
The "Find Duplicates" plugin works great for comparing titiles within a given library; and it works great. Take that idea but let it compare an external CSV list against the current library! And of course, in my case it would need to take large lists! Here is how I have to do it now, using the CSV catalog approach suggested by Kovid- 1. When the active library title/author lists are cleaned up, I export them using the catalog featur. 2. I open my spreadsheet containing my large list of current titles I already have. 3. I have to go through a complicated ONE-AT-A-TIME comparison of the current library list vs the existing title archive list. This is extremely tedious for say a 300 title library comparison. I chose accuracy over speed on purpose, but boy, it is slow! 4. Once I weed out the duplicates, say 50 of the 300 are dupes, I have to then generate the dupe list, and then go back into Calibre and individually delete 50 titles out of the 300 in the library; again this could be 'automated' much better! So there is the process as I have it now. It does work, but it is a whole bunch of tedious work. Hopefully others may have a similar need and a plugin similar to the "Find Duplicates" plugin can be worthwhile for a developer to come up with....I am ready to donate! MontyJ [Edit] Just wanted to add that my large master index has a very simple format: 2 Columns: Author and Title. Absolutely no other data! The author is a SINGLE name, not multiple. Titles are also simple, no series, no collection or volume info, etc. Last edited by MontyJ; 12-07-2012 at 02:11 PM. |
12-07-2012, 02:22 PM | #5 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@MontyJ - just use the "Import List" plugin which has the ability to import from a CSV file. Do that import into a second library, then use Find Duplicates plugin with its ability to compare libraries...
|
Advert | |
|
12-07-2012, 03:27 PM | #6 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Kiwidude, Hey, thanks! Will give it a shot
|
12-07-2012, 04:56 PM | #7 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Kiwidude,
Cannot get the plugin to get beyond Step 1. (If I need to take this to a different forum, let me know!) My CSV list populates properly (2 columns, 22,000 records for this test), but the "Next" button is greyed out and will not advance to whatever "Step 2" is! I have tried importing the CSV list with an empty active library as well as one with 72 books in it, same result. Did I miss something? |
12-07-2012, 07:47 PM | #8 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Did you click on the Preview button to show the results on the right hand side, after setting which column number is which on the title and author settings on the left...
|
12-07-2012, 08:07 PM | #9 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Yea, I have the preview listing in the contents panel, and the right hand panel shows the same contents, fully populated with the 2 columns of title/author.
Tried removing the plugin, reinstalling, even rebooting and of course restarting Calibre after reinstalling the plugin; no change... |
12-08-2012, 07:23 AM | #10 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@MontyJ - can you post a screenshot? Clicking the Preview button so that you can see some rows from the file displayed on the right side should be all that is required to enable the Next button of the wizard.
|
12-08-2012, 09:52 AM | #11 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Here it is.
|
12-08-2012, 10:49 AM | #12 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Very interesting. Looks like you have done everything correctly, so there must be a bug in there somewhere. Any chance you could PM or email me the file? Just zip and upload it somewhere...
|
12-08-2012, 02:16 PM | #13 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
kiwidude,
Here is the csv file; it was small enough to attach, and I didn't see a way to PM you with an attachment. If this somehow buggers the file, pm me an email address you want me to send the file to. Montyj Somehow the .csv or .7z file isn't attaching! Ok, got a .zip to attach ok. This is the file I am importing, I just gave it a different name than in the original screen grab. Last edited by MontyJ; 12-08-2012 at 02:21 PM. |
12-08-2012, 02:27 PM | #14 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Kiwidude,
Just for a lark, I took the first 23 records from this file and made another .csv file with only those records; I am able to get to Step 2 using those 23 records! So, it appears as tho there is a bad character/symbol somewhere in this file. I will look at my export settings from OOO Calc and see if I use quotes or some other setting if I can negate the error. MJ [Edit] I did an export from my ODS spreadsheet file with quotes (" ") surrounding all text in the two fields, and using a TAB as delimiter. I do have commas n the title fields, so I thought that may have been causing the hiccup somewhere further down in the .csv file. But even with the fields quoted and using a TAB as delimiter, it still hangs... [Edit] I split the .csv file roughly in two; authors A-J, and then the second half K-Z. The first half works ok and lets me go on to Step 2, but the second half still will not. So if it is not something to do with the number of records, then it must be a bogus character in the K-Z file. Last edited by MontyJ; 12-08-2012 at 03:03 PM. |
12-08-2012, 03:56 PM | #15 |
Addict
Posts: 224
Karma: 10
Join Date: Jul 2012
Device: Kindle
|
Kiwidude,
Ok, found a couple blank records in the k-z csv file. Once I got those cleaned up, all is well and I can get on to Step 2 with the entire a-z csv test file. When I checked the original ODS spreadsheet file, there are NO BLANKS. That means there is a bug in the conversion output to CSV... A question: Your earlier post seemed to indicate I can then save this list in a separate library to "compare" against when I add 'new' titles to the active archive? I did not see that option when I got to step 2, as it looked like it was making a compare of this csv list with what was in the active library? Have I got it backwards?? Thanks for the help! MontyJ Last edited by MontyJ; 12-08-2012 at 04:12 PM. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Parsing Index | Steven630 | Recipes | 0 | 07-06-2012 04:53 AM |
Index: Making a linked index in epub | virtual_ink | ePub | 21 | 10-19-2011 11:23 PM |
Index of plugins | kiwidude | Plugins | 0 | 01-27-2011 07:19 PM |
DR1000 How can I force a re-index | Spelly | iRex | 4 | 09-11-2010 09:30 AM |
PRS-600 Index corrupted? | CapnKirk | Sony Reader | 0 | 07-10-2010 06:56 PM |