Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 02-21-2021, 12:10 AM   #1
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
[GUI Plugin] LibraryThing Match

LibraryThing is a social cataloging web application for storing and sharing book catalogs and various types of book metadata. It is used by authors, individuals, libraries, and publishers.

LibraryThing Match is a calibre plug-in for comparing your calibre e-book list with your LibraryThing catalog.


Main Features:
  • Filter your LibraryThing books by Collection, Tags, N most recently added.
  • Sort you LibraryThing books by author, title, entry date, entry date reversed.
  • Display a list of your LibraryThing books for side-by-side comparison with your calibre e-books
  • Compare the two lists using a fuzzy matching algorithm to easily identify books which do/do not exist in both lists.
  • Save matches in a calibre custom column.
  • Save your LibraryThing ratings in a calibre custom column (half-star ratings).

Special Notes:
  • Requires calibre 5.0.0 or later.
  • The LibraryThing interface works with public LibraryThing catalogs only, which excludes books you have marked as "Private".

Installation Steps:
  • Install the plugin from calibre preferences (Preferences - Plugins - Get new plugins).
  • Alternatively, install it manually by downloading the attached zip file, and installing as described in the Introduction to plugins thread (look for "How do I install a plugin manually?").

Paypal Donations:
  • If you find this plugin useful please feel free to show your appreciation.


Version History:

Spoiler:
Version 0.2.0 - 29 Mar 2021
LibraryThing Match en español. Spanish translation thanks to @dunhill.
Added option to save LT book id to a calibre identifier.
Added "About" box with link to this thread.

Version 0.1.1 - 21 Feb 2021
Fix an edge case in the matching algorithm.

Version 0.1.0 - 18 Feb 2021
Initial release of LibraryThing Match plugin
Attached Thumbnails
Click image for larger version

Name:	Configure.png
Views:	402
Size:	45.6 KB
ID:	185500   Click image for larger version

Name:	Screenshot.png
Views:	365
Size:	77.6 KB
ID:	185501  
Attached Files
File Type: zip LibraryThing Match.zip (132.7 KB, 20236 views)
File Type: doc LibraryThing Match User Guide.doc (737.0 KB, 169 views)
File Type: pdf LibraryThing Match User Guide.pdf (525.2 KB, 267 views)

Last edited by Bradles; 03-28-2021 at 09:56 PM. Reason: Released version 0.2.0
Bradles is offline   Reply With Quote
Old 02-21-2021, 12:11 AM   #2
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
calibre 6 / Qt6 test version

This is a test version of the LibraryThing Match plugin for use with calibre 6 when it is released. It will also work with current versions of calibre from 5.0 upwards.

See the calibre 6 / Qt6 thread for details:

https://www.mobileread.com/forums/sh...d.php?t=344064

This version of the plugin can be installed manually for testing purposes. Report any issues at the bottom of this thread.
Attached Files
File Type: zip LibraryThing Match.zip (133.5 KB, 49 views)

Last edited by Bradles; 01-07-2022 at 01:06 AM. Reason: calibre 6 / Qt6 test version, works with calibre 5 too
Bradles is offline   Reply With Quote
Advert
Old 02-21-2021, 10:54 AM   #3
Duke9700
Member
Duke9700 began at the beginning.
 
Duke9700's Avatar
 
Posts: 13
Karma: 10
Join Date: Nov 2009
Device: Kindle Voyage/Galaxy Note Pro 12.2/Galaxy S20
I am very excited about this plugin. LT deserves far more love than it seems to get!

For those of us who use Authors names as FN LN (I know - scandalous!) the plugin will try and match calibre first names to LF last names e.g. Adam-Troy Castro is paired up with John Joseph Adams.

Are there any tricks or possible future configuration for this? I'm assuming I would just go back and forth in the Preferences>>Advanced>>Tweaks and switch between fn ln and ln, fn for Author Sort. If anyone has a better or more elegant method I'd love some ideas!
Duke9700 is offline   Reply With Quote
Old 02-21-2021, 05:37 PM   #4
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
Quote:
Originally Posted by Duke9700 View Post

For those of us who use Authors names as FN LN (I know - scandalous!) the plugin will try and match calibre first names to LF last names e.g. Adam-Troy Castro is paired up with John Joseph Adams.
Thanks Duke.

I would not expect a very high probability match in that situation. The algorithm splits each string into "tokens" (i.e. words), sorts them, then compares the shortest combination. It includes the book title, so only if the titles are very similar as well would you get a high probability match. You can expect quite a few low probability matches, e.g. 50% and under, just from these random word matches.

The LT API also returns author first last, so it would be fairly easy to add that as a LTMatch configuration option.

Let me know if you are seeing anything different and I'll look into it further.
Bradles is offline   Reply With Quote
Old 03-18-2021, 11:29 PM   #5
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
Responding to a post in the "LTMatch Beta" thread (which is now closed).

Quote:
Originally Posted by jjmcgaffey View Post
So I'm an LT and calibre user - theoretically they match perfectly, since most of my ebooks have been entered by importing a calibre-exported database. Theoretically. I'm delighted to see your plugin!
Once again, thanks for taking the time to test my plugin, and persisting despite frustrations.

Quote:
First run - very frustrating. The documentation is findable online (in this thread), but it's not reachable directly from the plugin - so I clicked on the plugin and it complained about no librarything_id column, so I made a librarything_id column (of the wrong type - floating point), which meant I had to restart calibre, then I noticed the part about the librarything_rating column...didn't make it, I didn't feel like restarting... The documentation is really necessary, as it's perfectly possible to make a mess pushing random buttons on the plugin. Is it possible to make a button that's a link to the documentation (and somehow point it out, especially on first run)?
It should be easy enough to include a button/link to the first post above. It might end up being in an "About" dialog.

Quote:
My full LT library is 11,000+ books; my ebook collection is 5,000+. I tried telling LTMatch to look for 10,000 books, and it timed out. So I reduced it to 1000. The loading was pretty fast - less than a minute, I think. The matching took longer, about 5-6 minutes; not unbearable. It's matching very nicely, and finding an awful lot of books I have listed in calibre that aren't showing up on LT (some of them may be matters of edited authors or titles, but determining that will require doing a full check).
One of the limitations of the LT API is not being able to search by collection. So if you use a LT Collection to identify your ebooks (as I do), the plugin has to download ALL your LT books, then filter for books in the "Ebooks" collection.

Many LT users use tags to identify ebooks, or newly added books, and searching by tags is fast. In fact, I also use tags to identify the source of the ebook, e.g. Kindle, Google, Apple Books. This makes for faster searching and the plugin only downloads what it needs.

Quote:
The first lot matched perfectly - maybe 50 or 90 books, with just a few gaps. Then there were huge blank spaces on the LT side; not sure why, I'm pretty sure those books are in fact there. Also some complete mis-matches, with low probability; in at least one case, there was a much better match higher up, not sure why it was ignored. But it's not hard to force the match (except that with a list this long, just scrolling through it is hard).
Hopefully the "50 or 90" was not from 1000 books! The gaps are simply "spacers" to ensure that matched books are always next to each other in the lists.

Failing to identify the best match is a concern, but this sometimes happens with series, where the series name is part of the book title. If you share the names of the books concerned (by DM if you like) I can have a look.

Quote:
I think I'll start over using only 100 at a time, now that (having found and read the documentation) I understand that once it's matched it doesn't need to be looked at again. Doing 1000 at a time, there's too much chance of a mis-match. So I closed the window without having it accept matches.
I think the key to successful matching, is to have a LT list that you expect will fully encompass the calibre list. This isn't really possible with small batches, as you'll always be getting the same LT list, and changing the sort order won't help much.

There are a number of ways to filter your calibre list to help though, e.g. Authors whose surname starts with A:
Code:
author_sort:~^A
Then B, then C, etc. LTMatch works with the currently displayed calibre list, so you can at least focus your efforts.

Quote:
Is this useful info? Are there other things you want to know? I'll be trying the plugin again soon, and will check back here to update my experience with it.
Very useful info. I'd be interested to know how you identify ebooks in LT. I haven't actually seen too many users with an Ebooks collection like me.

Quote:
I'm having some serious frustrations with the forum - I tried a quick reply and it logged me out, tried it again and it appeared as a blank post and then disappeared entirely. Hoping this one will work.
Sorry, I can't help with that.

You may be interested to know, I'm working on a LibraryThing Metadata plugin too. It's like the Amazon, Google, Goodreads, etc plugins, except it downloads your metadata and covers from LT. (If you have a matching book.) It builds on the LTMatch plugin, by using the librarything_id to go directly to your book.

It's not quite ready for testing yet, but I'll surely post something in this thread when it is.

Last edited by Bradles; 03-18-2021 at 11:32 PM. Reason: Added note re: source of JJ's comments
Bradles is offline   Reply With Quote
Advert
Old 03-19-2021, 04:09 AM   #6
jjmcgaffey
Member
jjmcgaffey began at the beginning.
 
jjmcgaffey's Avatar
 
Posts: 10
Karma: 10
Join Date: Apr 2014
Device: Moto G Power (2020), various tablets
Second run

OK, I didn't understand about the matched books not being matched again... I'll try your suggestion of filtering calibre by author's initial.

I do use a collection to mark all my LT ebooks. I suppose I could Power Edit and tag them all with something, but that's a little annoying. Can LTMatch deal with wildcards in tags? I do have most of them marked with _import(date) - that is, the last batch I added are tagged _import210316. If I could tell LTMatch to look for _import*, that would work (for most of them, and I should tag the ones that don't have it anyway).

OK, I tried to set LTMatch going with all the calibre authors that begin with A, and max results 12000 (to exceed my LT library). I got

"Can't connect to LibraryThing due to an error.
This could be due to an incorrect username. See error details below."

and when I clicked Show Details it said

"timeout('The read operation timed out')"

So I can't set it at my whole library. Hmm.

OK, weird! I tried it with tag _import* and max results 6000 - I got the same error, but the details say

"URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)'))"

Sounds like that asterisk messed something up. I guess I'm stuck at the moment. I'll check back with you later - thanks!
jjmcgaffey is offline   Reply With Quote
Old 03-23-2021, 04:58 AM   #7
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
Quote:
Originally Posted by jjmcgaffey View Post
and when I clicked Show Details it said

"timeout('The read operation timed out')"
I did some investigating, and the default timeout is None, i.e. no limit. I'm not specifying any timeout in the API call either. I tested (using the "Test Connection" button) with a 15,000 book library (thankyou CovenantNetwork whoever you are) and successfully retrieved 14980 books in around 2 minutes.

Perhaps there is something going on at your end, i.e. proxy server, VPN or something? Could speed be an issue? My broadband is 30Mbits, so not particularly fast.

Also, per your suggestion, I've added an "About" box, which includes a link to this thread. Not that it will do you much good, but some future user may benefit. An update with that and a couple of minor changes to be released in the next couple of days.

Also also, you could try firing off a test query in your browser, using a URL of the form:

Code:
https://www.librarything.com/api_getdata.php?userid=sample&showstructure=0&max=10&showCollections=1&showRatings=1&showTags=1&booksort=author,title&responseType=json

Last edited by Bradles; 03-23-2021 at 05:04 AM. Reason: added test link
Bradles is offline   Reply With Quote
Old 03-28-2021, 10:17 PM   #8
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
LibraryThing Match version 0.2.0 released. Updates:
  • LibraryThing Match en español. Spanish translation thanks to @dunhill.
  • Added option to save LT book id to a calibre identifier.
  • Added "About" box with link to this thread.
Enjoy!
Bradles is offline   Reply With Quote
Old 03-29-2021, 01:42 PM   #9
dunhill
Evangelist
dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.dunhill can program the VCR without an owner's manual.
 
dunhill's Avatar
 
Posts: 480
Karma: 196652
Join Date: Sep 2017
Location: Argentina
Device: moon+ reader, kindle paperwhite
Enjoy !!!
dunhill is offline   Reply With Quote
Old 03-30-2021, 07:17 PM   #10
aklau75
Junior Member
aklau75 began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Feb 2017
Device: Samsung Galaxy S6
calibre, version 5.14.0
ERROR: Install plugin failed: A problem occurred while installing this plugin. This plugin will now be uninstalled. Please post the error message in details below into the forum thread for this plugin and restart calibre.

Traceback (most recent call last):
File "calibre\gui2\dialogs\plugin_updater.py", line 709, in _install_clicked
File "calibre\customize\ui.py", line 472, in add_plugin
File "calibre\customize\ui.py", line 61, in load_plugin
File "calibre\customize\zipplugin.py", line 293, in load
File "importlib\__init__.py", line 127, in import_module
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "calibre\customize\zipplugin.py", line 192, in exec_module
File "calibre_plugins.fictiondb.__init__", line 11, in <module>
ImportError: cannot import name 'quote' from 'urllib' (C:\Program Files (x86)\Calibre2\app\bin\python-lib.bypy.frozen\urllib\__init__.pyc)
aklau75 is offline   Reply With Quote
Old 03-30-2021, 08:25 PM   #11
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
This looks like a fictiondb issue, which has not been updated for python3 and therefore doesn't work with calibre 5+. See this thread for more:

https://www.mobileread.com/forums/sh...=171674&page=7
Bradles is offline   Reply With Quote
Old 06-04-2021, 11:54 PM   #12
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
LibraryThing Custom Columns

Here are a couple of custom columns I added that link to the LT website. (See attached images for configuration.)

LT Find: Direct link to your book on librarything.com, or search if it doesn't exist.

Template

Code:
program:
    id = select(field('identifiers'), 'ltid');
    if id then
        strcat('<a href="https://www.librarything.com/work/book/', id , '">', field('title'), '</a>')
    else
        strcat('<a href="https://www.librarything.com/title/', field('title'), ' ', field('author'), '">', field('title'), '</a>')
    fi;

LT Add: Direct link to your book on librarything.com, or an "add books" search using Title + Author.

Template

Code:
program:
    id = select(field('identifiers'), 'ltid');
    if id then
        strcat('<a href="https://www.librarything.com/work/book/', id , '">', field('title'), '</a>')
    else
        strcat('<a href="https://www.librarything.com/addbooks?search=', field('title'), ' ', field('author'), '">', field('title'), '</a>')
    fi;
If you add tags when adding books you need to enter the tags and re-search.
Attached Thumbnails
Click image for larger version

Name:	LT Find Column.png
Views:	137
Size:	65.1 KB
ID:	187491   Click image for larger version

Name:	LT Add Column.png
Views:	128
Size:	68.1 KB
ID:	187492  
Bradles is offline   Reply With Quote
Old 12-10-2021, 03:45 PM   #13
mihrimah
Junior Member
mihrimah began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2012
Device: iPhone
Hi;

I'm trying to set up this plugin (calibre v5.33.2, plugin v 0.2.0), and I'm getting a "Can't connect to LibraryThing due to an error" message. The Details panel reads "JSONDecodeError('Expecting value: line 1 column 1 (char 0)')".

This only happens when I try to retrieve 20 or more books at a time; anything up to 19 works fine. Am I doing something wrong?
mihrimah is offline   Reply With Quote
Old 12-11-2021, 05:26 PM   #14
Bradles
Enthusiast
Bradles began at the beginning.
 
Bradles's Avatar
 
Posts: 39
Karma: 10
Join Date: Nov 2020
Location: Perth, Western Australia
Device: Apple Books & Kobo Libra H20
Hi mihrimah,

That error can occur if you have not specified "LibraryThing user" or "Max results" in the Customize LibraryThing Match" screen, which you can access from the "Configure..." button.

Please ensure those values are filled in and try again.

If you are still having problems you can message me. Just in case you like reading user manuals, there's a short one attached to the first post.

Brad.

PS: Sorry your first post had to be about an error with my plugin...
Bradles is offline   Reply With Quote
Old 12-11-2021, 10:19 PM   #15
mihrimah
Junior Member
mihrimah began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2012
Device: iPhone
I've got those filled in; it only happens when "Max results" is 20 or above. I'll take some screenshots and message them to you.
mihrimah is offline   Reply With Quote
Reply

Tags
calibre, librarything, ltmatch

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Save Virtual Libraries To Column (GUI) chaley Plugins 14 04-04-2021 05:25 AM
LibraryThing Match plugin - beta testers wanted Bradles Calibre 19 03-19-2021 03:55 AM
LibraryThing Match (calibre plugin) coming soon Bradles Plugins 5 02-18-2021 06:05 AM
LibraryThing metadata download plugin rtype Plugins 30 09-08-2015 07:24 PM
Librarything using Ctrl-D in GUI bwk Calibre 24 04-03-2012 11:34 AM


All times are GMT -4. The time now is 05:43 PM.


MobileRead.com is a privately owned, operated and funded community.