[GUI Plugin] Overdrive Link

[jhowell]

UI plugin to link books in calibre to Overdrive lending libraries

Many communities use Overdrive to handle their lending library services for electronic books. Checking Overdrive for books of interest can be tedious for large numbers of books and multiple libraries, especially since the libraries add to their collections over time.

The purpose of the Overdrive Link user interface action plugin is to search Overdrive lending libraries for borrowable e-books that correspond to selected books in the user's calibre library. This allows the user to add entries for desired books in calibre and then to check occasionally to see if they are carried by any Overdrive libraries at which they have borrowing privileges.

The plugin can also search for books from known authors that are available via Overdrive, but missing from the user's Calibre library. These can be selectively added as empty calibre books.

For the plugin to be useful the user must have borrowing privileges at a lending library powered by Overdrive. Check your local community library for possible access. In addition some libraries, such as the Free Library of Philadelphia, allow e-book borrowing by non-residents for a small fee.

Main Features:

• Searching of Overdrive for selected books from the calibre library.
• Searching by author for available Overdrive books that are missing from the calibre library.
• Checking if found books are available from specific lending libraries.
• Create links from calibre books to web pages from where they can be borrowed.
• Unlinking of books on demand.
• Selecting books based on the Overdrive links.
• Display of links in the Id section of the book details panel.
• Configuration of lending libraries and search parameters.
• Help display with additional information.

Version History:

Spoiler:

Version 1.5.0 - 14 Mar 2013
"Discard Existing" changed to "Update Existing" in the "Manage Search-by-authors results" dialog (Updates Overdrive links of corresponding calibre books that were manually added subsequent to a search.)
Optional configuration of a SOCKS proxy to be used when searching Overdrive
More frequent updates to search job status and progress

Version 1.4.0 - 13 Feb 2013
Support Overdrive library consortia that require a branch to be selected during sign in
Fix exception when changing the priority of a newly configured Overdrive library

Version 1.3.2 - 15 Jan 2013
Fix incorrect handling of authors in last,first format in some cases
Find book matches by including partial title when a search by only author exceeds the search limit

Version 1.3.1 - 06 Jan 2013
Fix exception when attempting to configure a new library

Version 1.3.0 - 29 Dec 2012
Optional sign in with user's credentials to detect additional books available through Overdrive Advantage
Handle further changes in the Overdrive website redesign
Extend timeouts for slow responding websites

Version 1.2.2 - 06 Dec 2012
Handle changes at Brooklyn Public Library website
Additional default sample libraries

Version 1.2.1 - 20 Sep 2012
Bug fix for occasional error in menu display

Version 1.2.0 - 19 Sep 2012
Generate the menu actions ahead of time for compatibility with keyboard shortcuts and the Favourites Menu plugin.
Show both regular search and search for other available books by author in the plugin menu.
Performance improvements in 'Manage Search-by-author results' dialog.

Version 1.1.0 - 06 Sep 2012
Added searching by author for available Overdrive books that are missing from the calibre library.
Improvements to the fuzzy matching algorithm for titles and authors.
Reorganization of configuration options.
Reorganization of search results report.

Version 1.0.2 - 14 Aug 2012
Bug fix. (Prevent possible crash!)

Version 1.0.1 - 14 Aug 2012
Bug fix. (object has no attribute 'ids')

Version 1.0.0 - 14 Aug 2012
Initial release.

Special Notes:

• Requires calibre version 0.8.61 or later.

Installation Notes:

• Download the attached zip file, install the plugin and restart calibre as described in the Introduction to plugins thread.
• Select Customize plugin from the plugin menu (or via Preferences -> Plugins).

Usage Details:

Spoiler:

Library Configuration

The plugin supports multiple Overdrive-powered lending libraries. Individual libraries and their relative priorities are set in the plugin's configuration. The default configuration contains some sample libraries that can be enabled for searching. Additional libraries can be added as needed using the (+) icon. Each library is identified by its host name (e.g.: freelibrary.lib.overdrive.com) and a short name for the user interface (e.g.: FLP). Configured libraries will only be used for searching and unlinking if they are enabled in the configuration by checking the last column.

The host name for an Overdrive-powered library can be found by examining the address bar after surfing to the library's web site.

Optional User Credentials

Overdrive Advantage allows individual lending libraries that are members of a consortium with a shared collection to have additional titles that are only available when their patrons are signed in. The plugin configuration has optional fields for the user's credentials for each lending library: branch ID number, library card number and PIN. Entering this data is not required and will only be useful if your lending library has Overdrive Advantage titles.

If you choose to configure your library credentials in the plugin, then a library card number is required. Some, but not all, libraries also require entry of a branch ID and/or PIN during sign in.

If your library normally prompts for the selection of a specific branch during sign in, the corresponding branch ID number will need to be entered as part of the configuration. This number can sometimes be found by observing the URLs being accessed during a manual sign in to the library. Look for something in the form of:

https://secure29.libraryreserve.com/library2go.lib.overdrive.com/80F91AB0-A5FE-41B7-8704-B2BEBE3A39E8/10/438/en/SignIn2.htm?branchid=120&URL=MyAccount.htm

As an alternative, you can enter your library card and PIN in the plugin configuration, but leave the branch ID field blank, check the last column to enable the library, and then perform a search for any book. The sign in process will fail, but the log of the search should contain a list of valid branches for the consortium along with their corresponding branch ID numbers.

Some libraries do not require a PIN or additional data field beyond the library card number. In those cases leave the PIN field blank in the configuration.

Searching Overdrive

To search for books at Overdrive first highlight the desired book entries within calibre and the choose Search Overdrive for selected books from the plugin's menu. Searches can take a long time and so are done as jobs. After the search completes a summary of results is shown. If accepted by the user, an identifier is added to the calibre book metadata for each match found to provide a link back to the lending library web page for that book. These links can be later used to open that web page, either through the plugin's menu or the book details panel.

Only one link per book will be shown under Ids: in the book's details, corresponding to the highest priority library from which the book is available. A plus sign after the library name indicates that the book is also available from other lending libraries (or in other editions) and the additional links can be found in the Overdrive Link plugin menu.

Additional Available Books, Missing from calibre

Selecting the plugin menu choice Search Overdrive for books by selected authors will cause the search to also check for any additional books by the author(s) of the selected books that are available from enabled Overdrive libraries, but are not present in the user's calibre book library.

After search completion, these results can be accessed using the Manage Search-by-authors results choice from the plugin's menu. A table of books is presented and the user can choose the action to be taken for each: add to the calibre library as an "empty" book with minimal metadata (author, title, series, publisher, odid and isbn), ignore so that undesirable books do not appear in subsequent searches, and discard which will eliminate the book from the search results, but allow it to appear again in subsequent searches.

Books previously marked to be ignored can be displayed again using the Show Ignored button. The Update Existing button can be used to check if any of the listed books were recently added to calibre and if so mark them to be updated in calibre (adding Overdrive links) and then discarded from the list of additional available books.

Viewing Links as a Custom Column

You can create a custom column to show books found to be available from an Overdrive library. Choose Preferences -> Add your own columns -> Add custom column. Enter a Lookup name of has_odid and a Column heading of Overdrive (or any other description you desire.) The Column type should be Column built from other columns, select Show checkmarks and Sort/search column by should be set to Yes/No. Enter the following in the Template field:

{identifiers:contains(odid,Yes,)}

You will need to restart calibre for the change to take effect.

Removing Links to Overdrive

Removing an Overdrive library from the plugin's configuration has no effect on books that have already been linked to that library. The only noticeable difference is that the links to an unconfigured library will be shown with a default name (based on the hostname) rather than the previously configured short name for the library. So selecting a book and then clicking the plugin's icon will always show the links established for that book, whether or not the library still exists in the configuration.

You can remove a link by either deleting the odid identifier under Ids in the Edit Metadata dialog for the book or by selecting Unlink selected book from the plugin's menu. (The unlink function only removes links for unconfigured and enabled libraries. It leaves unchanged any links to libraries that are configured, but not marked as enabled.)

Restore Default Configuration

The following steps can be used to restore the default configuration for the plugin: From the main Calibre screen select: Preferences -> Miscellaneous -> Open calibre configuration directory. Navigate to the plugins subdirectory and delete the file Overdrive Link.json.

Implementation Notes:

Spoiler:

The plugin works by first searching the main Overdrive website (search.overdrive.com) for matching e-books using the primary (first) author of the selected calibre books. The search results are then matched against the selected books. Matched books are checked against each configured Overdrive lending library to see if copies are available from that library and if so a link is created. If a library card number is entered in the configuration, the plugin will sign in with the user's credentials before checking book availability so that Overdrive Advantage titles can be found.

If the menu choice to search by author is selected then all books by the author found at Overdrive are checked for availability and those that are available, but not already present in the user's calibre book library, are retained for examination by the user. The list of these books is stored in a hidden part of the calibre library database.

Fuzzy matching is used to detect matches even when there are minor differences. The book matching process is not exact and may sometimes result in missing possible matches or association of the wrong books. A configurable confidence parameter controls the strictness of the matching process.

In matching titles it is assumed that text in parentheses or following a colon is an optional subtitle. A vertical bar ('|') can be used to separate alternate titles for the same book.

When new books are added to individual libraries they may not appear at the main Overdrive site for some time. Books will not be discovered by this plugin until they appear on that site. In addition, some searches return server errors from Overdrive. The plugin handles some common cases of this.

Links are stored in calibre as identifier metadata. The format is 'odid:uuid@host' where uuid is the 36 character identifier used by Overdrive to identify an edition of a book and host is the host name of the specific Overdrive library. If the book is available in multiple editions or from multiple libraries there will be additional uuid-host pairs, separated by ampersands. For example:

odid:01fffb99-bfe4-4210-8dd6-4fefcae83c45@freelibrary.lib.overdrive.com&01fffb99-bfe4-4210-8dd6-4fefcae83c45@ocls.lib.overdrive.com

A simple search for identifiers:"=odid:" can be done to find all books in calibre that have an Overdrive link. A search for identifiers:"=odid:freelibrary" will show only books available at that library. A set of these searches is available through the plugin's menu.

The Overdrive Link plugin contains an imbedded metadata plugin, Overdrive Link Helper, which provides the clickable links in the book details panel. This helper can be disabled in the main plugin's configuration if this feature is not desired.

[theducks]

Moderator Notice
Moved to plugins. The development forum is viewed mainly by developers

Just put your new plugins information (use existing PI thread as a model) into this Message with the plugin attached.

PM Me and I can change the title of this thread to what the PI is about (please supply the new subject line and I will paste it in place).

[jhowell]

Original post:

I have been working on a GUI plugin for Calibre to meet a need I have to find books of interest at my local e-book lending library, powered by Overdrive. I am continuing to tinker with the software, but it has reached the point where it is useful for me and might be of use to others.

This is my first software development using Python. I followed examples and borrowed code liberally from existing plugins. (Thanks kiwidude!) This plugin has only been tested under 64-bit Windows 7 with Calibre versions since 0.8.60, but I don’t see any reason why it shouldn’t work on the other platforms that Calibre supports.

[GRiker]

@jhowell - great idea!

I just installed your plugin, and added spl.lib.overdrive.com, and made it the first library in the list.

I then added an empty book in calibre, then populated it with 'A Short History of Nearly Everything' by Bill Bryson, then searched for the book.

Running in debug mode, I see that your plugin found the book with 100% confidence.

However, it then crashed with the following traceback:

Traceback (most recent call last):
  File "calibre_plugins.overdrive_link.dialogs", line 79, in do_book
AttributeError: 'ApplyProgressDialog' object has no attribute 'ids'

G

[jhowell]

Originally Posted by GRiker

...
However, it then crashed with the following traceback:
...

Thanks for the feedback. I had made a last minute change and should have tested it. A fix (version 1.0.1) is available in the first post.

[GRiker]

1.0.1 works as advertised.

G

[Gunnerp245]

Excellent! Will try after work...

[GRiker]

Removing libraries in the Customize dialog doesn't remove the corresponding entries from the plugin dropdown list.

G

Edit: Maybe I misunderstood the logic. Because the IDs were still associated with the book from the other libraries, maybe that's why they still show up, even though the other libraries were disabled.

[jhowell]

Originally Posted by GRiker

Removing libraries in the Customize dialog doesn't remove the corresponding entries from the plugin dropdown list.

G

Edit: Maybe I misunderstood the logic. Because the IDs were still associated with the book from the other libraries, maybe that's why they still show up, even though the other libraries were disabled.

Removing an Overdrive library from the plugin's configuration has no effect on books that have already been linked to that library. The only noticible difference is that the links to an unconfigured library will be shown with a default name (based on the host name) rather than the previously configured short name for the library.

So selecting a book and then clicking the plugin's icon will always show the links established for that book, whether or not the library still exists in the configuration.

You can remove a link by either deleting the 'odid' identifier under Ids in the Edit Metatdata dialog for the book or by putting the library back into the configuration and selecting 'Unlink selected book...' from the plugin's menu. (The unlink function only removes links for configured libraries. It leaves links to unconfigured libraries alone. I may rethink that in a future release.)

[GRiker]

If you don't normally keep the Book Details panel open (I don't), here's how to create a custom column to show books identified as available at your library by this plugin.

1) Create a custom column with a lookup name of 'at_library' and a Column heading of 'Available at Library'. Column type should be 'Column built from other columns', and the 'Show checkmarks' checkbox should be enabled.

2a) If you want the checkmarks associated with a particular Overdrive library, enter the following in the template field:

{identifiers:contains(ocls.lib,Yes,)}

'ocls.lib' should be replaced with the first part of the web address of your library. You can see these addresses in the Customize dialog of the plugin. They are unique for each Overdrive library.

2b) If you want the checkmarks associated with any discovered Overdrive books, enter the following in the template field:

{identifiers:contains(odid,Yes,)}

3) 'Sort/search column' should be set to 'Yes/No'

After creating the custom column, you will need to restart calibre for the changes to take place.

After restarting, any books identified as available from your local library by the Overdrive Link plugin will be shown with a checkmark.

G

[jhowell]

Found a bug that could potentially cause a crash of Calibre. Released version 1.0.2.

[Gunnerp245]

@jhowell - I like it!

[Gunnerp245]

Originally Posted by GRiker

If you don't normally keep the Book Details panel open (I don't), here's how to create a custom column to show books identified as available at your library by this plugin. G

@GRiker - Thanks! I was going to ask how to do just that. But what if more than one Overdrive library was used?

@jhowell - Couldn't this be added to the plugin so that it would be automatic and not require manual configuration.

[GRiker]

Originally Posted by Gunnerp245

@GRiker - Thanks! I was going to ask how to do just that. But what if more than one Overdrive library was used?

You can change the logic of the template to show a check mark whenever there's any odid content associated with the book:

{identifiers:contains(odid,Yes,)}

I'll edit my original post to reflect this alternative.

G

[jhowell]

Originally Posted by Gunnerp245

@GRiker - Thanks! I was going to ask how to do just that. But what if more than one Overdrive library was used?

@jhowell - Couldn't this be added to the plugin so that it would be automatic and not require manual configuration.

I’d also like to thank GRiker for the custom column idea. It does make the presence of library links much more visible.

Regarding the automatic creation of a custom column for library availability: While several existing plugins rely on custom columns, I don’t know of any that automatically creates a custom column. Perhaps someone with more experience with the internals of Calibre could comment on this.