View Single Post
Old 08-13-2012, 11:14 PM   #1
jhowell
Guru
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 940
Karma: 7832731
Join Date: Nov 2011
Location: Florida
Device: iPad Air 2, Kindle PW2, Nexus 7, Fire 5th Gen
[GUI PLUGIN]Overdrive Link

UI Plugin to Link Books in calibre to Lending Libraries

Many communities use OverDrive, (Baker & Taylor) Axis 360, (Bibliotheca/3M) Cloud Library, EBSCOhost, Enki Library, Freading, Hoopla and OneClickdigital to handle their lending library services for e-books and audiobooks. Some lending libraries use multiple service providers, with different book collections available though each. E-book lending is also available from Amazon, Open Library and Scribd. Free public domain e-books are downloadable from the Internet Archive (via Open Library) and Project Gutenberg. Checking libraries for books of interest can be tedious for large numbers of books and multiple libraries, especially since libraries add to their collections over time.

The purpose of the Overdrive Link user interface action plugin is to search lending libraries for borrowable e-books and audiobooks that correspond to selected books in the user's calibre library. This allows the user to add entries for desired books in calibre and then to check occasionally to see if they are carried by any lending library at which they have borrowing privileges.

The plugin can also detect books that are not available in the lending library's collection, but can be recommended for future acquisition by the library, and provides access to the web page from which the recommendation can be made.

The plugin can also discover books from known authors or about particular subjects missing from the user's calibre library. These can be selectively added as "empty" calibre books.

Once a book is found and linked the plugin can determine later on demand whether it is currently available to be borrowed or provide an estimate of the expected wait time (in weeks) if not.

The plugin is able to search a limited number of on-line bookstores to determine whether a title is available in e-book form and provides links to the web pages from which purchases can be made.

For the plugin to be useful the user must have borrowing privileges at a lending library powered by one of the supported service providers. Check your local community library for possible access. In addition some libraries, such as the Free Library of Philadelphia, allow borrowing by non-residents for a small fee.


Main Features:
  • Searching of OverDrive, Amazon, Audible, Axis 360, Cloud Library, EBSCOhost, Enki Library, Freading, Hoopla, OneClickdigital, Open Library, Project Gutenberg and Scribd powered lending libraries for e-books and audiobooks matching selected books from the calibre library.
  • Detection of books that can be recommended to a library for acquisition or purchased from a store.
  • Detection of books by known authors that are missing from the calibre library.
  • Detection of books that are missing from the calibre library using topic keywords.
  • Creating links from calibre books to web pages from where they can be borrowed, recommended or purchased.
  • Display of links in the Id section of the book details panel or the plugin menu.
  • Searching for and selecting calibre books based on previously established links.
  • Checking the current availability of linked books to be borrowed and estimation of hold/wait time for books that are not immediately available.
  • Unlinking of books on demand.
  • Configuration of lending libraries and search parameters.
  • Help display with additional information. (This document.)

Usage Details:
Spoiler:


Library Configuration

Library and other configuration options can be accessed using the Customize plugin menu choice.

The plugin supports multiple lending libraries. Individual libraries and their relative priorities are set in the plugin's configuration. Each library is identified by a short, unique name (chosen by the user), its service provider and a library-id. Configured libraries will only be used for searching if they are enabled in the configuration by checking the first column. Groups of libraries may be enabled as a set by configuring a plugin tweak.

The default configuration contains some sample libraries that can be enabled for searching. Additional libraries can be added as needed using the (+) button and existing libraries can be removed using the (-) button.

Before performing searches at least one library must be enabled and there must be no configuration errors (shown in red below the lending library table).


OverDrive

OverDrive is the largest provider of lending library services. Books are available in EPUB and PDF formats via Adobe Digital Editions and Kindle formats via Amazon in the US. Audiobooks are also available. Some OverDrive libraries allow recommendation of books for future acquisition.

You can find libraries supported by OverDrive at http://www.overdrive.com. For OverDrive powered libraries the library-id is the host name of the library web site (e.g.: freelibrary.overdrive.com for the OverDrive site of the Free Library of Philadelphia). This can be found by examining the browser address bar after surfing to the library.

Some OverDrive libraries require a user to sign in before the catalog of the library can be searched. Also, OverDrive Advantage allows individual lending libraries that are members of a consortium with a shared collection to have additional titles that are only available when their patrons are signed in.

The plugin configuration has optional fields for the user's credentials for each lending library: branch ID number, library card number and PIN. Entering this data is not required. However it be useful if your lending library has OverDrive Advantage titles or requires a sign in before allowing searches. Having credentials will also allow the plugin to provide more accurate reporting of current availability for already linked books.

OverDrive supports three types of sign-in: using credentials specific to each individual library, using an OverDrive account, and using Facebook. Only the first two types are supported by this plugin.

If you choose to configure your individual library credentials in the plugin, then a library card number is required. Some, but not all, libraries also require entry of a branch ID and/or PIN during sign in. Other libraries do not require a PIN or additional data field beyond the library card number. In that case leave the PIN field blank in the configuration.

If your library normally prompts for the selection of a specific branch during sign in, the corresponding branch ID number will need to be entered as part of the configuration. To find this number enter your library card and PIN in the plugin configuration, leave the branch ID field blank, check the first column to enable the library, and then perform a search for any book. The sign in process will fail, but the log of the search will contain a list of valid branches for the consortium along with their corresponding branch ID numbers.

If you have an OverDrive account you can use it to sign in to any OverDrive library linked to that account. Enter overdrive in the branch ID column, your email address in the library card column, and password in the library PIN column. (This type of sign in is not supported for libraries still using the old OverDrive website design.)

Important Note

A major change to the OverDrive website design is being rolled out to lending libraries in 2016. With this redesign the identifiers associated with books will change, making links to book web pages partially incompatible with the old website design. Links previously created by searching will still function after a library migrates to the new design, but future searches will create different links, resulting in both old and new links existing for the same book. The Check current availability of selected books menu choice can be used to remove old-style links and replace them with the equivalent new-style.

Usually the library-id (website host name) of an OverDrive library will change when the library is migrated. The plugin detects host name redirection during search and notifies you by an error message that a configuration change is needed. After updating the configuration you should use the menu choice Check and repair book links to migrate existing book links to the new library-id. (The plugin checks for host names changed by adding or removing ".lib.", which is what OverDrive does typically. If the host name is changed in some other manner it may not be possible to automatically migrate links and they will need to be re-established by performing new searches.)

The plugin detects and keeps track of the website type (old or new) for each lending library. This may sometime fail causing books links to work incorrectly. If this occurs a restart of calibre should correct the problem. Once all libraries have migrated to the new design the plugin will be updated to remove support for old websites and then this will no longer be an issue.


Amazon

Amazon Kindle Unlimited allows borrowing up to ten e-books at a time with no holds or waits for a monthly fee. Prime Reading offers a small subset of the Kindle Unlimited catalog to Amazon Prime members in the US. The Amazon Kindle Owners' Lending Library is available to Amazon Prime members and allows borrowing of one e-book per month on a registered kindle device. These services offer a limited collection of mostly self-published books from which to choose.

When configuring, the library-id should be prime for the Kindle Owners' Lending Library, reading for Prime Reading, unlimited for Kindle Unlimited, kuwn for Kindle Unlimited with Narration, w4v.store for purchasable e-books supporting Whispersync for Voice, or store for all purchasable Kindle e-books. Prefix the library-id with uk- for Amazon UK, with ca- for Amazon Canada, or with au- for Amazon Australia. Multiple collections can be searched by creating a separate library entry for each desired collection in the configuration table.

The general configuration option Set 'amazon' identifier for Kindle e-books causes the amazon book identifier in calibre to be updated to point to kindle editions discovered during search. An alternate identifier can be used by configuring a plugin tweak.

Currently, only the US Amazon site at http://www.amazon.com the the UK Amazon site at http://www.amazon.co.uk, the the Canadian Amazon site at http://www.amazon.ca, and the the Australian Amazon site at http://www.amazon.com.au are supported by the plugin. Searching the Kindle Owners' Lending Library is only supported for Amazon US. Only the store and unlimited collections are supported for Amazon Canada and Amazon Australia.


Audible

Audible allows the purchase of audiobooks in a proprietary format. Only audiobooks are supported, not radio programs or other media.

When configuring, the library-id should be store and the "Other/app/player" audiobook format must be enabled for searching.

Currently, only the US Audible site at http://www.audible.com is supported by the plugin.


Axis 360

Axis 360 provides access to a variety of titles, including a large collection of children's books. E-books and audiobooks are available using a proprietary app. Some libraries also have books available in EPUB and PDF format via Adobe Digital Editions.

Find libraries using http://axis360.site.baker-taylor.com/FindALibrary.aspx. The library-id can be found by inspecting the host name in the browser address bar for the portion preceding axis360.baker-taylor.com. For the Orange County (Florida) Library System the Axis 360 web site is http://ocls.axis360.baker-taylor.com/ and so the library-id is ocls.


Cloud Library

Many lending libraries use (Bibliotheca) Cloud Library in addition to, or instead of, OverDrive. (This service was formerly known as 3M Cloud Library.) Books are available in EPUB and PDF formats via an application compatible with Adobe Digital Editions. Audiobooks are also available from some libraries. Cloud Library supports recommendation of books for future acquisition at some libraries.

You can find libraries supported by Cloud Library by using Lookup your library at http://www.yourcloudlibrary.com/index.php/en-us/. The library-id can be found by going to the library web site and examining the browser address bar for the portion of the URL following /library/. For the Brooklyn Public Library the web site is http://ebook.yourcloudlibrary.com/library/bpl/ and so the library-id is bpl. The card number and pin configuration fields are optional.


EBSCOhost

EBSCOhost provides research databases mainly for academic libraries, but also supports e-books and audiobooks. These are mostly reference books and public domain titles, but can include popular books published by Random House. Books may be read online and some are available to download in EPUB or PDF format via Adobe Digital Editions. Audiobooks formats are MP3, WMA and M4B.

Access to the catalog of books for a library requires authentication. EBSCOhost supports a variety of authentication methods and the correct one must be used for each library. Some libraries require the user to be physically present for access. Some require a patron id to be assigned by library staff. Some require the user to sign in at the main library site, which then redirects the web browser to EBSCOhost.

The branch-id in the configuration selects the type of authentication used to sign in to the EBSCOhost library. It may take some experimentation to determine the authentication needed for a particular library. Only a subset of the authentication methods used by EBSCOhost are supported by the plugin.
  • athens - Athens Authentication - Not supported
  • cpid - Patterned ID - Library card number and pin.
  • custuid - Patron ID - Library-specific user id and password.
  • ip - IP Address - Sign in allowed only from a computer located at the library or via VPN.
  • lib=http://... - Library controlled - A login page at the main library site redirects to EBSCOhost after sign in. Login pages vary by library. This plugin may need changes to support some of these.
  • shib - Shibboleth Authentication - not supported
  • uid - User ID and Password - EBSCO user id and password
  • url - Referring URL - Use lib=... for this type of authentication.
  • user - Personal User Authentication - EBSCO user id and password

The library-id must match the EBSCOhost customer id assigned to the library. This can sometimes be obtained from the URL used for library access. For example, Fairfax County Public Library is accessed via:

http://search.ebscohost.com/CPIDLogin.asp?custid=s4132146&profile=ehost&defaultdb=nlebk

The card-number and pin fields in the configuration must be set to the user's id and password for the EBSCOhost library.

Due to the need for authentication, clicking links created by the plugin in calibre to open a book's page at an EBSCOhost library will not work if the user is not already signed in to the proper library in their web browser. It also will not work if EBSCOhost is accessed via a proxy URL provided by the library.


Enki Library

Enki Library provides access to a collection of mostly indie-publisher e-books available through multiple library systems in California. Books are in EPUB and PDF formats, delivered via Adobe Digital Editions.

The library-id can be found by inspecting the host name in the browser address bar for the portion preceding enkilibrary.org. For the Anaheim Public Library the Enki web site is http://apl.enkilibrary.org/ and so the library-id is apl. The card number and pin should be left blank in the configuration.


Freading

Freading provides access to many older titles. Books are available in EPUB and PDF formats via Adobe Digital Editions. It allows a set number of checkouts per week, based on a credit system, with no holds or waits. It appears that the same e-book collection is available at all libraries, however this may not always be the case.

Find libraries using http://www.freading.com/libraries/chooser. The library-id can be found by inspecting the host name in the browser address bar for the portion preceding freading.com. For the Free Library of Philadelphia the Freading web site is http://freelibrary.freading.com and so the library-id is freelibrary. The card number and pin configuration fields are optional.


Hoopla

Hoopla provides access audiobooks and a small collection of e-books and comics (and also music and videos) using a web browser or a proprietary app. It allows a set number of checkouts per month with no holds or waits. It appears that the same collection is available at all libraries, however libraries can block access to specific media types or more expensive titles to reduce cost. Users must register using "Get Started" at http://www.hoopladigital.com using their library credentials and establish an account (email address and password) to use the system.

If the library-id, card number and pin are all left blank in the configuration then searches will be performed without signing in to Hoopla. Users may instead configure the plugin to sign in to search based on their specific library's collection. The card-number and pin fields in the configuration must be set to the user's email address and password registered with Hoopla. If the library-id is left blank in the configuration, the plugin will detect and report the proper library-id when a search is attempted. The library-id in the configuration should then be changed to the proper one.


OneClickdigital

OneClickdigital provides access to audiobooks and a small selection of e-books. Audiobooks are playable using a proprietary app. E-books are in EPUB format using Adobe Digital Editions DRM.

The library-id can be found by inspecting the host name in the browser address bar for the portion preceding oneclickdigital.com. For the Austin (TX) Public Library the OneClickdigital web site is http://austinpublictx.oneclickdigital.com and so the library-id is austinpublictx. For those outside of the US enter the full host name as the library-id. For example, essex.oneclickdigital.eu.


Open Library

Open Library provides access to a very large collection of books. Books under copyright are available for borrowing in EPUB and PDF formats via Adobe Digital Editions. Public domain books are available in a variety of open formats. The PDFs contain scanned page images from print books and the other formats are generally of low quality with many OCR errors.

The library-id should be set to the collection to be searched:
  • read - A very large collection of older public domain books readable online and downloadable without borrowing.
  • lendinglibrary - Lending Library is a small collection of public domain books available for anyone to borrow.
  • inlibrary - In Library is a large collection of books available for borrowing by patrons of specific libraries. Participating libraries can be found at https://openlibrary.org/libraries.
  • printdisabled - This collection contains accessible books in protected DAISY format for those with print disabilities.
The card number and pin are configuration fields are optional.


Project Gutenberg

Project Gutenberg provides access to a large collection of public domain books, available in a variety of open formats.

Automated searching of Project Gutenberg is not allowed. Once the Project Gutenberg library is configured, a plugin menu choice, Build index for Project Gutenberg will appear. When used, it will download the catalog of available e-books from Project Gutenberg and process it to produce an index stored on the user's system. This index will be used for subsequent searches. (The downloaded catalog, rdf-files.tar.bz2, is about 35MB and the index file,Overdrive Link Project Gutenberg Index.json.bz2, will be about 2MB.)

The Project Gutenberg catalog is updated daily and should not be downloaded more than once per day.

The library-id should be left blank for Project Gutenberg.


Scribd

Scribd provides access to e-books and audiobooks via a web browser or a proprietary app. It allows unlimited reading from a small collection and limited monthly selections from a large collection with no holds or waits for a monthly fee. Book availability varies by country.

The library-id should be left blank in the configuration. Users can see the titles available to them by setting the card-number and pin fields in the configuration o the user-id and password registered with Scribd. If the card number and pin are all left blank in the configuration then searches will be performed without signing in and Scribd will attempt to determine your country of origin automatically.

This plugin supports only published e-books and audiobooks at Scribd, not user uploaded documents. Login with Facebook credentials is not supported.


Searching for Books

To search for books at configured libraries first highlight the desired book entries within calibre and the choose Search for selected books from the plugin's menu. Searches can take a long time and so are done as jobs. After the search completes, a summary of results is shown. If accepted by the user, identifiers are added to the calibre book metadata for each match found to provide a link back to the lending library web page for that book. These links can be later used to open that web page, either through the plugin's menu or the book details panel.

The Select previously linked books / Available books plugin menu item shows books available for borrowing from any of the configured lending libraries. When selected, these books will have a plugin menu choice of Borrow book from libraryname or Borrow audiobook from libraryname which opens a web browser to the page from which the user can borrow that book.

Up to 10 links per book will be shown under Ids: in the book's details, corresponding to the highest priority libraries (first in configuration) from which the book is available. A plus sign after a library name indicates that the book is also available from more lending libraries (or in other editions) and the additional links can be found in the plugin menu. A '#' before the library name indicates that the link is to an audiobook.


Discovering Books Missing from calibre

Selecting the plugin menu choice Search for books by selected authors will cause the search to look for the specific books selected and to also save a list of books by the author(s) of the selected books that are found at enabled lending libraries, but not present in the user's calibre library. (Note: Calibre books by the same author(s) that are not selected when a search is started are also included in search.)

By default only the primary (first) author of each book is used for searching. A configuration option (Max authors to search per book) can be used for increase the number of authors searched for books with multiple authors, such as collaborations and anthologies. This increases the likelihood of finding a match at some providers (such as Open Library), but slows the search process. The additional authors will also be used for the discovery of new books. (All of the authors of a book are considered during matching of books found by search, regardless of this configuration option.)

Selecting the plugin menu choice Discover books by keyword will cause the search to look for books that match the selected keywords(s) in their title (or description for some providers), but are not present in the user's calibre library. Keywords are configured using Customize plugin, one set of keywords per line. Examples of keywords are sherlock holmes and megapack. The discover-by-keyword menu choice will not appear unless at least one set of keywords is configured.

After search completion, these results can be accessed using the Manage discovered books choice from the plugin's menu. A table of books is presented and the user can choose the action to be taken for each:
  • No Action leaves the discovered book alone.
  • Add Book to the calibre library as a new "empty" book with minimal metadata (author, title, series, publisher, publication date, isbn, odid, odrid and odpid).
  • Update an existing calibre book with links from the discovered book. This is useful if the discovered book should have matched an existing calibre book, but failed to do so due to a variation of the title. When Update is selected the plugin will provide a list of suggested calibre books to choose from.
  • Ignore hides the book so that undesirable books do not appear in subsequent searches.
  • Discard will eliminate the book from the search results, but allow it to appear again in subsequent searches.
Books previously marked to be ignored can be displayed again using the Show Ignored button. The Discard Dupes button marks apparent duplicate discovered books to be discarded. (All duplicates are marked, change any you want to keep back to No Action or Ignore.) The Update Match button can be used to check if any of the listed books were recently added to calibre and if so mark them to be updated in calibre (by adding links and missing metadata) and then discarded from the list of discovered books.

A search string may be entered to filter books so that only those that match are displayed. Searches are based on the main calibre search interface and support many of its features including exact match (=text), contains (text), regular expression (~pattern), presence (true), and absence (false). The fields allowed for searching are: author, title, series, library, isbn, publisher, language, action (no action, add book, ignore, discard, or update), ignored (true or false), and search (name of a previously saved search). For example to only display books by an author named John available at BPL use: author:john and library:=@bpl

Saved searches cannot be created by this plugin, but it can use those created by the main calibre search interface.


Check Current Availability of Linked Books

The plugin menu choice Check current availability of selected books will check previously linked books in order to determine whether they are currently available to be borrowed and if not provide an estimate of the expected hold/wait time in weeks. Choosing this will cause the plugin to check each of the borrowable links for the selected books. (These links must have been previously established by using the search capability of the plugin.) All links will be checked, even those that that correspond to libraries that are not currently enabled for searching in the configuration. Any links that are found to be for books no longer available at all will be removed. Results are presented in the summary popup at job completion.

A configuration option, Update current availability of books with new links will also cause a check for the current availability/wait to be done for books whenever a new link is found (or a possible link removal is detected) during a search.

A custom column may optionally be created by the user in the calibre library in order to save the results. Choose Preferences -> Add your own columns -> Add custom column. Enter a Lookup name of ol_wait_weeks and a Column heading with any desired label. The Column type must be Text, column shown in tag browser. You will need to restart calibre for the new column to take effect.

The custom column will be populated with a series of comma separated entries, one for each link. Each entry is of the form nn@LIBNAME, where nn is the estimated wait time for the book in weeks and LIBNAME is the configured library name for that link. A wait time of "00" indicates available now, "99" weeks is the maximum that can be shown, and "XX" indicates that the availability could not be determined for that link. The library name will be prefixed by "#" if the link is to an audiobook. Links are sorted by wait duration, none to low to high, so that sorting the calibre library by this column by clicking on the column header will sort books in order of current availability.

The ability of the plugin to estimate availability and wait time varies based on the library provider. Providing credentials in the configuration to sign into a library will add further information for some providers as described below:
  • Amazon Kindle Unlimited and Kindle Owner's Lending Library: Shows released books as always available and takes into account the release date of pre-release books.
  • Axis360: Provides current availability or estimated wait based on the existing number of holds.
  • Cloud Library: Provides current availability or estimated wait. Providing credentials to sign-in will also take into account the user's current position on the hold list for the book, if any.
  • Freading: Shows released books as always available. Will take into account the release date of pre-release books only if sign-in credentials are provided and the user has unused tokens.
  • Hoopla, Project Gutenberg, Open Library (read and printdisabled collections): Shows all books as always available.
  • Open Library (lendinglibrary and inlibrary collections): Provides current availability or estimated wait based on the existing number of holds. Providing credentials to sign-in will also take into account the user's current position on the hold list for the book, if any.
  • OverDrive: Provides current availability or estimated wait based on the existing number of holds. Also takes into account the date of availability for pre-release books. Providing credentials to sign-in will also take into account OverDrive Advantage copies and the user's current position on the hold list for the book, if any.
  • Project Gutenberg: Shows books as always available.
  • Scribd: Shows unrestricted books as always available. Shows books that require an account credit as having a one week wait.
  • Audible, EBSCOhost, Enki Library and OneClickdigital: Not yet supported.

Recommending Unavailable Books for Library Acquisition

The search configuration option Check for books that can be recommended for acquisition will cause subsequent searches to find books not currently in the library's collection, but potentially available to recommend for acquisition. (Some books may be detected that are not truly available for library acquisition due to restrictions by the publisher, such as geographic restrictions.)

The Select previously linked books / Recommendable plugin menu item shows books not currently available in any of the configured lending libraries. When selected, these books will have a plugin menu choice of Recommend book to libraryname which opens a web browser to the page from which the user can issue a recommendation for the library to acquire that book.

Some libraries do not support book recommendations.


Searching for Books Available for Purchase

When the Amazon kindle book store is configured as a search library, the books that are found are listed by the plugin as purchasable.

The Select previously linked books / Purchasable books plugin menu item shows books linked by this search. When selected, these books will have a plugin menu choice of Purchase book from libraryname which opens a web browser to the page from which the user can purchase that book. This can be used to quickly determine which calibre book entries are available in Kindle format via Amazon.


Author Name Variants

The plugin searches for matching books using one or more authors of the selected calibre books. Searches are done multiple times with variants of the author's name to improve results, e.g., replacing middle names with initials. This usually works well, but can produce extraneous results for common author names and can miss results for pseudonyms and unusual name variants.

In order to provide more control over author name searching and matching the plugin configuration has a place for optional Author name variants. These are entered one author per line with the author name as it appears in your calibre library first and any variants on the name to be used for searching following, separated by '&' characters. For example: Stephen King & Richard Bachman

If only a single name is entered then that only that name will be used in searching and no variants will be generated automatically. For example, entering just John Joseph Adams will cause searches for books by that author to exclude books by John Adams or John J. Adams.

Placing an author name in square brackets will prevent searches using that name, even if books by that author are selected for searching. However books that are found while searching for another author will still be found even if the prevented author happens to be a co-author. For example: [Deepak Chopra]

Placing an author name in angle brackets will prevent discovery of new books by that author. Matching of any existing calibre books by that author will still be allowed. For example: <Mark Twain>

Placing an author name in curly brackets will completely exclude books by that author, even as a co-author. For example: {SparkNotes}

For a match to occur between a calibre book and a library book they must have at least one listed author in common.


Title Variants

You can adjust the titles of your books in calibre to improve matching by the plugin. When matching book titles, text in parentheses, following a colon (':'), or preceding a double colon ('::') is optional. A vertical bar ('|') can be used to separate alternate titles for the same book. Some examples of usage:

A book with an optional subtitle with match with or without it: The Ancestor's Tale: A Pilgrimage to the Dawn of Life

A book that was published under multiple titles can match either one: The Barbie Murders (And Other Stories) | Picnic On Nearside

A book title that has an optional prefix: (The Adventures of) Huckleberry Finn

A book title sometimes prefixed with the series name in library catalogs will match either way if entered in calibre with a double colon separator: High Druid of Shannara:: Jarka Ruus

Note that in book matching the plugin will treat the the series name as an optional prefix to the title in order to properly match books that are listed that way at libraries, so the above case is usually handled automatically.


Caching of Book Details

As part of searching, the plugin accesses the web page for each book found by the search in order to obtain all of the information needed for filtering results and matching against calibre books. A cache of library book details is kept in a database file in order to avoid accessing these pages multiple times. Using the cache provides much faster results when searches for books are repeated, but introduces a small chance that outdated information may be used if the book details are updated at the library, which rarely happens.

In order to balance speed against accuracy the plugin defaults to keeping book information in the cache for a maximum of 90 days. This can be changed using the plugin configuration option Days to cache library book info. Setting this to zero disables the cache and setting it larger causes information to be kept longer.


Display of Links

Links to libraries are shown in search results and in the discovered book table with the following special characters: "@" indicates a link to an available book, "!" indicates a link to a recommendable book, and "$" indicates a link to a purchasable book. Links to audiobooks contain "#", while links to e-books do not.


Viewing Links as a Custom Column

You can create a custom column in calibre to show books found to be available from a lending library. Choose Preferences -> Add your own columns -> Add custom column. Enter a Lookup name of has_odid and a Column heading of OD (or any other description you desire.) The Column type should be Column built from other columns, select Show checkmarks and Sort/search column by should be set to Yes/No. Enter the following in the Template field:

{identifiers:contains(odid:,Yes,)}

Replace odid with odrid for recommendable books or odpid for purchasable books.

You will need to restart calibre for the change to take effect.


Removing Links to Lending Libraries

Removing a lending library from the plugin's configuration has no effect on calibre books that have already been linked to that library. The only noticeable difference is that the links to an unconfigured library will be shown with a default name (based on the provider and library-id) rather than the previously configured name for the library. So selecting a book and then clicking the plugin's icon will always show the links established for that book, whether or not the lending library still exists in the configuration.

The plugin menu choice Check current availability of selected books will detect links for books that are no longer present in a library's collection and automatically remove them for some library providers. Recommendable links will be automatically unlinked if a search finds that the book has become available in the library collection. Purchasable links must be manually unlinked.

You can manually remove links by selecting Unlink selected books from the plugin's menu. Submenu choices allow removal of all links, links for currently enabled libraries, or individual links (when a single book is selected). You can also delete the odid, odrid and odpid identifiers under Ids in the Edit Metadata dialog for a book to remove all links.

Links that are damaged or outdated can also be removed by selecting Check and repair book links from the plugin's menu.


Configuring Tweaks

The plugin has a number of configuration settings for advanced users that are managed by setting "tweaks" for the plugin. From the main Calibre screen select: Preferences -> Tweaks -> Plugin tweaks. This will open a dialog where tweaks may be set by adding Python variable assignments. The supported tweaks are:

overdrive_link_amazon_ident = 'mobi-asin' controls the identifier used for Amazon books when the Set 'amazon' identifier for Kindle e-books configuration option is enabled. The default identifier is country based: 'amazon', 'amazon_uk', etc.

overdrive_link_avg_loan_weeks = n This changes the duration in weeks that loans are assumed to last on average. It is used in the calculation of the ol_wait_weeks custom column. The default is 2.5. The value applies to all libraries of all types.

overdrive_link_disable_ssl_verification = True
This allows SSL certificate verification to be disabled within the plugin. Doing this will eliminate the SSL: CERTIFICATE_VERIFY_FAILED error which can occur with version 2.16 or newer of calibre. It is recommended the the underlying cause of the certificate problem be corrected and this tweak removed as soon as possible since it may indicate a security problem with the user's system.

overdrive_link_ignore_login_failure = True
This will cause searches to be performed even if sign in to a library fails. It can be useful in rare cases where library sign in is found to be unreliable.

overdrive_link_library_groups = dict mapping group names to lists of library names
This establishes named groups of libraries that can be used to perform searches. It can be used to avoid having to reconfigure the plugin to enable and disable libraries when only a subset of libraries is desired for a search. Once this tweak is set a new plugin menu item, Set library group will appear that lists the groups established by this tweak. Selecting a group will configure the plugin to enable only the libraries named in the group. For example, using: {'Favorite Libraries': ['lib1', 'lib2', 'lib3'], 'Others': ['lib4']} will establish two groups of libraries. Library names are case sensitive and must match those in the configuration exactly.

overdrive_link_max_book_panel_links = n
This changes the maximum number of links to show in the book details panel to n (a number). The default is 10. (Requires calibre 2.19 or later.)

overdrive_link_wait_groups = list of lists of library names
This establishes groups of libraries to override the default sorting of the ol_wait_weeks custom column, which is by number of weeks and then by library name. Once set the sorting order changes so that any links to libraries in the first group come before any in the second, and so on. Any information for libraries not contained in any group is discarded. The value is a list of lists containing the quoted names of libraries as set in the configuration. A library name by itself refers to links to e-book at that library, a library name prefixed by a number sign (#) represents links to audiobooks at that library. For example, using: [['lib1', 'lib2', 'lib3'], ['lib4'], ['#lib4']] will sort ol_wait_weeks so that e-book links to lib1, lib2 or lib3 will come first (or order of wait time), then e-book links to lib4 and finally audiobook links to lib4. Library names are case sensitive and must match those in the configuration exactly.

Additional tweaks are also implemented in the plugin for testing. These are unsupported and may be changed or removed in future plugin releases without notice.


Restore Default Configuration

The following steps can be used to restore the default configuration for the plugin: From the main Calibre screen select: Preferences -> Miscellaneous -> Open calibre configuration directory. Navigate to the plugins subdirectory and delete the file Overdrive Link.json.

The library book information cache can be cleared by deleting the file Overdrive Link Cache.db in the same directory.

The Project Gutenberg index file, Overdrive Link Project Gutenberg Index.json.bz2, is also in the same directory and may be safely deleted.

Tweaks set for the plugin can can be removed using: Preferences -> Tweaks -> Plugin tweaks and deleting any that begin with overdrive_link.


Behind the scenes

The plugin searches each configured library website for matching e-books and audiobooks using variants of the name of the author of the selected calibre books. By default only the primary (first) author is searched, but a configuration option (Max authors to search per book) allows searching for multiple authors for the same book.

If the configuration option to search for recommendable books has been selected then an additional search to find those books is done, increasing search time.

The results returned by a library search request usually do not contain enough information to properly identify books, so a request is made to get the details for each book, unless they were already cached due to a prior search.

The search results are matched against the selected books and a link is created for each matched book. Fuzzy matching is used to detect matches even when there are minor differences. This may sometimes result in missed matches or association of the wrong books.

If the menu choice to save discovered books was used then any search results that do not match books already present in the user's calibre library are retained for examination by the user.

Links to books discovered at libraries are stored in calibre as identifier metadata. The identifier for a book available in a library's collection is odid, the identifier for a book recommendable for acquisition by a library is odrid and the identifier for a book that can be purchased by the user is odpid.

The format of these is book_id@provider_id/library_id for e-books and book_id#provider_id/library_id for audiobooks, where book_id is the identifier used by the provider to identify an edition of a book (e.g.: an integer or a 36 character content reserve identifier for OverDrive), provider_id identifies the provider of library services (none for OverDrive Legacy, '3m' for Cloud Library, 'ak' for Amazon, 'ax' for Axis 360, 'eh' for EBSCOhost, 'en' for Enki Library, 'fr' for Freading, 'ho' for Hoopla, 'ocd' for OneClickdigital, 'ol' for Open Library, 'sc' for Scribd) and library_id identifies the specific library.

If a book is available in multiple editions or from multiple libraries there will be additional links, separated by ampersands. For example:

odid:1381708@freelibrary.overdrive.com&ZnJlYWQ6OTc 4MDQ4NjExMDUzMDpl@fr/ocls&dot8az9@3m/bpl

A search for identifiers:"=odid:" can be done to find all books in calibre that are available at any lending library. A search for identifiers:"=odrid:@freelibrary.overdrive.com" will show only e-books recommendable at that specific library. Commonly used searches can be performed through the plugin's menu under Select previously linked books. Regular expressions may need to be used for more complex search cases.

The list of discovered books is stored in a hidden part of the calibre library database. This data can be examined in the file metadata_db_prefs_backup.json in the calibre library folder.

The plugin contains an embedded metadata plugin that provides the clickable links in the book details panel for available books (and optionally purchasable books.) It can be disabled in the main plugin's configuration if this feature is not desired.


Version History:
Spoiler:


Version 2.5.0 - 06 Dec 2016

Fix search at Scribd not finding any books due to website change.

First attempt to support external authentication for new OverDrive sites. Untested and will probably need further work.


Version 2.4.0 - 02 Dec 2016

Fix error signing in to Cloud Library. (Fixes "KeyError (SessionId)")

Handle books that have been removed from specific libraries at Freading. (Fixes "Server response error (missing ".//div[@class='book-info']")")

Prevent caching of details for unavailable books at new OverDrive sites since they sometimes have incorrect format information.

Update the internal list of libraries that use the new OverDrive website layout.


Version 2.3.0 - 18 Nov 2016

Assume the most common format (EPUB or MP3) when no formats are listed for pre-release e-books and audiobooks at new OverDrive sites.

Fix search errors at Scribd. ("KeyError (content)")


Version 2.2.1 - 15 Nov 2016

Fix search error at new OverDrive sites. ("Server response error (missing "h2 class=\bsearch-text\b")")


Version 2.2.0 - 14 Nov 2016

Allow search for Kindle Unlimited books at Amazon Australia.

Show "XX" for availability when a book is present in a library's collection, but no copies of the book are currently owned by that library.

Show decimal series index for discovered books when needed.

Improve detection of books that require a volume number to form a unique title.

Improve migration of links for libraries that recently converted to new OverDrive websites.


Version 2.1.2 - 29 Oct 2016

Fix search error at new OverDrive sites when signed into a branch library. (Fixes "Server response error (Unexpected anchor href:...)".)


Version 2.1.1 - 29 Oct 2016

Fix inability to sign in to new OverDrive sites that do not require a PIN.


Version 2.1.0 - 25 Oct 2016

Allow search of the Amazon Prime Reading catalog. (Configure a lending library with provider "Amazon" and library-id "reading".)

Fix inability to determine pre-release dates at Freading due to a website change.

Fix inability to sign in to OverDrive sites using the new 2016 website design. Also improve checking current availability of books at those sites.

Faster searching for recommendable books at Cloud Library.

Change error messages to indicate that "Check and repair book links" should be used following an OverDrive library-id configuration change.

Make link migration performed by "Check and repair book links" function properly for more libraries.

Improve detection during search of books removed from library collections. (Only effective if the "Update current availability of books with new links" configuration option is selected.)


Version 2.0.0 - 16 Sep 2016

Separate the logic of finding new links from the removal of outdated links. Searches will now only add new links, never remove them (unless a current availability check is also done.) The configuration option to Unlink books if no longer found by search has been removed. Unchanged links will no longer be reported in the search results summary.

The Check current availability of selected books menu choice is now available even when the ol_wait_weeks custom column does not exist. Besides determining the wait to borrow, this will also check for books no longer available at the lending library and remove the link if so. All links are checked, even those for disabled libraries.

A new configuration option, Update current availability of books with new links will cause a check for the current availability/wait to be done for books whenever a new link is found by a search.

The Search for books by selected authors menu choice will now update links for all books by the same author(s) in the calibre library, even if they are not selected when the search is started.

Allow multiple search jobs to be active for the same set of books. Results may be inconsistent if the same books are updated in different ways.

The Unlink selected books menu choice now presents a submenu to allow more flexibility in link removal.

The overdrive_link_library_groups plugin tweak now enables a single menu choice, Set library group, that will select a group of libraries for subsequent searching.

In the Discovered Books dialog: The new Update button will match a single selected book to an existing calibre book, providing a list of suggested books to choose from. (Useful if a discovered book should actually match an existing calibre book, but failed to do so due to a mismatch in the title.) The new Discard Dupes button will mark all duplicate discovered books to be discarded. The Update Existing button was renamed to Update Match.

The plugin tweak overdrive_link_amazon_ident can be set to override the identifier used for the Set 'amazon' identifier for Kindle e-books configuration option. See the help section on Configuring Tweaks for more information.

Faster, but more strict, book matching algorithm.

Rename the embedded metadata plugin to "Overdrive Link*" to keep it separate from the main plugin.

Fix errors and missed books when searching Amazon, Axis 360, Hoopla, OverDrive (2016 website layout), and Scribd resulting from various website changes.

A major change to the OverDrive website design is being rolled out to lending libraries in 2016. With this redesign the identifiers associated with books will change, making links to book web pages partially incompatible with the old website design. Links previously created by searching will still function after a library migrates to the new design, but future searches will create different links, resulting in both old and new links existing for the same book. The Check current availability of selected books menu choice can be used to remove old-style links and replace them with the equivalent new-style.

Signing in to OverDrive sites with the new 2016 layout is not yet supported.


Older Versions
The version history for plugin releases prior to version 2.0.0 can be found in in the Overdrive Link thread on MobileRead Forums.


Special Notes:
  • Requires calibre version 1.17.0 or later.

Installation Notes:
  • Install the plugin and restart calibre as described in the Introduction to plugins thread.
  • Select Customize plugin from the plugin menu (or via Preferences -> Plugins).

Attached Files
File Type: zip Overdrive Link.zip (380.7 KB, 10 views)

Last edited by jhowell; Today at 10:04 AM. Reason: Version 2.5.0
jhowell is offline   Reply With Quote