[GUI Plugin] Author Watch

[vasbinde]

Overview:

Author Watch is a Calibre plugin that tracks a list of favorite authors -- independent of what you already own -- and checks for new releases and books you don't yet have, cross-referenced against your Calibre library. It replaces the need to manually bookmark author pages or repeatedly search retailers to see what's new or what books you are missing from your favorite authors.

CRITICAL NOTE: This plugin was developed with Claude AI, based on my requirements, UI design, etc. I am a technology industry veteran, with decades of experience in systems, integrations, etc. However, my expertise in the actual syntax of programming ended in the era of COBOL, Visual Basic, Ada, Fortran, and even AppleSoft BASIC. As such, I used Claude to develop the back end Python code for this solution, based on my requirements. I use this solution daily in my use of Calibre and do work with Claude to correct issues when I find any. At this point, I consider this tool to be fairly feature complete for my purposes and would like to share it with the wonderful audience here. Please note that as I am not a Python expert, any defect fixes or enhancement requests would need to go through either Claude or another, human, Python expert. Please keep that in mind if considering the use of this plugin.


Recent Update (v2.18.3 - 2026-08-13):

• Added Goodreads as a source. A lot more books should show up for people now.
• Tried and dropped Fantastic Fiction - Current JavaScript blocks are a hard pass
• Added better diagnostics to determine why items are being dropped.

Key Features:

• Track any number of favorite authors, with per-author notes, genre, and a 5-star rating
• Three data sources: Goodreads, Apple Books (via iTunes) for most authors, and Amazon/Kindle Unlimited for exclusive authors
• Real publication dates (not just years) for accurate "new release" detection
• Same-name author disambiguation -- correctly tells apart two different authors who happen to share a name
• Language filtering, checked against each book's actual detected language
• A permanent verification cache, so repeat checks are fast and don't re-fetch data unnecessarily
• "Ignore" support for omnibus/collection editions, so component books don't keep showing as missing
• Multi-select actions -- check several authors, or detect several Author URLs, in one action
• A combined "All Missing Books" view across every tracked author
• Export/Import to back up your configuration or move it to a new computer

LICENSE:

This plugin is released as fully open source. You are welcome to use it for any purpose, as long as credit is given to this original solution.

DOCUMENTATION:

A complete set of documentation with screenshots is attached as a .DOCX and .PDF file.

Version History:

Spoiler:

# Author Watch — Version History

*Current version: 2.18.3* 2026-08-13

## 2.18.3 — Fixed a real, serious bug: blocked fetches were being permanently cached as confirmed facts

- Reported after 2.18.2: still no dump, still the old generic message, for the same specific titles. Root cause found: `set_verified_book(...)` was being called *unconditionally* after the per-book fetch attempt — even when `GoodreadsBlockedError` was caught. That meant every blocked attempt got cached as a confirmed "no ASIN found" fact, indistinguishable from a genuine no-Kindle-edition book. Every future run then hit that cache entry and reused it forever — no new request, no new dump, no chance to ever notice the block had lifted.
- Fixed: a blocked fetch is now never cached at all, only a fetch that actually completed (blocked or not) gets its result stored. This stops the poisoning going forward, but **won't retroactively fix titles already poisoned by this bug** (any Goodreads title cached as "no ASIN" going back through 2.16.0–2.18.2 could be affected) — those need one more "Clear Verification Cache" to actually get retried under the corrected logic.

## 2.18.2 — Fixed: hard connection failures were being silently swallowed with no diagnostic detail

- Explains "no dump was saved despite an Inconclusive result": the reason text in a real run read the generic "fetch failed or no ASIN found" rather than the specific WAF-challenge message — meaning the per-book fetch never even reached the WAF-detection code at all. Root cause: if the connection itself failed outright (a real HTTP error, timeout, or reset — distinct from the "soft" 202-with-a-challenge-page response seen before), the code caught the exception and silently returned `None` with zero detail, before ever getting a response body to check or dump.
- Fixed: a hard connection failure is now treated the same way as a confirmed WAF block — surfaced with the real underlying error message in diagnostics, and tripping the same circuit breaker (stop attempting further per-book fetches for the rest of that run). There's still genuinely nothing to dump in this case (no response body was ever received), but the failure itself is no longer invisible.

## 2.18.1 — Fixed: ownership matching now tolerates whitespace differences in author names

- Confirmed root cause of the "nothing shows as Already Owned" report: the tracked author nam had initials with no spaces and the name actually stored on the books in Calibre had initials with spaces differed only by whitespace around the initials — and ownership matching was an exact (if case-insensitive) comparison, so every single check silently failed.
- Fixed with a narrowly-scoped author-name comparison that strips all whitespace before comparing. Deliberately kept narrow rather than a general fuzzy match — misattributing library ownership to a *different* real author with a similar name would be a worse failure than the one being fixed, so this targets specifically the kind of formatting difference confirmed here, not general "close enough" similarity.

## 2.18.0 — Goodreads hit a new AWS WAF block on book pages; fixed how unverified titles are handled

- The user's real dump confirmed Goodreads' individual book pages are now hitting an AWS WAF bot-challenge gate (HTTP 202, `gokuProps`/`AwsWafIntegration` JS challenge, no real content) — the same category of structural block that ruled out Fantastic Fiction, just a different vendor, and confirmed via explicit signature detection this time rather than inference. The author list page itself is still working.
- **Found and fixed the real architectural bug this exposed**: a title was being marked "trusted, no Amazon check needed" *before* the per-book verification even ran — so when that verification failed (as it now does, universally, thanks to the WAF block), the title was trusted with zero real data *and* simultaneously excluded from Amazon's gap-fill, which could have actually verified it. This is why wrong-language titles kept appearing (no language data means the filter has nothing to check) and date/price stayed empty even after the 2.17.0 fix. Now a title is only trusted and excluded from Amazon's gap-fill once real verified data actually comes back — otherwise it falls through to Amazon/Get Books, where it gets properly verified, including real language checking.
- Added a circuit breaker: once one per-book fetch confirms the WAF block, every remaining book in that run skips straight to falling through instead of wasting a request that's already known to fail — directly addressing the "be kinder to Goodreads" concern, without giving up the richer per-book data entirely for whenever the block eventually lifts.
- **Not yet resolved, needs your input:** the "no book is showing as Already Owned" issue looks separate from the above — the dialog title bar reads AUTHORNAME (no spaces) where every earlier run showed AUTHORNAME (with spaces). If this doesn't match how the author is actually stored in your Calibre library, ownership lookups would silently return nothing regardless of what you actually own. Worth checking directly before assuming a code fix is needed here.

## 2.17.1 — Fixed a real pagination bug: Goodreads was stopping after 1-2 pages

- Reported after the first real run of 2.17.0: only about a third of the test author's books came through (one missing entirely), and no HTML dump was saved despite the incomplete result. Root cause found: the reverse-pagination early-stop was being fed `previously_known` — the *global*, all-sources known-titles set — rather than one scoped to Goodreads specifically. An author already well-cataloged by Amazon/Get Books across many earlier runs meant most titles looked "already known" even on Goodreads' own very first real run, so the early-stop triggered almost immediately and broke out after the newest page or two, never reaching older pages that held titles Goodreads itself had never actually told us about (their exact date, ASIN, and language specifically).
- Fixed: the early-stop now uses a Goodreads-specific known-titles set, built from the verification cache's own `"goodreads"`-tagged entries — so it only ever kicks in on Goodreads' *own* repeat runs, not because some other source happened to find the same titles independently.
- The missing HTML dump was a secondary symptom, not a separate bug: the "save only when results are empty" condition was working correctly — the results weren't empty, just badly incomplete. No change needed there once the real cause is fixed.

## 2.17.0 — Goodreads: real fix for missing Language/Price/Date, sourced from Goodreads' own embedded data

- Root cause of all three originally reported gaps: Goodreads has redesigned its book pages to a React/Next.js layout (`FeaturedDetails`) — the old `EditionDetails`/DescList markup this module originally targeted **no longer exists on live pages at all**, confirmed via a real page dump the user provided (parser found zero fields). Rebuilt on a fundamentally more stable foundation: every Goodreads book page embeds a `__NEXT_DATA__` script tag with the book's full structured data as real JSON, including ASIN, format, exact publication date, language, and (found separately, in a `KindleLink` object) price — the actual data layer both the old and new visual layouts were always rendered from. Parses this via real `json.loads()` and a recursive search, not regex field-matching, so it isn't tied to any particular DOM structure and should survive future redesigns better than a layout-specific scraper could. The old DOM-based parser is kept only as a fallback.
- **Fixed the missing language filter** — a real, separate bug: Goodreads' author page lists foreign-language translations (French, German, Spanish, Italian confirmed in the report) as fully independent entries with their own ASINs, and nothing was filtering them out. The language allow-list check now runs for Goodreads too (previously computed too late in the function to apply there at all), matching the same exclusion behavior already used for the Amazon and Apple Books paths.
- Price now flows all the way through: extracted from the real `KindleLink.ebookPrice` field, cached, and shown in Missing Books instead of `?`.
- Verified end-to-end against real data from the user's dump before shipping — ASIN, exact date, language, and price all correctly extracted together in one pass.

## 2.16.2 — Fixed a real Goodreads parsing bug: mixed quote styles

- The HTML-dump fix from 2.16.1 paid off immediately: the user's real dump file proved this was never a bot block at all — the page was fully populated with real book listings. Root cause found by direct comparison: Goodreads' live markup uses **single quotes** on some tags (`<span itemprop='name' ...>`) while using double quotes on others in the exact same page, and the book-title regex hard-coded double quotes only, matching a static sample that happened to be consistently double-quoted but not what the live server actually sends.
- Fixed the title regex to accept either quote style, verified against both the original sample and the new real live sample (correctly extracting titles with genuine apostrophes, e.g. "I Don't Want to Be"). Proactively hardened two other regexes (`_CURRENT_PAGE_RE`, `_EDITION_FIELD_RE`) the same way, since the mixed-quote-style pattern is now proven, not hypothetical, and both use the same kind of hard-coded double-quote assumption that just failed.
- Confirmed no regression: both the original double-quoted samples and the new live single-quoted sample now parse correctly.

## 2.16.1 — Fixed: Goodreads list-page fetch had no HTML-dump support at all

- Reported after the first real run on an author: Goodreads found "0 books" on both URL forms, but the HTML dump folder was completely empty — nothing to actually diagnose with. Root cause confirmed: `fetch_author_catalog()` (the author book-list page fetch) never had `save_html_path` support at all — only the per-book detail fetch did. A real gap in the 2.16.0 build, not a mystery.
- Fixed: `fetch_author_catalog()` now saves the first page's raw HTML whenever the overall result is empty, and returns an explicit diagnostic note (HTTP status, byte count, books parsed) the same way the Fantastic Fiction and Amazon-search paths already do — Author-Match Diagnostics will now show real data instead of just "found 0 book(s)."
- Next run should finally show either real content (if this was a fetch/parsing issue) or a genuinely empty response (the same soft-block signature seen with Fantastic Fiction and the Amazon search page) — either way, something concrete to work from instead of continued guessing.

## 2.16.0 — Goodreads as a new primary source, fully wired

- New `goodreads_scraper.py`: parses an author's paginated Goodreads book-list page (title, Goodreads book ID, publication year) and each book's own "EditionDetails" panel (exact publication date, real Amazon ASIN, format, language) — every parsing rule verified against real HTML the user provided directly before being wired in. Caught and fixed a real bug during testing: the *current* page number never appears in its own `?page=N` links (it's plain text, not a link), so last-page detection initially undercounted by exactly one.
- Unlike Fantastic Fiction, **both pages were independently confirmed live** (a real non-JS browser, w3m, loaded full content with no CAPTCHA) *before* this module was built — so `source_goodreads_enabled` defaults **on**.
- New per-author **"Goodreads Author URL"** field and grid column (Tracked Authors, right after Amazon Author URL) — Goodreads requires a numeric author ID that can't be derived from a name the way Fantastic Fiction's URL could, so this mirrors the existing manually-pasted Amazon Author URL field exactly.
- `_fetch_exclusive_author_books` rewired to the requested priority: **Goodreads first** (primary, trusted as-is when available) → **Amazon runs its full search regardless** (to catch anything Goodreads missed), but only spends a verification request on titles Goodreads didn't already cover → **Get Books** as the final fallback. This required no new gap-filtering logic — Goodreads simply feeds into the same `covered_titles` set Amazon's existing gap-fill already reads from.
- Diagnostics logs Goodreads-sourced entries with the same two-stage pattern already used for Fantastic Fiction (a fetch-status row, "trusted via Goodreads," plus the ownership-stage row every book gets).
- Verified programmatically that all 19 diagnostics call sites (13 before, 6 new) produce a consistent tuple shape.

## 2.15.0 — Fantastic Fiction now defaults OFF: confirmed non-functional

- Independently confirmed via a real non-JS browser (w3m): Fantastic Fiction requires solving a JavaScript-driven CAPTCHA before serving any real content. This is a structural requirement of the site, not a temporary rate limit — a plain HTTP fetch (this plugin's entire approach) cannot execute JavaScript or solve a CAPTCHA, so this source cannot work as currently built, regardless of headers, retries, or waiting.
- Also confirmed the earlier `curl_cffi` TLS-fingerprint-impersonation approach (dropped from the Amazon path earlier in this project) wouldn't help here either — it impersonates a browser's *connection signature*, not JavaScript execution, so it's equally unable to pass a genuine JS challenge for the same underlying reason.
- `source_fantastic_fiction_enabled` now defaults to **off**. Amazon and Get Books remain on by default, unaffected — the tiered fallback continues to work exactly as before for the two sources that actually function. Fantastic Fiction's code, tests, and toggle all remain in place (re-enable by hand if the site's policy ever changes), but the module docstring, pref comments, and the checkbox label itself now state plainly that it's expected to be non-functional, rather than implying it's a live, working option.

## 2.14.1 — Fantastic Fiction: more realistic headers, sharper diagnosis of the HTTP 202 signature

- The real diagnostic data from 2.14.0 paid off immediately: confirmed the Fantastic Fiction fetch is receiving **HTTP 202 (Accepted) with an empty body** — a recognized bot-mitigation gateway signature, not a normal response for a static HTML page. Diagnostics now calls this out explicitly by name when it happens, rather than the more generic "possible soft anti-bot block" message.
- Added more realistic browser headers (`Accept`, `Connection`, `Upgrade-Insecure-Requests`) to the Fantastic Fiction fetch, on the chance the gateway is triggered by missing headers rather than a genuine JavaScript challenge. Deliberately did **not** add `Accept-Encoding: gzip/br` — plain `urllib` doesn't auto-decompress a response the way a browser does, so advertising support for an encoding this code can't actually decode would risk turning an inconclusive result into an actively wrong one.
- **Honest assessment, not a promised fix:** a 202-with-empty-body pattern is often a genuine JavaScript-challenge gateway, which no header change can pass — this may need the same "let it settle" time Amazon needed earlier, rather than being fixable in code at all. The header change is a reasonable, low-risk thing to try, not a confirmed solution.

## 2.14.0 — Explicit HTTP status/byte-count diagnostics, true cached-source tracking, and Amazon search-page dumps

- **Made the "empty response" mechanism explicit rather than inferred.** Following a real-world report of a 0-byte Fantastic Fiction dump file, both `fantastic_fiction.py` and `amazon_search_scraper.py`'s fetch functions now capture and report the real HTTP status and byte count. `fetch_author_bibliography()` now returns a diagnostic note (e.g. `"HTTP 200, 0 bytes, 0 book(s) parsed -- EMPTY response body (possible soft anti-bot block)"`) that flows directly into Author-Match Diagnostics, distinguishing a genuinely empty "successful" response — a known soft-block technique — from a normal response that simply didn't parse into anything.
- **Fixed the Source-column ambiguity**: the verification cache's generic `"amazon"` tag never distinguished which Amazon path actually produced a cached entry. Split into `"amazon_direct_search"` and `"amazon_get_books"` (old entries still read correctly for backward compatibility). Diagnostics now shows each row's **true cached origin** — a `[cached]` entry correctly reflects whichever path *actually* produced it historically, not just whichever path happens to be running on the current check.
- **Added HTML-dump support to the Amazon search-scraper's own listing page**, which had none before (only per-book product pages did) — saved whenever the first page returns zero listings. This doubles as a fix for an older, related blind spot: a CAPTCHA-blocked search page returns `200 OK` with nothing on it, previously indistinguishable from genuinely reaching the end of a small catalog.
- Verified programmatically (not just by eye) that all 13 diagnostics call sites still produce a consistent tuple shape after these changes.

## 2.13.0 — HTML dump now covers Fantastic Fiction too

- The "Save full HTML for inconclusive checks" debugging feature (Backup and Diagnostics) previously only covered the Amazon path — Fantastic Fiction fetches had no equivalent, meaning a result like "fetched successfully but found zero books" had no raw HTML to actually inspect afterward. Fixed: both `fetch_author_bibliography()` (the author page) and `fetch_kindle_details_from_url()` (each book's own detail page) now save their raw fetched HTML whenever the result is inconclusive (zero books parsed, or no Kindle format found), the same "only save what's actually ambiguous" rule already used for Amazon.
- With this enabled, the next time Fantastic Fiction's fetch reports "found no books," the actual HTML it received will be sitting in the HTML dump folder — the missing piece needed to actually diagnose that unresolved mystery from 2.12.1, rather than continuing to guess at it.

## 2.12.1 — Fixed: Get Books fallback was never actually triggering

- Found and fixed the real bug behind "it never falls back to Get Books": the fallback check was `if amazon_results is None`, but an empty (but successfully-returned) result from the Amazon search-scraper is `[]`, not `None` — only an actual exception ever set it back to `None`. A scraper run that completed without error but genuinely found zero candidates silently never triggered the Get Books fallback at all. Fixed to trigger on any empty/falsy result, not just an outright failure.
- Added the same diagnostic visibility to the Amazon-search attempt that Fantastic Fiction got in 2.11.1 — a new `(Amazon Search)` row in Author-Match Diagnostics now shows how many raw candidates were found and how many survived after excluding titles already covered by another source, or the actual error message if the search itself failed.
- Verified programmatically that all 13 diagnostics call sites (11 before, 2 new) produce a consistent tuple shape before shipping.
- **Not yet resolved:** the Fantastic Fiction fetch reported as "found no books" despite returning a real HTTP response is still unexplained — this needs live testing against the actual site to diagnose further, which isn't possible from this environment. One real possibility worth considering: given this happened the same night as heavy, repeated automated testing against both Amazon and now Fantastic Fiction, it's plausible Fantastic Fiction is reacting defensively the same way Amazon did earlier, serving different (bot-detection) content to an automated request than what a real browser sees — in which case, like Amazon, it may simply need time to settle rather than a code fix.

## 2.12.0 — Author-Match Diagnostics: new Source column

- Every diagnostics row now shows exactly which of the four data sources it came from — **Fantastic Fiction**, **Amazon**, **Calibre Get Books**, or **Apple Books** — instead of leaving that to guesswork from the reason text (e.g. distinguishing a `[cached]` Amazon entry from a Get Books one previously required reading carefully; now it's just a column).
- For the Amazon-exclusive path specifically, the Source column correctly reflects whichever of Amazon's direct search or Get Books actually produced that candidate on that run (tracked live, not inferred after the fact) — directly answering the exact question that motivated this: "where did this row's data actually come from?"
- Verified programmatically (not just by eye) that all 11 places diagnostics get logged now consistently produce the same 5-element shape before shipping, to avoid a silent runtime mismatch.

## 2.11.1 — Fixed silent failure visibility for the Fantastic Fiction fetch

- Reported after the first real run on an author under 2.11.0: nothing from Fantastic Fiction showed up in Author-Match Diagnostics at all — every entry was Amazon-sourced. Root cause: the Fantastic Fiction fetch was deliberately built to fail silently (never crash the whole check), but that meant a real failure there — network issue, the site blocking automated requests, markup mismatch, or genuinely nothing found — was completely invisible and indistinguishable from "this author just isn't on Fantastic Fiction."
- Fixed: every Fantastic Fiction fetch attempt now logs to Author-Match Diagnostics under `(Fantastic Fiction)` — a real success shows how many books were found, a fetch that succeeds but finds zero books is flagged Inconclusive (ambiguous and worth investigating, not treated as a silent non-event), and a genuine fetch failure now shows the actual error message.

## 2.11.0 — Full three-source tiered pipeline: Fantastic Fiction → Amazon → Get Books

- `_fetch_exclusive_author_books` fully rewired to the tiered flow: **Fantastic Fiction tried first** (one bibliography-page fetch, then a per-book detail-page fetch only for titles not already cached) — every title it covers is trusted as-is with zero further Amazon requests. **Amazon's direct search fills in the gap** — only for titles Fantastic Fiction genuinely didn't cover (verified via a simulated real-world scenario: a series fully covered by Fantastic Fiction correctly dropped from the Amazon candidate list, while an Omnibus edition it didn't have correctly survived to be discovered and verified). **Get Books remains the final fallback**, gap-filtered the same way, used only when Amazon's own search is disabled or unavailable.
- Extended the verification cache (`verified_books`) with `detected_price` and `detected_store_url`, needed to cache Fantastic Fiction's richer per-book facts (backward-compatible — older cache entries simply lack these fields).
- Author-Match Diagnostics now shows Fantastic Fiction-sourced books too, tagged with a distinct "trusted via Fantastic Fiction (no Amazon check needed)" reason, alongside the existing ownership-stage (Missing/Already Owned) entry every book gets regardless of source.
- All three sources remain independently toggleable (Backup and Diagnostics, added in 2.9.0) — turning any one off doesn't disturb the others, and the fixed tiered order stays the same regardless of which are enabled.

## 2.10.0 — Fantastic Fiction: author-page parser complete and validated

- `parse_author_page()` / `fetch_author_bibliography()` / `fetch_author_kindle_catalog()` added to `fantastic_fiction.py` — the piece that was deliberately left unbuilt in 2.9.0 pending real HTML. Tested against the user's real, complete author page: all 43 books across every series (including two differently-co-authored series, each routing through a different series-page prefix but still correctly resolving to this author's own book pages) and both collection entries, with zero false positives from unrelated page sections. A book's own real Fantastic Fiction detail-page link is taken directly from the page rather than re-derived by guessing a slug — more reliable than the original design.
- **Still not done:** the full three-source tiered orchestration isn't wired into the actual check pipeline yet — this release completes and validates the last missing building block (module-level), not the integration. Next step is wiring `_fetch_exclusive_author_books` to actually call Fantastic Fiction first, cache its facts (source-tagged, mirroring the existing Amazon-cache pattern), and only fall through to Amazon/Get Books for whatever titles it didn't cover.

## 2.9.0 — Fantastic Fiction integration (foundation) + independent source toggles

- New `fantastic_fiction.py` module: fetches and parses a book's Fantastic Fiction detail page for its real per-format (Kindle/Audio/etc.) publication date, price, and Amazon ASIN — sourced entirely from Fantastic Fiction's own servers, meaning a title fully covered there needs zero Amazon requests at all. Every URL-construction and parsing rule was verified against real markup the user provided directly (the author-page URL pattern, two book-detail-page URL patterns, and a real book-shop div with two competing formats) before being written — the same discipline that caught real problems with the Amazon scraper earlier in this project.
- **The Backup and Diagnostics tab now has three independent checkboxes** — Fantastic Fiction, Amazon (direct search), and Get Books — replacing the single "use Amazon search scraper" toggle. All three default **on**. Deliberately not reorderable (fixed tiered order: Fantastic Fiction first, Amazon fills gaps, Get Books is the last resort) to keep the fallback logic manageable, but any one can be turned off independently — e.g. to avoid Amazon entirely during a period of anti-bot pressure, while keeping the other two active.
- **Not yet done:** the Fantastic Fiction *author* page (the series/title list that would drive the "primary source" role) isn't parsed yet — only the *individual book* page is built and verified so far, since the author-page HTML hasn't been validated against real markup yet (only a plain-text rendering, which isn't reliable enough to build a parser against). The three-way tiered fallback itself also isn't fully wired into the check pipeline yet — this release lays the foundation (module, verified parsing, config UI) for the next increment.

## 2.8.2 — Fixed a real regression: audiobook/Kindle title collision in the verification cache

- Found and fixed the actual root cause of a regression reported right after 2.8.1: Amazon's search results can include a book's **Audiobook edition as its own separate listing** (different ASIN, but usually the same displayed title) alongside its Kindle listing. The scraper had no way to tell these apart, so both could become independent candidates sharing one title. Since the verification cache is keyed by normalized title (not by ASIN), when both raced through the same threaded check, whichever one's write landed *last* silently overwrote the other — a real, non-deterministic race condition, which is exactly why re-running the check kept producing different (and shrinking) results, and why clearing the cache alone didn't fix it (a fresh run just re-raced the same collision).
- Fixed at the source: the search-scraper now requires a listing's own format label to read exactly "Kindle" before accepting it as a candidate at all — filtering out Audiobook (and other non-Kindle-format) duplicates before they can ever collide in the cache. Verified against a simulated same-title Kindle+Audiobook collision to confirm only the Kindle listing survives.
- The 2.8.1 audiobook-detection fix (labeling a resolved-to-audiobook page as MISMATCH rather than Inconclusive) remains in place as a secondary safety net, but this closes the actual source of the problem.
- **Post-release validation:** tested against real, complete Amazon listing HTML the user provided (including a co-authored book and, critically, a listing offering both Kindle and Audiobook formats together). This corrected the original theory of the bug's mechanism: Audiobook editions don't appear as a separate competing search-result listing with their own ASIN as originally assumed — they appear as a second format option *within* the same listing, right after the Kindle one. The fix still works, and for the right reason: taking the *first* format label in a listing reliably picks Kindle over Audiobook, since Amazon always lists it first. Code comments updated to reflect the confirmed mechanism (no functional change, version unchanged).
- Added the plugin version number to every dialog's title bar (e.g. "Author Watch — Results — v2.8.2"), so it's always clear exactly which version is running during troubleshooting — requested directly after this regression made the gap obvious.
- **If you hit this regression**, a fresh check after upgrading (clearing the verification cache first, to discard any already-poisoned entries) is recommended to get clean results going forward.

## 2.8.1 — Audiobook editions now show as a real MISMATCH, not Inconclusive

- The new search-scraper path can occasionally resolve a candidate to an Audiobook (Audible) edition's product page rather than the Kindle/ebook edition. These genuinely lack the author-identity attribute the plugin checks for (a real difference in that page's markup, not a fetch/parse failure), but were showing up as generic "Inconclusive" in Author-Match Diagnostics, indistinguishable from an actual bot-check block.
- Fixed: detected via the page's own title (Amazon marks these plainly, e.g. "... (Audible Audio Edition): ..."), now reported as a genuine **MISMATCH** with a specific reason, leaving "Inconclusive" reserved for real parsing failures and bot-check blocks. Verified against the exact page titles from real Diagnostics output before shipping.

## 2.8.0 — Experimental Amazon search-based catalog discovery (opt-in)

- Second increment of the search-scraper overhaul: `amazon_search_scraper.py` (built and unit-verified against real HTML in the previous session) is now actually wired into the exclusive-author check, as an alternate to Calibre's Get Books search.
- **Off by default** — Get Books remains the default and recommended path. Enable via the new "EXPERIMENTAL: use a real Amazon search page..." checkbox in Backup and Diagnostics.
- When enabled AND the author has an Amazon Author URL set (the scraper searches by author ASIN directly, unlike Get Books' name-based keyword search, so it needs one), the check uses real, paginated Amazon search results sorted newest-first, stopping once a full page returns nothing new relative to the cache — falling back to Get Books automatically on any failure (network error, no ASIN set, markup mismatch) rather than failing the whole check.
- Fixed a real bug caught during this session's own testing before shipping: the per-listing parsing window was far too narrow (a fixed guess that worked for title but missed price entirely on real markup, which is far more verbose than expected). Replaced with a window bounded by the start of the *next* listing on the page, verified against a simulated two-listing page to confirm no title/price bleeding between adjacent results.

## 2.7.3 — Fixed the real cause: live sorting corrupting rows during toggle/populate

- Fixed the actual root cause behind both the "toggles do nothing" and "rows go blank" symptoms reported against 2.7.2: `_apply_visibility()` (and the dialogs' own initial population) modified table items — including `setVerticalHeaderItem()` calls in a loop — while the table had live sorting enabled. Modifying items while sorting is active risks Qt re-sorting *mid-loop*, which shifts which physical row each subsequent iteration touches and corrupts cells along the way (confirmed via user-provided screenshots: every row but one going blank except Title, on a table that had an active sort restored from a previous session).
- Fixed by disabling sorting before any loop that modifies table items (both the constructor's population loop and `_apply_visibility()`'s hide/show loop, in both Missing Books windows), and re-enabling it afterward — the same safe pattern already used correctly elsewhere in this plugin (`ConfigWidget.refresh_table()`).
- This was a genuinely hard bug to pin down from static code review alone (the earlier 2.7.2 fix was real and necessary, just not sufficient) — real screenshots showing the corrupted state were what made the actual cause identifiable.

## 2.7.2 — Fixed the already-owned/ignored toggles doing nothing

- Fixed a real bug: toggling "Show already-owned matches" or "Show ignored books" had no effect at all — every row stayed visible regardless. Root cause: the two flags were stored under separate `Qt.ItemDataRole.UserRole`/`UserRole + 1` roles, and arithmetic on that enum doesn't round-trip reliably through `setData`/`data` in this Qt binding, so the "ignored" flag was silently never read back correctly (and both flags effectively always evaluated as `False`, which happened to leave every row visible no matter what).
- Fixed by storing both flags together as a single dict under one plain role instead of splitting them across two roles — verified the round-trip across all four have/ignored combinations before shipping.

## 2.7.1 — Raised the Get Books search result cap

- Raised the default result cap for the Amazon/KU-exclusive Get Books search from 40 to 1000, after confirming via Author-Match Diagnostics that some volumes of a prolific author's catalog never appeared as candidates at all — not excluded by any of this plugin's own filtering, just never returned by Amazon's search in the first place.
- Honest caveat: this is still a relevance-ranked keyword search, not a true "every book by this author" API — a higher cap narrows the gap but doesn't guarantee closing it entirely, since Amazon's own ranking still decides what comes back.

## 2.7.0 — Ignored books get the same visibility as owned books

- Both Missing Books windows (per-author and All Missing Books) now have a **"Show ignored books"** toggle, working exactly like the already-owned toggle — off by default, so ignored books stay out of the way until you ask to see them.
- The Status column now shows three states: ✓ Owned, ○ Missing, and **⊗ Ignored** (icon shown once a toggle reveals more than the default missing-only view). Ignored rows get a distinct grey-strikethrough style (owned rows stay grey-italic), and a book that's both owned and ignored gets both styles combined.
- **Right-click a book to toggle its ignore status** — "Ignore Missing Book" on a normal row, or "Remove from Ignore List" on an already-ignored one, both updating the cache and the row's display immediately without needing to reopen the window.
- Ignoring a book no longer deletes it from the plugin's cached data — it's tagged instead, so it can be found again later via the toggle (previously, ignoring a book removed all record of it until the next fresh check).
- The summary label now combines both filters intelligently, e.g. "4 books already owned and 2 books ignored, filtered from display," adjusting automatically based on which toggle(s) are off.
- One known limitation: un-ignoring a book via right-click restores its correct status and styling immediately, but doesn't re-fetch its original purchase/library links within that same window session — those come back on the next reopen or fresh check.

## 2.6.2 — Sequential row numbers and an owned-books summary

- Fixed confusing row-number gaps in both Missing Books windows: hiding already-owned matches previously left Qt's default row numbers as-is (e.g. 2, 3, 4... 7, 9), which looked like missing data even though nothing was actually missing. Row numbers are now always sequential (1, 2, 3...) for whatever's currently visible.
- Added a summary line ("N books already owned and filtered from display") shown whenever the already-owned toggle is off and there's at least one owned match being hidden — disappears automatically once the toggle is checked.

## 2.6.1 — Fixed a crash in the new already-owned toggle

- Fixed a real crash (`NameError: name 'QCheckBox' is not defined`) when opening Missing Books: the new already-owned toggle used `QCheckBox` without it being imported. Also scanned the whole file for any other missing Qt imports — found none.

## 2.6.0 — Full visibility into the two-stage matching pipeline

- Author-Match Diagnostics now shows **both** stages of the process, not just verification: every book that passes author/language matching now also gets a logged "Missing" or "Already Owned" result, so you can immediately tell whether a book is absent from Missing Books because of a parsing/match problem or simply because you already own it.
- Both Missing Books windows (per-author and All Missing Books) now have a **"Show already-owned matches"** toggle (off by default) that reveals books the plugin correctly matched to the author but that are already in your library — previously these were silently discarded before you ever saw them.
- Already-owned rows get a distinct look: lighter grey, italicized text, a dedicated **Status** column showing "✓ Owned" or "○ Missing" (icons shown once the toggle reveals both kinds together), and no purchase/Get-Books links (nothing to act on for a book you already have).
- The Tracked Authors grid's "Missing" count now correctly counts only actually-missing books from the now-richer cache (which stores both missing and owned matches together).
- Along the way: caught and fixed a stale column-index bug in `AllMissingBooksDialog`'s Action-column click handler (still pointing at the old index after the Status column shifted everything over), and made the owned/missing flag robust to row sorting and removal (stored on the cell itself, not a separate row-indexed structure that could desync).

## 2.5.x — Unified header-state persistence, and three real bugs found through it

- **2.5.4** – The Missing cell's clickable-link styling now only appears where it's actually clickable (the main window), not in Preferences; fixed a related ordering bug that would have broken this styling on the main window's first render too.
- **2.5.3** – Clicking the Missing-count cell in the Preferences screen (which has no access to Calibre's main window) now shows an explanatory message instead of silently doing nothing.
- **2.5.2** – Fixed a real data-corruption bug: a header state saved under an old column count (before the Library column existed) was being restored onto the new column count, corrupting the whole row display. Fixed by keying saved state to the exact column count, so a mismatched state can never be restored again.
- **2.5.1** – Fixed a crash (`no attribute '_table'`) caused by `restoreState()` firing save signals before the table reference was assigned.
- **2.5.0** – Replaced a hand-rolled sort/order-only persistence mechanism with `QHeaderView.saveState()`/`restoreState()`, applied consistently across all four grids (width, order, and sort together).

## 2.4.x — Full publication dates everywhere, All Missing Books polish

- **2.4.1** – Widened the Published column, which was still sized for a bare 4-digit year.
- **2.4.0** – Added the missing Library (Libby) column to "All Missing Books"; upgraded "Year" to full publication dates everywhere (with correct chronological sorting, not alphabetical on the display text).

## 2.3.0 — Multi-author aggregated checks

- Multi-select "Missing Books for Selected Author" now runs one real aggregated check across all selected authors (reusing the same Results summary as "Check Favorite Authors Now"), instead of opening one dialog per author and blocking on any author with nothing new.

## 2.2.x — Apple Books language verification

- **2.2.4 / 2.2.3** – Fixed the Missing Books progress dialog not appearing at all, then appearing as a blank unpainted window, for non-exclusive-author checks.
- **2.2.2** – Fixed diagnostics always showing "Match" for any detected language, even when it didn't actually match the selected filter.
- **2.2.1** – Extended Author-Match Diagnostics and the HTML-dump tool to cover the Apple Books path too (previously Amazon-only).
- **2.2.0** – Real per-book Apple Books page language check for non-exclusive authors, using the same generic verification cache built for Amazon.

## 2.1.x — Ignore feature, multi-select, and a real selection bug

- **2.1.1** – Fixed the Ignore button itself only processing the first selected book, not all selected books.
- **2.1.0** – "Ignore Missing Book" (right-click or button, per-book, for Omnibus/Collection editions); multi-select support for Missing Books and Detect Author URL; fixed a real bug where the selected author was lost after either action (row indices going stale after a table rebuild).

## 2.0.0 — Major: real publication dates

**2.0.0 (decision)** – Dropped the `curl_cffi` experiment entirely after further real-world testing showed 100% success with plain `urllib` combined with the verification cache. The plugin has used only `urllib` since, with no vendored dependencies — fully multi-platform again.

- Extracted real publication dates from Amazon product pages (previously only a known-titles-diff heuristic).
- Switched exclusive authors' "new" detection to real date-vs-cutoff logic, matching non-exclusive authors.
- Extended the verification cache to include publication date.

## 1.20.x — Resolving the Amazon-blocking question

- **1.20.2** – Fixed a real co-authored-book bug: only the first author's ASIN on a page was ever checked, so a tracked author listed second (or later) on a shared book could be wrongly flagged as a mismatch.
- **1.20.1** – Flipped the default to `urllib`-first based on that result.
- **1.20.0** – Real controlled A/B test: `curl_cffi` got blocked by Amazon where plain `urllib` succeeded on the same book immediately after. Added a "Force urllib" toggle for controlled testing.

## 1.11.3 – 1.19.x — HTML diagnostics, and the Amazon-blocking investigation

- **1.19.0 – 1.19.4** – Vendored `curl_cffi` directly into the plugin zip (no separate install needed); fixed a chain of real packaging bugs along the way — a zip-import path-detection bug, compiled extensions being unable to load from inside a zip at all, and a missing `.dist-info` folder needed by `curl_cffi`'s own version check.
- **1.18.0** – Experimental `curl_cffi` TLS-fingerprint-impersonation fetch method added.
- **1.17.0** – Confirmed a real Amazon anti-bot CAPTCHA gate (via saved HTML dump) as the root cause of intermittent fetch failures.
- **1.16.0 – 1.16.1** – Filter box above the Tracked Authors grid (author/genre/notes); added a clear (X) button.
- **1.15.1** – Fixed column-order persistence (was saving sort/order but not surviving reopen).
- **1.15.0** – Permanent per-book verification cache (author ID + language), skipping re-fetches on future checks.
- **1.14.0** – "Detect Author URL for Selected": auto-fills when unambiguous, prompts only when genuinely ambiguous.
- **1.13.1** – Fixed star-click hit-zone alignment, enabled column drag-reorder, switched to Calibre-style edit-on-second-click, added sort persistence.
- **1.13.0** – Click-to-rate stars; Export/Import moved fully out of the main window; Preferences split into "Tracked Authors" / "Backup and Diagnostics" tabs.
- **1.12.1** – Window geometry persistence across all dialogs.
- **1.12.0** – 5-star Rating column (typed digit); Export/Import All Data.
- **1.11.5** – Diagnostics-menu toggle and Save-HTML toggle moved out of the main window into Preferences only.
- **1.11.4** – Removed retry logic after confirming it didn't help (a deliberate CAPTCHA gate, not probabilistic).
- **1.11.3** – Optional raw-HTML-dump diagnostic tool for inconclusive fetches.

## 1.x — Foundation (Amazon/iTunes dual-source architecture)

**1.10.0 – 1.11.2**
- Real language filtering for exclusive authors via a product-page HTML attribute.
- Shared thread pool across a full check (not one pool per author).
- Retry logic for Amazon's "generic Amazon.com" placeholder page.

**1.8.0 – 1.9.3**
- Same-name author disambiguation via `data-authorasin` product-page fetch (the "Aiden Phoenix" case — two different real authors sharing a name).
- Author-Match Diagnostics viewer built after two earlier silent-failure attempts.
- Concurrent ASIN verification via a thread pool.

**1.0 – 1.7.x**
- Initial plugin: track favorite authors, check new releases against the Calibre library.
- iTunes Search API for non-exclusive authors; Calibre's own Get Books (Amazon Kindle store) for KU/exclusive authors, after direct Amazon scraping was tried and consistently blocked.
- Author-field matching added to both paths (fixed cross-author contamination in search results).
- Non-modal dialogs, autosave-on-edit, Notes/Primary Genre/Libby integration, New/is_new column, title-filter boxes, results-filter dropdown (New or Missing / Missing only / New only).

[Terisa de morgan]

First of all, thank you for the plugin, I think I'm going to use it a lot. In any case, I've got a comment:

• Some amazon books put the series in front of the name. In those cases, the plugin doesn't recognize the book in your library. Example: Kathi Daley's books, The Inn at Holiday Bay: Verification in the Video -> Verification in the Video

As I say, thank you again.

[sgmoore]

Was trying this out, but it is not bringing some details I would have expected.

For example, I added John D MacDonald and set the amazon author url to https://www.amazon.com/stores/John-D...hor/B000AQ52CK

When I check for missing books it only brings up one book.

If I use Apple (by unticking the amazon exclusive box), then it brings the same book up along with 13 others including this one https://books.apple.com/us/book/the-...er/id887496700

But that book is also on amazon and under the correct author name see https://www.amazon.com/stores/author...2-5fdd68033b69

Why doesn't it appear when using amazon?

Another minor issue is - if I use the 'Get Books' option on any of the books that do appear, it doesn't just find that book, but lots of false positives. (I realise that may not be your fault as the 'Get Books' filtering does not make sense to me, but perhaps you can find a way around this issue)

[Sirtel]

Originally Posted by sgmoore

Was trying this out, but it is not bringing some details I would have expected.

For example, I added John D MacDonald and set the amazon author url to https://www.amazon.com/stores/John-D...hor/B000AQ52CK

When I check for missing books it only brings up one book.

If I use Apple (by unticking the amazon exclusive box), then it brings the same book up along with 13 others including this one https://books.apple.com/us/book/the-...er/id887496700

But that book is also on amazon and under the correct author name see https://www.amazon.com/stores/author...2-5fdd68033b69

Why doesn't it appear when using amazon?

I had the same problem - the plugin didn't find any new books on Amazon at all, although they're definitely there when I search by the author name. As there are many Amazon-only authors I'd like to watch for, this makes the plugin pretty useless to me.

[vasbinde]

Originally Posted by Terisa de morgan

First of all, thank you for the plugin, I think I'm going to use it a lot. In any case, I've got a comment:

• Some amazon books put the series in front of the name. In those cases, the plugin doesn't recognize the book in your library. Example: Kathi Daley's books, The Inn at Holiday Bay: Verification in the Video -> Verification in the Video

As I say, thank you again.

If the book title varies beyond a certain amount, the detection will not be able to match the book.

Can you include a screenshot of the book diagnostics grid after you perform a check for missing books? That would help a lot to determine what part of the "fuzzy" matching was not fuzzy enough. :-)

Thank you for your help!

[vasbinde]

Originally Posted by sgmoore

Was trying this out, but it is not bringing some details I would have expected.

For example, I added John D MacDonald and set the amazon author url to https://www.amazon.com/stores/John-D...hor/B000AQ52CK

When I check for missing books it only brings up one book.

If I use Apple (by unticking the amazon exclusive box), then it brings the same book up along with 13 others including this one https://books.apple.com/us/book/the-...er/id887496700

But that book is also on amazon and under the correct author name see https://www.amazon.com/stores/author...2-5fdd68033b69

Why doesn't it appear when using amazon?

Another minor issue is - if I use the 'Get Books' option on any of the books that do appear, it doesn't just find that book, but lots of false positives. (I realise that may not be your fault as the 'Get Books' filtering does not make sense to me, but perhaps you can find a way around this issue)

I do not have anything by John D. MacDonald in my library. However, I added the author into the on mine, but had the plugin choose the author URL by clicking on "Detect Author URL for Selected". The author URL it chose was slightly different from the one you pasted in, "https://www.amazon.com/stores/author/B000AQ52CK/allbooks". I then clicked on "Missing Books for Selected Author". I am showing 10 books missing using the Amazon search: Screenshot attached. I also see the same results when I manually paste in the URL you have here for the author.

In looking at the diagnostics screen, I see the results shown in the other screenshot.

Can you take a screenshot of the diagnostics screen and add it so I can see what the plugin is matching or not for that author? Thank you for your help!

[vasbinde]

Originally Posted by Sirtel

I had the same problem - the plugin didn't find any new books on Amazon at all, although they're definitely there when I search by the author name. As there are many Amazon-only authors I'd like to watch for, this makes the plugin pretty useless to me.

Can you please take and attach a screenshot of the diagnostics screen for an author or two so that I can run the tests on my end to compare? Thank you for your help!

[vasbinde]

I have realized something as I went through all of this: authors with massive back catalogs do not have all of their books appear through "Get Books" - the built-in Calibre tool for scraping ebook sellers. As such, my Amazon exclusive checks are missing some times more than 50% of the author's data.

Given that, I have been investigating how to manually suck down all of an author's pages, parse them, and cache the results so that we do not miss anything. Once that is complete, I'll begin running tests on that myself.

This will be a new function that is off by default, but added in to the forthcoming version 3 of this plugin to allow for much more data to be pulled directly from Amazon.

Hopefully posting this soon!

[vasbinde]

Made a lot of progress tonight until Amazon's anti-bot checks kicked in. I was able to pull in the entire truckload of books from a specific author that I use as a test case. However, there was a bug in the code that I wanted to address that caused audiobooks to show up as "Inconclusive" in the diagnostics results, rather than a status of "Mismatch - Audio Book" instead.

After changing that and testing again, I am now encountering the annoying Amazon anti-bot checks.

I will also see if I can add Fantastic Fiction as another data source.

[Terisa de morgan]

I've found another issue (I'll add the diagnostics later for the previous one). I've added one author to my list: Karen Rose, and it's mainly finding books by Karen Rose Smith....

[vasbinde]

Unfortunately, the option I wanted to try, similar to the Import-List plugin, was to use Fantastic Fiction. That now has a silly anti-bot JavaScript check that pukes all over any attempt to pull down an author page (HTTP 202 Error). This prevents that site from being used entirely. Back to the drawing board.

[sgmoore]

Originally Posted by vasbinde

Can you take a screenshot of the diagnostics screen and add it so I can see what the plugin is matching or not for that author? Thank you for your help!

See attached screenshots after I changed the Url to https://www.amazon.com/stores/author...Q52CK/allbooks

As well as not being able to detect the author name, there should be at lot more books than shown here. 200 in all or 86 English Kindle books.

That Url is one of those web pages that loads more as you scroll down in the browser and so perhaps it is only returning you some (15) of the books.

BTW, I'm not seeing anything in the author_watch_html_dumps despite enabling that option.

[vasbinde]

Originally Posted by sgmoore

See attached screenshots after I changed the Url to https://www.amazon.com/stores/author...Q52CK/allbooks

As well as not being able to detect the author name, there should be at lot more books than shown here. 200 in all or 86 English Kindle books.

That Url is one of those web pages that loads more as you scroll down in the browser and so perhaps it is only returning you some (15) of the books.

BTW, I'm not seeing anything in the author_watch_html_dumps despite enabling that option.

I just tried John D. MacDonald. I used him with the new 2.18.3 version that I posted, along with the author's Goodreads page. That was able to get around most of the Amazon blocks and suck down 87 books.

When you get a chance, please upgrade to the new 2.18.3 version and give that a try with Mr. MacDonald.

[vasbinde]

Would any of you be interested in dramatically enhancing this plugin with a separate "Scan Helper" that would be a full, JavaScript capable headless browser in a docker container? It would run constantly, scanning periodically (e.g. 1x per week) to see if new books are found from multiple sources and would then add it to a REST API accessible SQL DB. The plugin would then easily, as an option, communicate with the new Scan Helper to suck down data, as opposed to direct pulls. Avoiding a lot of the JavaScript requirements for anti-scanning bots, etc., whilst caching everything so that your pulls of data are very friendly for the sites we're using as data sources.

This would require some configuration, but could be much more robust for getting complete and accurate data.

[sgmoore]

Originally Posted by vasbinde

When you get a chance, please upgrade to the new 2.18.3 version and give that a try with Mr. MacDonald.

2.18.3 is better, but still doesn't show any books I am missing.

Without a goodreads url, Amazon detects the same 15 books as before, but this time it knows the author is correct, but matches them against a book in my library.

If I add a goodreads url (I had to add that manually, as it was not detected by the 'detect' button) then it found a lot more (maybe over 500), but didn't trust any of the results and so ignored them all.

The diagnostics screen is too big to show, so I have taken a couple of small screengrabs