[GUI Plugin] QuarantineAndScrub

[DaltonST]

[GUI Plugin] QuarantineAndScrub: A Calibre Add-on

This plugin is actually a Calibre add-on, not just a plugin. You need this add-on if you frequently add ebooks (especially fiction) with malformed and misplaced author, title, series, series index, and tags, and want them automatically and effortlessly corrected ("scrubbed"). It saves an enormous amount of time and energy correcting trashed out metadata.

It is somewhat of a "Swiss Army Knife" metadata tool, although like all knives, it requires proper handling and care. Furthermore, it is a knife that is designed and intended to only be wielded by a self-selected set of Calibre Users who are somewhat technical in nature, comfortable with ambiguity, essentially autodidactic, and already fluent in Calibre's functionality and library architecture.

Q&S is not a "tool bag", but rather an integrated set of tools that are designed to be used in a mostly predefined sequence. The menu ToolTips plus the User Guide explain what the sequence is.

Application Summary:

Spoiler:

1. Calibre's “Plugin Updater” summary description of Q&S:
   • [For Advanced Calibre Users Only: Must Download Special Database From mobileread.com]
   • Clean Malformed, Misplaced or Missing Author, Title, Series, Index, ISBN;
   • Tag Scrubber, Deriver, Propagator, Defaulter & Minimizer;
   • Series Deriver & Consolidator;
   • Series & Index Validator/Corrector;
   • Genre Deriver;
   • Library of Congress Code & Dewey Decimal Code Deriver;
   • Book Awards;
   • Author Pseudonyms;
   • Automatic Author Renaming;
   • Derive User-Defined Custom Columns from Title;
   • Derive User-Defined Custom Columns from Tags;
   • Special Manga Metadata Processing;
   • Special Graphic Novel Metadata Processing;
2. Total pages in its two-volume User Guide: 67
3. Number of Q&S-specific, non-Calibre SQLite Tables: 64
   • Number of User Maintained Q&S Tables: 18
   • Number of Q&S Maintained Q&S Tables: 46
4. Number of Q&S-specific, non-Calibre SQLite Views: 55
5. Number of Menu Choices: 63
6. Number of Foreground GUI Update Transactions: 34
7. Number of Background Update Jobs: 19
8. Number of Current Users: approximately 991 as of 29 December 2015
9. Percentage of Successful Users Who Read and Followed at Least the 1st Volume of the User Guide: 100%
10. Percentage of Total Calibre Users to Whom This Add-On is Targeted: : <0.001%

General:

Spoiler:

This add-on holds your ebooks with dirty (i.e., malformed and misplaced) metadata, and then scrubs the author, title, series, series index and tags. Afterwards, you move the scrubbed ebooks to your "Pristine Library". Refer to the User Guide, which includes a workflow diagram.

Scrubs normal ebooks, as well as Manga and Graphic Novels.

Refer to the User Guide (a.k.a. User Instructions) plus many other important information files and other resource files attached below.

Installation of the Q&S Special Calibre Library:

Spoiler:

Download the latest Q&S Official Library file. As of 13 February 2020 the latest file is: "QuarantineAndScrub_OfficialLibrary_v497_rename_to _zip.jar"

Place it in the parent directory of your many Calibre libraries, rename the extension from .jar to .zip, and extract it as a new Calibre Library, QuarantineAndScrub.

Example: X:\eBooks\QuarantineAndScrub\

Please note: the above .zip file is a compressed Q&S Calibre Library, not a book with a format of .zip! Do not emulate one Einstein who did a drag-and-drop of the official .zip library (containing a metadata.db and a .json file) onto the Calibre GUI, and then complained vociferously that one of his file conversion plugins "broke" after he "installed" Q&S. No, he did not read the User Guide, or even the paragraph just above this one. The moral of this story: Read the User Guide after thoroughly reading this post.

The Q&S menu icon and many menu choices will not respond to mouse clicks unless the current Calibre library name begins with this exact phrase: "QuarantineAndScrub".

You can have as many Q&S libraries as you wish, but they all must begin with "QuarantineAndScrub_ ".

The special Q&S library contains the add-on's metadata.db and metadata_db_prefs_backup.json files.

You may not use this add-on with any Calibre library other than the official Q&S Library that you download from the attached .jar file. That is why this is an add-on, not just a plugin.

Execution Notes:

Spoiler:

Refer to the submenu named “Read Q&S Instructions", which opens the User Guide .pdf file.

Q&S requires that you closely review every Q&S Job Log, preferably while it is executing. This is for many reasons, all of which are for your benefit. If you do not wish or intend to review each and every Q&S Job Log, and follow any instructions or take note of any warning or error messages they may provide, then Q&S will not work properly for you and you should uninstall it.

Q&S is resource-intensive, so you should never have more than ~150 books in your Q&S Library at any one time. Use Q&S in small 'batches'.

Refer to the User Guide, which includes a workflow diagram.

Special Usage Notes:

Spoiler:

This add-on is intended for use only by "Advanced Calibre Users" who are somewhat technical in nature, comfortable with ambiguity, essentially autodidactic, and also fluent in Calibre's functionality and library architecture.

The above requirement also applies to the person installing Q&S.

If someone wants to use Q&S but does not have the skills to install it, then they should not attempt to use it, nor should they have someone else install it for them. If you do not have the skills to install it, then you do not have the skills to use it.

The User Guide must be read prior to installing and using this add-on. It is 54 pages in length for a reason. Q&S Derive Genres has an additional 13 page User Guide due to its complexity. In total, the User Guides for Q&S total 67 pages.

The User Guide is your source of support for Q&S.

99.9999% of Calibre users do not need Q&S, and should not install it.

Q&S is designed for users who have already been using Calibre for quite some time, so the Q&S User Guide assumes the reader is already familiar with Calibre in great detail.

The User Guide is your source of support for Q&S.

If you are not comfortable with ambiguity, or not essentially autodidactic, then you should not attempt to install or use Q&S.

If you have an aversion to reading instructions and related examples, you should not attempt to install or use Q&S.

If you cannot understand or follow the instructions in this post or the User Guide itself, then it will be less frustrating to just uninstall Q&S rather than to continue being frustrated.

The User Guide is your source of support for Q&S.

Remember to always run the "After Version Upgrade" job near the top of the menu after upgrading to any new version of Q&S. It adds new tables and views, plus additional Global Authors used for Author validation. Each of your Q&S libraries must be upgraded individually, since each has its own metadata.db file originally downloaded from this thread.

Important Message for New Users of Q&S:

Spoiler:

@NewUsersOfQ&S:

Please do yourself a huge favor and read these Q&S OP attachments before you begin to do anything with Q&S. They will save you a great deal of time, energy and frustration:


https://www.mobileread.com/forums/att...6&d=1424120599


https://www.mobileread.com/forums/att...3&d=1435073179


Also, it would be a great idea to peruse these posts after you have thoroughly analyzed and digested the above 2 attachments to the OP:

https://www.mobileread.com/forums/sho...65602#poststop

https://www.mobileread.com/forums/sho...64#post3103064

It goes without saying that you must read the User Guide after you have read the above.

Afterwards, please "learn the Q&S process" with your new Q&S library when it has only a few (~ 10, not 100 or 1,000 or 5,000 or 10,000...) copies of random or representative books in it, each with a different author. Take them through the "process" from start to finish. Along the way, you will have questions. Please reread the attachments and links listed above. Then, add a few more copies of your books that have different types of issues that you want Q&S to solve, and run them through the process too. Again, you will have questions, and should again reread the User Guide and other information.

Taking "baby steps" when first learning Q&S pays huge dividends later.

Companion Plugin - Essential: View Manager
Companion Plugin - Recommended: Multi-Column Search


Critical OS Notes:

Spoiler:

[1] If you use Calibre Portable: CALIBRE_TEMP_DIR is used by Calibre for its temporary file directory, and TMPDIR et al are used by generic Python for its temporary file directory. CALIBRE_TEMP_DIR must be consistent with TMPDIR et al. Implement [2], below. Also, ensure the PATH variable includes Calibre.

[2] If you have personalized your "temporary" directories, such as using a RAMDisk for your temporary files, you must set certain standard environment variables such that Python finds and uses the one that you want Python to find and use. From the Python documentation:

Spoiler:

The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the TMPDIR, TEMP or TMP environment variables

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

• The directory named by the TMPDIR environment variable.
• The directory named by the TEMP environment variable.
• The directory named by the TMP environment variable.

• A platform-specific location:
  On Windows, the directories C:\TEMP, C:\TMP, \TEMP, and \TMP, in that order.

• On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.

• As a last resort, the current working directory.

Minimum Calibre Version: 6.15


Version History:

Spoiler:

Remember to always run the "After Version Upgrade" job on the menu after upgrading to any new release or version of Q&S.


Version 3.6.122 - 20 April 2023 New "link" column in all normalized custom columns now supported. Removed DDC/LCC functionality due to deprecation of classify API; use Library Codes plugin instead. Minimum Calibre Version of 6.15.

Version 3.6.121 - 15 April 2023 Links added to normalized custom column tables.
Version 3.6.120 - 06 February 2023 Qt.core; List Editor Qt changes.
Version 3.6.119 - 06 October 2022 ~2592 new Global Authors have just been added to the previously existing ~20194.
Version 3.6.118 - Development Use Only
Version 3.6.117 - 14 April 2022 Qt6 Compatibility. Minimum Version Calibre 5.99.8.
Version 3.6.115 - 23 January 2021 Fictiondb.com as a source of Web Source Series Validation Data (WSSVD) replaced. New menu item: Download Web Series, Indexes, LCCN, ISBN, PermaLinks, Tags, Publisher, PubDate [Selected Single Real Author] The primary purpose of this is to add new WSSVD to Q&S. Extra metadata is a "gift", and totally secondary to the primary purpose. The new ability to create "empty books" with full metadata for a single author's books NOT currently in Q&S is a bonus as well. Also, several menu selections have had their item names improved to clarify their use and dependencies, if any.
Version 3.6.114 - 24 December 2020 Miscellaneous.
Version 3.6.113 - 10 December 2020 Allow changes to a Q&S Library's library_id UUID to be acceptable.
Version 3.6.112 - 30 September 2020 Config Dialog changes.
Version 3.6.111 - 09 June 2020 Technical changes in two Series-Scrubbing jobs that both create and use WSSVD.
Version 3.6.110 - 08 June 2020 Technical changes in several Series-Scrubbing jobs that either create or use WSSVD.
Version 3.6.109 - 03 June 2020 Technical changes in: "Download Work Series/Titles/Indexes from Web Source for All Series for a Single Author [Single Author Only]"
Version 3.6.108 - 16 May 2020 Technical changes in Author-level scrubbing and Series-level consolidation and renaming
Version 3.6.107 - 18 February 2020 Technical changes to Series-level renaming and consolidation job.
Version 3.6.106 - 13 February 2020 Q&S has been overhauled, refurbished, refactored and restructured to greatly improve its performance and reliability,
and to make it fully Python 3.8 compatible. Retested with both Calibre 4.10.1 (Python 2.16+) and Calibre 4.99.4 (Python 3.8).
Version 3.6.105 - 20 July 2019 Python 3 compatibility; Minimum Calibre Version now 3.41.3; Many performance improvements.
Version 3.6.104 -15 April 2019 Miscellaneous Series Scrubbing's personal BeautifulSoup changed to use Standard Calibre's BeautifulSoup
Version 3.6.103 -19 May 2017 Book-Level Scrubbing Tweaks
Version 3.6.102 -05 May 2017 Allow themes with user-defined icons.
Version 3.6.101 -16 February 2016 New Global Authors (total now: ~ 20,215) and New Book Awards (total now: ~ 4,194).
Version 3.6.99 -08 February 2016 Revised User Guide.
Version 3.6.98 -06 February 2016 ToolTips added to Menus; Technical webscraping tweaks.
Version 3.6.97 -04 January 2016 Updated User Guide.
Version 3.6.96 -01 January 2016 Additional data-mining of historical web source series detail for use by Miscellaneous Scrubbing at the end of Book-Level Scrubbing; Series-Level Scrubbing job "Rename Work Series Name to Web Series Name" has been enhanced and optimized for speed and performance.
Version 3.6.95 -31 December 2015 Major speed improvements at the end of Book-Level Scrubbing while Miscellaneous Scrubbing is changing Work Title to Web Title when they are different yet similar but the Work Series and its Index are identical to the Web Series and its Index.

Version 3.6.94 -30 December 2015 Automatic display of Calibre Job Dialog after submission of Author-Level and Book-Level Scrubbing Jobs.
Version 3.6.93 -29 December 2015 Major speed improvements at the end of BookLevel Scrubbing while Miscellaneous Scrubbing is changing Work Series Index to Web Series Index when appropriate. Improved job progress notifications for that functionallity as well.
Version 3.6.92 -04 September 2015 Minor Tweak to accomodate a change in Standard Calibre's usage of a particular user-defined SQLite function in Triggers.
Version 3.6.91 -14 June 2015 Job Speed Increases, especially in Book-Level Scrubbing.
Version 3.6.90 -08 June 2015 New Title Scrubbing Rules; New Scrubbing Rules for CBZ/CBR/Graphic Novels.
Version 3.6.89 -07 June 2015 Speed enhancement for huge libraries when performing foreground "Delete Non-ISBN/ISSN/OCLC Identifiers".
Version 3.6.88 -05 June 2015 Miscellany
Version 3.6.87 -03 June 2015 Miscellaneous Enhancements.
Version 3.6.86 -02 June 2015 New Menu Item: Q&S>>>[Menu]Special Tools>>>Update Pseudonym Custom Column Using Real Authors [Selected Books]; Revised User Guide.
Version 3.6.85 -01 June 2015 ~500 New Global Authors; New Global Pseudonym Table with ~559 Author Pseudonyms to assist in maintaining the Author Mapping Table.
Version 3.6.84 -31 May 2015 Miscellaneous Enhancements.
Version 3.6.83 -29 May 2015 New Menu Item: Q&S>>>[Menu]Scrub at Series Level>>>Change a Specific Work Series Index [Single Book]; Miscellaneous tweaks.
Version 3.6.82 -28 May 2015 New Menu Item: Q&S>>>[Menu]Special Tools>>>Update Custom Column(s) per Table "Tags CC Mapping Control" [Selected Books]; Miscellaneous Changes in Series Level Renaming using WSSVD; Enhancement in Work Title Renaming using WSSVD; Revised User Guide.
Version 3.6.81 -25 May 2015 New Menu Item: Q&S>>>[Menu]Special Tools>>>Change Work Author (Green Only) [Single Book Only].
Version 3.6.80 -24 May 2015 Miscellaneous Enhancements.
Version 3.6.79 -23 May 2015 Manga Scrubbing: Enhanced; New Menu Item: Q&S>>>[Menu]Scrub at Series Level>>>Search&Replace Specific Work Series Name [Selected Books]; New Menu Item: Q&S>>>[Menu]Scrub at Series Level>>>Search&Destroy Specific Work Series Name [Selected Books]; New Menu Item: Q&S>>>[Menu]Scrub at Series Level>>>Add a Specific Work Series Name & Index [Selected Books].
Version 3.6.78 -22 May 2015 New Menu Item: Q&S>>>[Menu]Scrub Work Tags Using Power Tools>>>Search&Replace Specific Work Tag [Selected Books]
Version 3.6.77 -20 May 2015 Miscellaneous Enhancements.
Version 3.6.76 -20 May 2015 New Menu Item: Q&S>>>[Menu]Scrub Work Tags Using Power Tools>>>Add a Single Work Tag [Selected Books]
Version 3.6.75 -19 May 2015 Creation of New Work Tags from Book Awards: tweaked.
Version 3.6.74 -19 May 2015 New Menu Item: Q&S>>>[Menu]Scrub Work Tags Using Power Tools>>>Search&Destroy Specific Work Tag [Selected Books]
Version 3.6.73 -18 May 2015 ~140 New Book Awards Added via the "After Version Upgrade" Job; Q&S Intra-Ecosystem Copy Job enhanced.
Version 3.6.72 -15 May 2015 New Scrubbing Rules; Revised User Guide.
Version 3.6.71 -13 May 2015 Miscellaneous improvements.
Version 3.6.70 -11 May 2015 New Menu Item: Q&S>>>[Menu]Special Tools>>>Import Q&S Reference from Service Pack .db. The file qs_service_pack.zip and instructions are attached to the very first post after this thread's Original Post.
Version 3.6.69 -30 April 2015 Download of Web Source Series Validation Data tweaked due to changes in Web Source's available data.
Version 3.6.68 -22 April 2015 New Book Awards and Other Miscellaneous.
Version 3.6.67 -19 April 2015 Miscellany.
Version 3.6.66 -13 April 2015 Miscellany.
Version 3.6.65 -12 April 2015 Miscellany.
Version 3.6.64 -07 April 2015 Transient DB-locked error in one of the Series-Level Scrubbing jobs resolved. If it reoccurs (usually after switching among multiple Q&S libraries quickly, then running the aforementioned job immediately upon arrival), simply exit from Calibre, then restart it.
Version 3.6.63 -03 April 2015 ~100 New Book Awards; New Title Scrubbing Rules.
Version 3.6.62 -01 April 2015 Propagation of Work Tags to/from Books with Identical Work Series: changed to not propagate Work Tags derived from the Book Awards Table, as they are specific to a particular Book, not the Work Series it belongs to.
Version 3.6.61 -30 March 2015 ~331 New Book Awards; ~85 New Global Authors; Enhanced Automatic Reformatting of Initials of Global Authors, Pristine Authors and Authors in the Book Awards Table.
Version 3.6.60 -29 March 2015 New Title Scrubbing Rules; Improved Scrubbing of Publishers from Title.
Version 3.6.59 -28 March 2015 New Customization Option for Book Tag Minimization to ignore all Book Awards previously created as new Work Tags by Book Level Scrubbing or Standalone Tag Scrubbing.
Version 3.6.58 -27 March 2015 New Title Scrubbing Rules; Miscellaneous enhancements.
Version 3.6.57 -27 March 2015 ~450 new Book Awards Table entries added; Revised User Guide.
Version 3.6.56 -26 March 2015 New:Book Awards Table Preloaded with ~2,700 Books with Author and Awards; New Work Tags are created for books with a matching entry in the Book Awards Table; ~1,000 New Global Authors Added. Total Global Authors now: ~19,050; New DB Version 4.96.0. Revised Title Scrubbing Rules.
Version 3.6.54 -25 March 2015 Miscellany.
Version 3.6.53 -24 March 2015 New Menu Item for Series Level Scrubbing: Download Web Series/Titles/Indexes from Web Source for All Fiction Series for a Single Real Author (Regardless of Work Series, if any); Revised User Guide.
Version 3.6.52 -22 March 2015 Miscellany; Revised User Guide.
Version 3.6.51 -21 March 2015 Many New Title Scrubbing Rules, Especially for CBZ/CBR Types of Titles.
Version 3.6.50 -20 March 2015 New: Automatic Graphic Novel Book-Level Scrubbing for Formats CBZ/CBR (or Other Formats with a Tag of "Graphic Novel"); New Title Rules and Series Rules for Capitalizing Common CB Words; ~1000 New Global Authors Added. Total Global Authors now: ~18,100+
Version 3.6.49 -19 March 2015 Derive Genres (Q&S Version): Miscellaneous enhancements.
Version 3.6.48 -19 March 2015 ~350 New Global Authors Added. Total Global Authors now: 17,000+.
Version 3.6.47 -18 March 2015 ISSN now supported along with ISBN for retrieving DDC and/or LCC with OCLC-OWI. If a book has both, the ISBN takes priority.
Version 3.6.46 -17 March 2015 New: Author Mapping Table (User Maintained) to Change an Author's Name to Another Name (e.g. Pseudonym to Real Name or to Another Pseudonym); Revised User Guide.
Version 3.6.44 -16 March 2015 Miscellaneous enhancements; Revised User Guide.
Version 3.6.43 -14 March 2015 New: The OCLC OWI identifier is now added whenever DCC or LCC are updated; Revised User Guide.
Version 3.6.42 -13 March 2015 New Menu Item: Dewey Decimal Classification and Library of Congress Classification - See Page 63 of the Revised User Guide.
Version 3.6.41 -11 March 2015 All Q&S Menus are now "Tear-Off"; Revised User Guide; Miscellaneous Enhancements.
Version 3.6.40 -09 March 2015 Author Level Scrubbing refinement; New Global Authors; Miscellaneous Enhancements.
Version 3.6.39 -08 March 2015 New Title Scrubbing Rules; New Global Authors; Enhancement to automate (otherwise manual) certain table Tag Priorities maintenance tasks.
Version 3.6.38 -07 March 2015 New Title Scrubbing Rules; New Global Authors; Bug Fix in Copy Rules etc. from/to Best to/from Target Q&S Libraries.
Version 3.6.37 -06 March 2015 Author Level Scrubbing changes.
Version 3.6.36 -06 March 2015 Miscellaneous improvements.
Version 3.6.35 -05 March 2015 New Author Level and Book Level Scrubbing Rules; Revised User Guide.
Version 3.6.34 -04 March 2015 New Menu Item: "Swap Work Title and Work Series" (Under "[Menu] Q&S Special Tools"); Many New Title Scrubbing Rules; Pristine Authors Now Copied Back from "Target" Q&S Libraries to "Best" Q&S Library.
Version 3.6.33 -03 March 2015 New Author Level Scrubbing rules.
Version 3.6.32 -02 March 2015 Changes to Author Level Scrubbing
Version 3.6.31 -02 March 2015 New Title Scrubbing Rules; New Series Scrubbing Rules; Enhanced Author Level Scrubbing.
Version 3.6.30 -01 March 2015 Many New Title Scrubbing Rules; Enhanced Author Level Scrubbing; New DB Version 4.95.
Version 3.6.29 -28 February 2015 New: Derivation of User Custom Columns from Title (see Page 62 of the User Guide); Revised User Guide; New DB Version 4.94.0.
Version 3.6.28 -26 February 2015 Miscellaneous enhancements.
Version 3.6.27 -25 February 2015 New DB Version 4.93.0; Miscellaneous enhancements to Author Level Scrubbing, Series Consolidation/Renaming, and Validation of Work Series Using Web Series Data; Revised User Guide.
Version 3.6.25 -24 February 2015 Job Speed Increased for Work Series Consolidation/Renaming; New Dialog Box for Selecting Work Series Consolidation/Renaming Series Name Source Priority Sequence; New DB Version 4.92.0.
Version 3.6.24 -23 February 2015 New Dialog Box for Selecting Table Maintenance of Tags by Comments Candidates and then Tags by Comments.
Version 3.6.23 -22 February 2015 Global Authors Table updated with more permutations of their names and initials to increase Author Level Scrubbing accuracy; Ditto for Pristine Authors Table; Series Level Renaming of Work Series Name per Validated Web Series Name enhanced to include the new functionality added in Version 3.6.22 for Book Level Scrubbing.
Version 3.6.22 -21 February 2015 Book Level Scrubbing enhanced to add a missing Work Series if that Web Series for the same Work Author exists already in the Historically Cumulative Web Series Detail downloaded previously as part of the validation of Work Series and Work Series Index (See Page 16 of the Revised User Guide for an example); Miscellaneous enhancements to Author Level Scrubbing.
Version 3.6.21 -20 February 2015 Miscellaneous Enhancements and Job Speed Improvements.
Version 3.6.20 -19 February 2015 Author Level Scrubbing Enhanced; New Work Title Scrubbing Rules; Jobnames Improved; New DB Version 4.91.0.
Version 3.6.19 -16 February 2015 Many New Work Title Scrubbing Rules; Improved Author Level Scrubbing; Revised User Guide; New DB Version 4.9.0.
Version 3.6.18 -14 February 2015 Many New Work Title Scrubbing Rules.
Version 3.6.17 -14 February 2015 New Menu Item: Copy (just) Real Series to Work Series (i.e., standalone); User Guide Revised; After Version Upgrade Job: No Longer Inception-to-Date Capable. If you have not kept your Q&S Libraries up to the current major version (4.), as of this release you will not be able to do so. Only totally new or newly changed tables and views will be delivered starting now. As previously forewarned in the OP in the "use it or lose it" comment, if your DB Version is currently less than 4.0.0, you will have to redownload the "official library", extract it, and move your books from your non-current Q&S library to the new up-to-date one. And keep it current.
Version 3.6.16 -10 February 2015 New DB Version 4.8.0; Tags by Comments Candidates Creation: Speed Improvement.
Version 3.6.15 -10 February 2015 Revised User Guide.
Version 3.6.14 -07 February 2015 New Global Authors added.
Version 3.6.13 -06 February 2015 Author Level Scrubbing: Improvements; Book Level Scrubbing: New Title Scrubbing Rules.
Version 3.6.12 -03 February 2015 New DB Version 4.7.0; Increase in performance of Series Consolidation/Renaming
Version 3.6.11 -02 February 2015 Miscellany; Revised User Guide.
Version 3.6.10 -31 January 2015 Miscellaneous enhancements.
Version 3.6.9 -30 January 2015 Miscellany; Revised User Guide; New DB Version 4.6.0; New "Manga Awareness".
Version 3.6.8 -28 January 2015 New "Tag String Replacement Rules" table, which is user maintained; Revised User Guide; New DB Version 4.5.0.
Version 3.6.7 -27 January 2015 New "Tag Capitalization Rules" table, which is user maintained; Revised User Guide; New DB Version 4.4.0.
Version 3.6.6 -26 January 2015 New Database Version 4.3.0
Version 3.6.5 -25 January 2015 New Scrubbing Rules for Work Title.
Version 3.6.4 -24 January 2015 Miscellany.
Version 3.6.3 -22 January 2015 Menu System Enhanced.
Version 3.6.2 -20 January 2015 New Global Authors for Validation; Miscellaneous enhancements; Revised User Guide.
Version 3.6.1 -19 January 2015 New Database Version 4.2.0; Miscellaneous enhancements.
Version 3.6.0 -18 January 2015 New "Tag Combinations Rule Table", which Tag Scrubbing uses to derive new Work Tags based on combinations of up to three Work Tag Keywords per Rule; Revised User Guide.
Version 3.5.10-17 January 2015 Major enhancements to Author Level Scrubbing; New Book Level Scrubbing Rules for Title and Author; New Database Version 4.1.0.
Version 3.5.9- 16 January 2015 Enhanced Book Level Scrubbing and Book Level Tag Scrubbing to automatically insert missing Work Tags into table Tag Priorities so that table can be fully maintained prior to executing Minimize Tags Using Tag Priorities; enhanced Auto-Populate Tags-by-Comment Rule Table; Revised User Guide.
Version 3.5.8- 15 January 2015 New Menu Item: Auto-Populate Tags-by-Comment Rule Table using Tag Rules Table plus Comments of All Books; Revised User Guide.
Version 3.5.7- 15 January 2015 Book Level Scrubbing db lock at end of job: fixed; Author Level Scrubbing: enhancement.
Version 3.5.6- 14 January 2015 Standalone Book Level Tag Scrubber: Enhanced to include derivation of new Work Tags from Comments (table _tags_by_comment), just like Book Level Scrubbing already has; this obviates the need to run BL Scrubbing just for that purpose. Enhanced inter-library rules cloning within the Q&S ecosystem.
Version 3.5.5- 13 January 2015 Validate Work Series from Web Source: improvements; Propagation of Tags by Series: bug fix.
Version 3.5.4- 13 January 2015 New database version 3.9.0
Version 3.5.3- 13 January 2015 Enhancements to Q&S Derive Genres; New Q&S Derive Genres User Guide; Q&S User Guide revised.
Version 3.5.2- 12 January 2015 Miscellaneous Enhancements; Revised User Guide.
Version 3.5.1- 11 January 2015 Enhancements to Q&S Derive Genres, especially in validating/correcting user errors made while maintaining Boolean Tag Rules in the Q&S DG Rules tables; Integration of Q&S DG rules tables into Q&S inter-library rules copying added.
Version 3.5.0- 10 January 2015 New: Derive Genres core functionality has been integrated with Q&S, using Q&S database tables instead of any .csv files like the generic Derive Genres plugin uses. User Guide has been revised.
Version 3.4.2- 09 January 2015 New User-Maintained "Tags by Comment Table" to derive Work Tags from Book Comments; Revised User Guide.
Version 3.4.1- 08 January 2015 New Scrubbing Rules and Work Series Validation tweaks.
Version 3.4.0- 07 January 2015 New User-Maintained "Series Rules Table" that uses regular expressions to change/delete words in Work Series (analogous to the Title Rules Table); Revised User Instructions.
Version 3.3.9- 06 January 2015 New Scrubbing Rules for Work Title and Work Series; Speed optimization of Default Work Tag by Author.
Version 3.3.8- 04 January 2015 New Scrubbing Rules for Work Title, Work Series and Work Tags; UTF8 bug fixes.
Version 3.3.7- 03 January 2015 UTF8 bug fix in Book Level Scrubbing.
Version 3.3.6- 01 January 2015 New Scrubbing Rules for Work Title, Work Series and Work Tags; Some menu icons revised for clarity of purpose.
Version 3.3.5- 30 December 2014 Database Table and View Changes; Revised User Guide; Official Q&S Database to Download [for new users only] Upgraded to DB Version 3.5.0.
Version 3.3.4- 27 December 2014 New Menu Item: "Change Default Tag of 'Real' Author to Selected Work Tag [Selected Books]"; Revised User Guide.
Version 3.3.3- 26 December 2014 "Change Selected Work Tags to Default Tag of 'Real' Author [Selected Books]": improvements.
Version 3.3.2 - 25 December 2014 New Scrubbing Rules for Title and Series.
Version 3.3.1 - 23 December 2014 Bug fix.
Version 3.3.0 - 22 December 2014 New Menu Item: "Change Selected Work Tags to Default Tag of 'Real' Author [Selected Books]"; Revised User Guide; New user-maintained table: _tags_by_author_default .
Version 3.2.10-21 December 2014 Tag Minimizer: tweaks to improve performance.
Version 3.2.9 -20 December 2014 "Easy-Add Tags-to-Purge": new column added to specify whether /regex/ should be used for new Tag Purge Rules (previously always did); Miscellaneous Enhancements; Revised User Instructions.
Version 3.2.8 -19 December 2014 "Easy-Add Tags-to-NEWTAG Mappings": new column added to specify whether /regex/ should be used for new Tag Rules (previously always did); Miscellaneous Enhancements.
Version 3.2.7 -18 December 2014 Speed optimizations; Changes to SQLite Tables and Views to improve functionality and performance; Book Work Title Renaming per Web Source Web Title enhancements.
Version 3.2.6 -16 December 2014 Major enhancements to the "Rename Work Series" job; now all historically cumulative Web Source Series Data downloaded to validate Work Series/Title/Index (from anywhere in the Q&S Library ecosystem at any time in the past) is available for renaming Work Series/Title/Index in any Q&S Library. Download once, then use the results anywhere in any Q&S Library at anytime in the future.
Version 3.2.5 -15 December 2014 Bug created in version 3.1.9 fixed.
Version 3.2.4 -15 December 2014 This Version does for Work Tags and Web Source Series/Title/Index Data what the prior version did for Real Tags. The User Guide has been revised to reflect all of the new functionality.
Version 3.2.3 -14 December 2014 Real Tags from all Q&S Libraries now consolidated into the Best Q&S Library and made available as choices in the "Easy-Add Real-Tags-to-Purge" menu option used to easily add new "purge rules" to the Tag Rules table. See Version 3.2.0 below.
Version 3.2.2 -13 December 2014 New Scrubbing Rules; Miscellaneous Enhancements.
Version 3.2.1 -12 December 2014 Miscellaneous Enhancements.
Version 3.2.0 -11 December 2014 New Menu Item: Automatically Copy Tag & Title Rules From/To Multiple Q&S Libraries; New Customization Items; Revised User Guide.
Version 3.1.9 -10 December 2014 Changed From "All Books" to "Selected Books": Validate Work Series, Work Titles & Work Indexes Via Web Source [Fiction Series of Selected Books]
Version 3.1.8 -10 December 2014 Menu Changes for Clarity plus User Guide Improvements.
Version 3.1.7 -09 December 2014 Changed From "All Books" to "Selected Books": Tag Scrubber (standalone) & Tag Minimizer.
Version 3.1.6 -08 December 2014 New Scrubbing Rules
Version 3.1.5 -08 December 2014 Miscellaneous Enhancements.
Version 3.1.4 -07 December 2014 New Menu item added: "Purge All Work Data [Selected Books Only]"; Other Enhancements.
Version 3.1.3 -07 December 2014 Miscellaneous speed optimizations.
Version 3.1.2 -06 December 2014 Job "Copy Q&S All Dirty Data to Work Data [All Books]" optimized for large libraries.
Version 3.1.1 -06 December 2014 "Copy Q&S All Dirty Data to Work Data [Selected Books (Max=100)]" Menu item added.
Version 3.1.0 -06 December 2014 Q&S DB Version 3.1.0
Version 3.0.9 -05 December 2014 "Easy-Add Tags-to-NEWTAG Mappings": tweaks.
Version 3.0.8 -03 December 2014 Miscellaneous Enhancements.
Version 3.0.7 -02 December 2014 Copying Work Tags to Real Tags: UTF8 bug fix.
Version 3.0.6 -01 December 2014 Miscellaneous Enhancements; Revised User Instructions.
Version 3.0.5 -30 November 2014 Web Source Series Detail Retrieval in Validation of Work Series: HTML Parsing Change.
Version 3.0.4 -28 November 2014 Improvement in ISBN Derivation from Title, Author, and Tags.
Version 3.0.3 -28 November 2014 New Scrubbing Rules.
Version 3.0.2 -25 November 2014 Q&S Automatically Converts Valid ISBN10s to ISBN13s, Removes ISBN13 Dashes & Prefixes, and Deletes Garbage Posing as ISBNs.
Version 3.0.1 -23 November 2014 Enhancement to "Minimize Work Tags Using Tag Priorities" for Work Tags having no priorities.
Version 3.0.0 -22 November 2014 New Tool to Minimize Work Tags Using Tag Priorities [New Table: _tag_priorities]; Revised User Instructions;
Version 2.9.9 -21 November 2014 New Menu Option to Copy Only Real Tags to Work Tags [e.g. Prior to Standalone Tag Scrubber for Pristine Books Copied Backwards to Q&S just to Scrub Their Tags with No Other Scrubbing].
Version 2.9.8 -20 November 2014 Unicode in Work Tag Scrubbing: Bug Fix.
Version 2.9.7 -20 November 2014 Propagation of Work Tags by Series Improvements, including BISAC format capitalization.
Version 2.9.6 -20 November 2014 DB Upgraded to Version 2.9.6
Version 2.9.5 -20 November 2014 New Series Derivation from Title Rules; Copying Work to Real improvements; bug fixes.
Version 2.9.3 -19 November 2014 Enhancements to Easy-Add Work Tags to NEWTAG Mappings to Tag Rules Table
Version 2.9.2 -18 November 2014 Miscellaneous Enhancements.
Version 2.9.1 -18 November 2014 Miscellaneous Enhancements.
Version 2.9.0 -17 November 2014 New User-Maintained "Title Rules Table" that uses regular expressions to change/delete words in titles; Revised User Instructions.
Version 2.8.2 -16 November 2014 Revised User Instructions [within the .zip]
Version 2.8.1 -16 November 2014 Miscellaneous Functionality Improvements; bug fixes.
Version 2.8.0 -15 November 2014 New "Easy-Add Tags-to-NEWTAG Mappings" to Tag Rules Table feature; Revised User Instructions.
Version 2.7.1 -14 November 2014 New Book Level Scrubbing Rules
Version 2.7.0 -13 November 2014 New Feature: "Easy-Add Tags to Purge" to the Tag Rules Table; Revised User Instructions.
Version 2.6.1 -12 November 2014 Miscellaneous Functionality Enhancements
Version 2.6.0 -12 November 2014 Expanded Functionality for Renaming Work Series/Titles/Indexes to Web Source Names/Values; Revised User Instructions.
Version 2.5.9 -12 November 2014 Bug Fixes.
Version 2.5.8 -11 November 2014 New Series Derivation Rules; Revised User Instructions.
Version 2.5.7 -11 November 2014 New Tool: Rename Work Title to Similar Web Title for Identical Series/Index.
Version 2.5.6 -11 November 2014 Miscellaneous Functionality Enhancements.
Version 2.5.5 -10 November 2014 Miscellaneous Functionality Enhancements.
Version 2.5.4 - 9 November 2014 Series Deriver & Web Validation of Work Series: Enhancements.
Version 2.5.3 - 8 November 2014 Miscellaneous Functionality Enhancements.
Version 2.5.2 - 7 November 2014 Miscellaneous Functionality Enhancements.
Version 2.5.1 - 7 November 2014 Bug fix.
Version 2.5.0 - 6 November 2014 New Scrubbing Option: Rename Work Series Name to Web Series Name [Previously Verified Fiction Series Only]; Revised Instructions; and bug fixes.
Version 2.4.2 - 5 November 2014 Enhancements to Verify Work Series Index Using Web Lookup
Version 2.4.1 - 5 November 2014 Minor tweaks.
Version 2.4.0 - 4 November 2014 New Miscellaneous Scrubing Option: Verify Work Series Index Using Web Lookup; Revised Instructions; and bug fixes.
Version 2.3.2 - 31 October 2014 New Standalone Purge Pristine Authors/Series Job[Use if Pristine Library isn't so pristine]; Enhancements to Author Level Scrubbing for libraries with a huge number [>10K] of Pristine Authors; Revised Instructions; and bug fixes.
Version 2.3.1 - 30 October 2014 Booklevel Tag Scrubber Now Available as Standalone Job; Revised Instructions; and bug fixes.
Version 2.3.0 - 29 October 2014 New Tag Propagator by Series; and Revised Instructions.
Version 2.2.0 - 28 October 2014 Enhancements to Series Consolidator/Renamer
Version 2.1.2 - 28 October 2014 Enhanced Scrubbing Rules for Series derivation from Title
Version 2.1.1 - 27 October 2014 Tag Scrubber now removes Author, Title and Series found in Tags; New Scrubbing Rules; and Revised Instructions.
Version 2.1.0 - 26 October 2014 New Series Scrubber/Consolidator/Renamer and Revised Instructions.
Version 2.0.0 - 24 October 2014 New User-Maintained (via a GUI) "Tag Rules Table" that can use regex to scrub (change or delete) tags. GUI is by http://sqlitebrowser.org/ that is designed for frequent end-user table maintenance.
Version 1.6.5 - 22 October 2014 New Work Tag Derivation Rules and bug fixes.
Version 1.6.4 - 22 October 2014 Additional Global Authors; Improvements in Copying Work Tags to Real Tags; and bug fixes.
Version 1.6.3 - 20 October 2014 New Scrubbing Rules; Revised Instructions; and bug fixes.
Version 1.6.2 - 18 October 2014 New Scrubbing Rules; New Global Authors for Validation
Version 1.6.1 - 17 October 2014 New Scrubbing Rules
Version 1.6.0 - 15 October 2014 New "Tags To Purge When Found" Table; Revised Instructions; New Scrubbing Rules
Version 1.5.9 - 14 October 2014 New Scrubbing Rules
Version 1.5.8 - 13 October 2014 New Scrubbing Rules
Version 1.5.7 - 13 October 2014 New Scrubbing Rules
Version 1.5.6 - 12 October 2014 New Functions and Features; New Scrubbing Rules
Version 1.5.5 - 11 October 2014 New Scrubbing Rules
Version 1.5.4 - 11 October 2014 New Scrubbing Rules;New Job to Upgrade Q+S DB; Revised Instructions; and bug fixes.
Version 1.5.3 - 10 October 2014 New Job to Purge Work Data;New Scrubbing Rules; bug fixes.
Version 1.5.2 - 10 October 2014 Create new SQLite table not in Version 1.0.0
Version 1.5.0 - 9 October 2014 New Scrubbing Rules and Revised Instructions.
Version 1.4.0 - 8 October 2014 New Scrubbing Rules, enhancements, bug fixes.
Version 1.3.0 - 7 October 2014 Prevention of DB Locking Errors.
Version 1.2.0 - 7 October 2014 New Scrubbing Rules; bug fixes.
Version 1.1.0 - 6 October 2014 New functions and features.
Version 1.0.0 - 4 October 2014 Initial release.

[DaltonST]

Future use.

[DaltonST]

This is reserved for future use.

[DaltonST]

Reserved for future use for this plugin.

[DaltonST]

New release: Version 1.2.0 - 7 October 2014 New Scrubbing Rules; bug fixes.

New users (only) must download the file at https://www.mobileread.com/forums/att...7&d=1412424298 . Refer to the original post for details.

[DaltonST]

New release: Version 1.3.0 - 7 October 2014 Prevention of DB Locking Errors.



New users (only) must download the file at https://www.mobileread.com/forums/att...7&d=1412424298 . Refer to the original post for details.

[DaltonST]

New Release: Version 1.4.0 - 8 October 2014. New Scrubbing Rules, enhancements, bug fixes.

[DaltonST]

New Release: Version 1.5.0 - 9 October 2014 . New Scrubbing Rules; Revised Instructions; bug fixes.



New users (only) must download the file at https://www.mobileread.com/forums/att...7&d=1412424298 . Refer to the original post for details.

[DaltonST]

New Release: Version 1.5.1 - 10 October 2014 Create new SQLite table not in Version 1.0.0

The Booklevel Scrubbing Job requires this new table, and will fail without it.

Please use the very latest release, which is not 1.5.1.

[DaltonST]

Version 1.5.2 - 10 October 2014 Create new SQLite table not in Version 1.0.0

New Release: Version 1.5.2 - 10 October 2014 Create new SQLite table not in Version 1.0.0

The Booklevel Scrubbing Job will fail without this patch.

This new version is required.

[DaltonST]

The latest release is: Version 1.5.3 - 10 October 2014 New Purge Work Data Job; New Scrubbing Rules; bug fixes

IMPORTANT: The Index that Calibre uses to determine if your installed plugin has an update may lag by a long time between the time I upload the latest release and it gets updated. The Index process is tedious and burdonsome on the Admins and not at all designed for frequent updates. So, ALWAYS check this thread to determine if there is a newer release available than you have. If so, please download it manually.

In summary, do not trust what Calibre says about what the latest release of QuarantineAndScrub is. Always check the OP.

This plugin gets tested thoroughly prior to uploading on a Dell Laptop (Windows 7) with a test library of 600+ books with mostly malformed and misplaced metadata, and a Dell Touch-Screen 23" all-in-one (Windows 8.1) with a test library of 200+ of both clean and unclean books. However, you-know-what happens; when it does, I fix it immediately and upload the newest version as soon as it is tested.

The menu has a "PM the Developer" submenu option, so I hear about issues as fast as they are PM'd to me. Please continue to use that submenu option to communicate with me.

Please always check this thread prior to using this plugin. I am actively developing new functions and features every day.

Thanks for your understanding.


DaltonST

[davidfor]

DaltonST: If you update the plugin, you don't need to update the index thread. That is used by the server to get the list of plugins. Once a plugin is in the index, the server checks the plugin thread for any changes once an hour. That happens at about half past the hour.

[DaltonST]

Version 1.5.4 -11 October 2014 New Scrubbing Rules;New Job to Upgrade Q+S DB; Revised Instructions; and bug fixes.



New users (only) must download the additional .jar file (which is really a .zip) in the OP, and follow the instructions.


DaltonST

[DaltonST]

I am always hunting for malformed and/or misplaced metadata to use as templates for new Scrubbing Rules. I am running low, and I could use your help.

If you have any that Q+S is not already fixing, please Private Message me after attaching a screen snip of any book’s GUI row in Calibre that is not being scrubbed properly by this plugin. If many books have the identical problem, just send the most complex sample. If you can create a .txt file via "calibredb.exe list --fields=all ...", that would be better than screen snips, since I could copy&paste it into Calibre into a new fake book just for testing.

All of my test books are fake, no covers, .txt format, with just x's and spaces to read in the .txt book file, and obviously 100% copyright free. Metadata is just data about data, and hence cannot be copyrighted (just ask a John Smith).

If you have “dirty tags” that always need deleting or changing, and the plugin is not already scrubbing them, Private Message me a single snip or a simple text list of those too via "calibredb.exe list --fields=tags ..."


For help on calibredb.exe: calibredb.exe --help > "an_output_file.txt" or
http://manual.calibre-ebook.com/cli/...calibredb-list



Thank you.


DaltonST

[PeterT]

I'm confused by the insistence that support requests go via PMs to you. What ever happened to community involvement and for others to learn from the other experience of fellow users.