[GUI Plugin] Integrity Analysis

[dunhill]

Integrity Analysis is a powerful plugin for Caliber designed to automate database diagnostics and deep file integrity checks, ensuring your library remains healthy, structured, and free of corruption.

Key Features & Technical Overview

Deep Library & Database Scanning:
* Direct interaction with caliber's internal database (`metadata.db`) to query book records, authors, and associated formats.
* Maps relative file paths and inspects storage integrity without disrupting the main application workflow.

Advanced File Integrity Checks:
Physical Existence Verification: Scans the disk to ensure physical files actually exist and handle potential I/O errors gracefully.
EPUB Validation: Automatically checks if EPUB files are valid archives containing essential core components (`mimetype`, `META-INF/container.xml`).
PDF & Generic Format Diagnostics: Inspects headers for massive null-byte padding (`\x00`), which typically indicates disk sector failures, memory corruption, or incomplete transfers.

Smart Caching & Reporting:
Persistent Memory Cache: Automatically stores the results of your latest scan in memory so you can instantly review findings without re-scanning thousands of books.
Detailed Report Viewer: Displays clean, resizable summaries using monospaced format for quick analysis.
Export to TXT: Easily export scan summaries and diagnostic logs directly to a `.txt` file for troubleshooting or record-keeping.

Seamless Interface Integration:
Toolbar button featuring an intuitive dropdown menu for quick access: run analysis, view the last report, export data, or open preferences.

Configuration Options
Scan Depth: Toggle between fast structural checks (metadata-only) or deep physical file validation.
Automated Logging: Set preferences to automatically display error logs or summary reports immediately upon scan completion.
Check Filters: Customize specific integrity checks to suit your library management style.

This is what it looks like if there are no problems:

==================================================
INTEGRITY ANALYSIS REPORT
==================================================
Total books checked: 23
Issues found: 0
------------------------------------------------------------
No security or integrity issues detected. Your library is healthy.
===========================================

And so if you find them:

========================================================
INTEGRITY ANALYSIS REPORT
========================================================
Total books checked: 23
Issues found: 2
------------------------------------------------------------
[ID: 12] Don Quixote of La Mancha -> Issue: Missing format file (EPUB/PDF) in book directory.
[ID: 45] One Hundred Years of Solitude -> Issue: Corrupted or unreadable OPF metadata file.

==================================================

Generating a text file that can be exported to identify the problem that needs fixing.

[dunhill]

Reserved

[sgmoore]

?

It's not April 1st, so what gives?

[dunhill]

Version 1.3.0 Minor bugs fixed. Thanks @Comfy.n for the feedback

[dunhill]

Version 1.4.0 in the configuration options now allows you to check for missing or corrupted metadata.opf files since having these files is a personal decision.

[PeterT]

My big issue here is the NAME of the plugin.

Surely what you are doing is verifying the integrity of a book NOT the security.

Might I suggest a far more meaningful name would be "Integrity Analysis"

[BetterRed]

Originally Posted by dunhill

Version 1.4.0 in the configuration options now allows you to check for missing or corrupted metadata.opf files since having these files is a personal decision.

IMO: that implies calibre has a setting to do away with having metadata.opf files in the book folders… can you tell me where I can find that setting?

I looked in the obvious places, Tweaks, Env Vars, and elsewhere; so, if it exists it's illuding me… perhaps I hallucinated they are mandatory if the Restore Database feature is ever to work.

BR

[dunhill]

Originally Posted by BetterRed

IMO: that implies calibre has a setting to do away with having metadata.opf files in the book folders… can you tell me where I can find that setting?

BR

It doesn't have the option to delete them, but rather the option to create them; here I'll show you where.


When I started developing the plugin, I checked my author/titles folder and found those three files. Then a user asked me what the OPF files were, and I realized that having those files is a personal choice for the user. Therefore, I updated the plugin's configuration window to allow for selection.

[dunhill]

Originally Posted by PeterT

My big issue here is the NAME of the plugin.

Surely what you are doing is verifying the integrity of a book NOT the security.

Might I suggest a far more meaningful name would be "Integrity Analysis"

Security refers to ensuring that if you make a backup, your library will be intact. This plugin originated from a user comment in this thread: https://www.mobileread.com/forums/sh...d.php?t=374875

But obviously, I'm open to suggestions.

[PeterT]

Those opf files are created by the backup library subcommand found in the drop down menu of the ebook library icon

[JSWolf]

There is no option not to create the metadata.opf files for each book directory.

[theducks]

Originally Posted by dunhill

It doesn't have the option to delete them, but rather the option to create them; here I'll show you where.


When I started developing the plugin, I checked my author/titles folder and found those three files. Then a user asked me what the OPF files were, and I realized that having those files is a personal choice for the user. Therefore, I updated the plugin's configuration window to allow for selection.

Nope: That controls EXPORTS to devices that do not have their own drivers.

[dunhill]

Following the suggestions, I've renamed the plugin to Integrity Analysis and replaced the images and names in the tab. I would appreciate it if a moderator could change the thread title.

[BetterRed]

Originally Posted by dunhill

Following the suggestions, I've renamed the plugin to Integrity Analysis and replaced the images and names in the tab. I would appreciate it if a moderator could change the thread title.

Done

BR

[BetterRed]

Originally Posted by dunhill

It doesn't have the option to delete them, but rather the option to create them; here I'll show you where.


When I started developing the plugin, I checked my author/titles folder and found those three files. Then a user asked me what the OPF files were, and I realized that having those files is a personal choice for the user. Therefore, I updated the plugin's configuration window to allow for selection.

Save to Disk does not determine what is written to the library directory or its author and book subdirectories, including the library/author/book/metadata.opf files, the presence of which is mandatory… i.e. you can't not have them.

BR