[GUI Plugin] CalibreSpy

DaltonST · 08-21-2018, 01:20 PM

[GUI Plugin] CalibreSpy

Summary:

Light GUI for viewing, searching using VLs and many other search tools, and using many libraries simultaneously by multiple users without running the Calibre GUI itself.
CalibreSpy is always 100% read-only for book-related data and files in a Calibre Library.
View Multiple Libraries simultaneously without running Calibre.
View Multiple Libraries simultaneously while Calibre is already running.
Run CalibreSpy simultaneously from a command file (CLI) as many times as you wish. CalibreSpy is read-only.
Run CalibreSpy from the Calibre GUI, viewing any single Library. However, that single instance of CalibreSpy will be closed automatically by Calibre as the latter shuts down. Use a command file to run CalibreSpy completely independently of the Calibre GUI.
Virtual Libraries.
User Categories via Virtual Libraries.
Saved Searches via Virtual Libraries.
Search for Duplicate Books using any 2 Columns, Standard and/or Custom.
Perform multi-Library searches for matching books using "Clipboard Search Queries" while simultaneously viewing multiple Libraries in CalibreSpy. Watch video.
For Tags within an intrinsic (informal) "hierarchy", filter down from the "top" of the hierarchy to its bottom. Filtering-down is a method of drilling-down.
Drag-and-Drop functionality from third-party full-text search applications.
Cross-Book (Inter-Book) Search Queries.
Raw SQL Search Queries.
Metadata Column "Contains" Another Metadata Column Search Queries.
Library Browser with book details.
Email books while Calibre is not running.
Visualize Metadata with the option of exporting the visualized raw data to a .csv file.
FTP books to a server or device. FTP Delete that same book.
'Arbitrary Program and File' execution tool.
Comments may always be viewed with two types of dedicated Comments Viewers.
Copy a single book's CalibreSpy metadata into a spreadsheet or text document as tab-delimited.
Save a copy of a book to a selected folder.
Author Links
Preferences are entirely Library-specific, and all are stored in the Library's metadata.db so copying a Library to another PC will automatically transfer all of the CalibreSpy settings for that Library.
User-defined Library-specific application paths in Customizing for Reading, Viewing, Editing, and Arbitrary Other Program.
On a per-Library basis, most columns are entirely optional for loading: Series; Tags; Publisher; Languages; Added Date; Last Modified Date; Custom Columns; Ratings; and Identifiers.
Pre-filtering of books selected to be loaded is an option.
Mini touchscreen/mouse-less tablets (running Calibre, of course) are easier to use with CalibreSpy than the Calibre GUI.

Multiple Users Simultaneously:

Spoiler:

Documentation:

All of the ToolTips in CalibreSpy comprise its "User Guide".
Refer to all of the attached images which describe and explain its uses.

OS Comments:

The paths in Calibre Preferences > Plugins > CalibreSpy > Customize Plugin as-delivered are likely not correct for Linux and OSX. You must verify them.
The OS command file for executing CalibreSpy via CLI is delivered as a Windows .bat file (only), and resides here: ...\calibre\config\plugins\calibrespy_cli\ and is named CalibreSpy.bat. There is a second .bat file, CalibreSpy_with_options.bat. There is a third .bat file along with a .py script file in a subdirectory. You should create OS-specific command files using those .bat files as templates. Users should create a copy of their personalized command files on their desktop or other convenient location to enable executing CalibreSpy while Calibre is not running, and to avoid having new versions of CalibreSpy overwrite any personalized CalibreSpy.bat file in the directory indicated above. A common personalization for a Windows .bat file is to remove the "pause" command at the end so the temporary CLI window closes when CalibreSpy exits.

Requires Calibre Version: 6.0.0

Version History:

Spoiler:

DaltonST · 08-21-2018, 01:20 PM

Reserved for future use.

BetterRed · 08-21-2018, 05:16 PM

Should the second Book Other Program Path be Book Other Program Icon Path?

Currently I run multiple version 2 calibre servers to achieve a similar result. It works but having and running multiple versions of calibre is something I'd rather not do, and I really don't like browser based applications.

But, I need to have multiple calibre libraries open simultaneously with the Library Management (LM) program (calibre.exe), except the one that it has open. I appreciate there'd be an issue if the LM switched to another library - maybe that could be disabled if CalibreSpy was active.

Is that something you'd consider? And if so, would it be feasible to display additional columns.

And a word of thanks for providing yet another innovative and useful addition to calibre.

BR

BetterRed · 08-21-2018, 06:04 PM

Quote:

Originally Posted by BetterRed

Should the second Book Other Program Path be Book Other Program Icon Path?

Aaah - I see it is in the install, it's just the screen shot that's wrong, should've looked

Glitches

Series index is 1.0 when there's no value in series column.
Series count is 2 (under series selector) when there are no series in the entire library.
The Book Path column header has no right edge vertical line, so it looks like it can't be stretched, but it can if you get the mouse over the invisible line

Suggestions

Make the config settings per library rather than global.
Add a Configure button to the CalibreSpy Book list window.

BR

DaltonST · 08-21-2018, 08:55 PM

CalibreSpy tells you what metadata.db says. Series Index is 1.0 in table Books as a default, so that is why you see 1.0.

All books without a series are assigned a series of "" (Null or None are too ugly) so they can be easily filtered upon and the table sorted quickly. I guess I could change the Series Index to "0.0" for those rows, but the sort sequence would change if a "" were used, as numbers sort before empty strings.

The counts are the counts of the choices in the Filter ComboBoxes, as the label says: "choices: 2". Series Filter Count is 2 because there are 2 choices: Series-Any/All and "", meaning no series.

CalibreSpy does not do any searches, but rather does only filtering.

Regarding global preferences, having them by metadata.db rather than globally would add some to the startup time, which I have optimized to be as fast as it is now. I guess I can make that decision a global preference, so if you do not want to wait for it on tiny tablets that have little juice to begin with, you won't have to. Personally, I would never use that functionality, which is why CalibreSpy does not have it, as I originally conceived it to solve my own issue with using the GUI on my 8" Win10 Tablet. As with all of my plugins excluding Zotero Metadata Importer (which I originally wrote as a favor, since I don't use Zotero (such software didn't exist back when I could have really used it), I originally wrote them for their primary user: me.

As for adding "new" columns, I was waiting for that. Again, startup time will increase depending on quantity of new columns and the datatypes (normalized custom columns with 2 tables that must be joined are much slower than unnormalized ones with only 1 table to load). Also, loading "comments" that are data-length-long translates to using much more bandwidth to load the data, translating into potentially much longer startup times. That would have to be a per-metadata.db choice, which is another reason CalibreSpy 1.0 does not have that functionality (all choices are global in 1.0).

Your comments about disabling Quick Switches in the GUI if CalibreSpy is running are confusing to me. CalibreSpy is meant to be run via a command file so it is entirely independent of the GUI. If the ability to run CS from the GUI is enabled by a user, then that instance of CS would die as soon as the GUI shuts down. Also, multiple instances of CS cannot be run simultaneously from the GUI. Another reason to use the command file.

DaltonST

BetterRed · 08-22-2018, 01:00 AM

Quote:

Originally Posted by DaltonST

CalibreSpy tells you what metadata.db says. Series Index is 1.0 in table Books as a default, so that is why you see 1.0.

All books without a series are assigned a series of "" (Null or None are too ugly) so they can be easily filtered upon and the table sorted quickly. I guess I could change the Series Index to "0.0" for those rows, but the sort sequence would change if a "" were used, as numbers sort before empty strings.

The counts are the counts of the choices in the Filter ComboBoxes, as the label says: "choices: 2". Series Filter Count is 2 because there are 2 choices: Series-Any/All and "", meaning no series.

Fair enough - the 1.0 just looks odd, but it isn't important. How about an option to not display the Series columns? Three of my four primary libraries don't use the Series column - I recognise that is not the norm.

Quote:

Originally Posted by DaltonST

Regarding global preferences, having them by metadata.db rather than globally would add some to the startup time, which I have optimized to be as fast as it is now. I guess I can make that decision a global preference, so if you do not want to wait for it on tiny tablets that have little juice to begin with, you won't have to. Personally, I would never use that functionality, which is why CalibreSpy does not have it, as I originally conceived it to solve my own issue with using the GUI on my 8" Win10 Tablet. As with all of my plugins excluding Zotero Metadata Importer (which I originally wrote as a favor, since I don't use Zotero (such software didn't exist back when I could have really used it), I originally wrote them for their primary user: me.

My reason for wanting per library configuration is that I would use different viewer, editor, and other programs for different libraries - in the calibre GUI I always use View Specific Format and I make quite a lot of use of the built-in Open With tool.

I can work around this issue via the Open Path feature

Quote:

Originally Posted by DaltonST

As for adding "new" columns, I was waiting for that. Again, startup time will increase depending on quantity of new columns and the datatypes (normalized custom columns with 2 tables that must be joined are much slower than unnormalized ones with only 1 table to load). Also, loading "comments" that are data-length-long translates to using much more bandwidth to load the data, translating into potentially much longer startup times. That would have to be a per-metadata.db choice, which is another reason CalibreSpy 1.0 does not have that functionality (all choices are global in 1.0).

I suspected something like that, I'd settle for just the built-in columns, excluding Comments

Quote:

Originally Posted by DaltonST

Your comments about disabling Quick Switches in the GUI if CalibreSpy is running are confusing to me. CalibreSpy is meant to be run via a command file so it is entirely independent of the GUI. If the ability to run CS from the GUI is enabled by a user, then that instance of CS would die as soon as the GUI shuts down. Also, multiple instances of CS cannot be run simultaneously from the GUI. Another reason to use the command file.

I assumed CalibreSpy would not be able to use library A if the main GUI was using library A, because the main GUI could change library A - hence my comment about quick switching.

Maybe I have misunderstood something fundamental, so:

Can one run the GUI simultaneously with multiple command line initiated instances of CalibreSpy?

I understood one couldn't, but I haven't tried.

Any reason why you couldn't use the font set in Preferences->Look and Feel

Added: The Tooltip for the Exit button states - Save current CalibreSpy window settings and then exit. - that does not seem to include the column width settings

BR

DaltonST · 08-22-2018, 05:50 AM

Quote:

Added: The Tooltip for the Exit button states - Save current CalibreSpy window settings and then exit. - that does not seem to include the column width settings

For what I designed it to be used for, the buttons at the bottom to either resize all columns to a fixed width or to resize all columns based on their actual contents width were adequate. I will look at it.

If 100% of the Series column values in the matrix are empty, I will hide that column plus Series Index automatically. Also the Series Filter combobox.

Adding the other Standard Columns excluding comments is reasonable, although Tags will add some to the startup time since that is not a normalized item. I will enhance the Preferences to allow each user to decide on a global basis.

DaltonST

BetterRed · 08-22-2018, 06:28 AM

Quote:

Originally Posted by BetterRed

<snip>

Can one run the GUI simultaneously with multiple command line initiated instances of CalibreSpy?

I understood one couldn't, but I haven't tried.
<snip>

Just tried it, and discovered one can run the GUI and multiple CalibreSpy Windows concurrently - very, very

You might want to make that clear in the first post.

Quote:

Originally Posted by DaltonST

For what I designed it to be used for, the buttons at the bottom to either resize all columns to a fixed width or to resize all columns based on their actual contents width were adequate. I will look at it.

If 100% of the Series column values in the matrix are empty, I will hide that column plus Series Index automatically. Also the Series Filter combobox.

Adding the other Standard Columns excluding comments is reasonable, although Tags will add some to the startup time since that is not a normalized item. I will enhance the Preferences to allow each user to decide on a global basis.

DaltonST

Seems more than reasonable

Speaking for myself: I would rank my outstanding 'issues' as follows - but no hurry or expectations

#1 Additional standard columns
#2 Per library programs paths
#3 column width saving
#4 hide empty series

Thanks again

BR

BetterRed · 08-22-2018, 08:13 AM

If I use the Open Path option, the book folder opens in File Explorer, so far so good.

If I select and double click (or hit return) on any file I'm thrown back to CalibreSpy, rather than the default handler for the file type kicking in. If I use the File Explorer->Open context menu option the file is opened by the default handler.

BR

DaltonST · 08-22-2018, 04:53 PM

Version 1.0.1 - 22 Aug 2018 Added option to load Tags, Publisher, Languages, Date, Date Modified. New Filter by Tags. New Context Menu actions. Miscellaneous other enhancements.

An image is attached to the Original Post showing in yellow what has changed since Version 1.0.0.

The next version will pick up with any issues reported today (my today), such as one about Open Path.

DaltonST

BetterRed · 08-22-2018, 07:04 PM

Looks good - I am out for most of my day, so it'll be a while before I can have a look at new version.

Suggestion, provide a Copy Metadata (Ctrl+C) feature that is conformant with the one in the calibre itself, this would enable copying of metadata from Book One in Library Alpha to Book Two in Library Beta (or Alpha even). Would need to honour the Exclude fields when copy/pasting metadata Tweak.

BR

BetterRed · 08-22-2018, 11:53 PM

Quote:

Originally Posted by BetterRed

Looks good - I am out for most of my day, so it'll be a while before I can have a look at new version.

Good grief, my medicos didn't keep me waiting for hours on end

Three issues:

1. It doesn't seem to be using the font from Preferences->Look and Feel, viz:

Click image for larger version

Name: 1.jpg
Views: 2698
Size: 308.6 KB
ID: 165796

2. There is no selector for Publisher

3. Would be good if the columns could be moved, either by dragging the headers, or via an up/down list Preferences - or both

BR

DaltonST · 08-23-2018, 12:46 PM

Version 1.0.2 - 23 Aug 2018 New Filter by Publisher. New Context Menu actions. Miscellaneous other enhancements and technical changes.

A new image was uploaded to the OP.

Still in work: some things that BR has requested but have not yet been implemented. Next release(s).

DaltonST

BeckyEbook · 08-23-2018, 01:12 PM

If user never changed font in Calibre > Preferences > Look & Feel then in gui.json not exist "font" section.

Spoiler:

This code works (tested in Windows only):

Spoiler:

Additional question on the screenshot.

DaltonST · 08-23-2018, 04:54 PM

Quote:

Originally Posted by BeckyEbook

Additional question on the screenshot.

Answer: CalbreSpy always tries to use Calibre's global Preferred Output Format if such a format exists; otherwise, it uses the first available format it finds. Calibre > Preferences > Behavior.

Code:

from calibre.utils.config import prefs as mainprefs
DEFAULT_BOOK_FORMAT = mainprefs.get('output_format')

Have you set your default format? Or have, but it is not working for you? I will add a debug print statement so you can see what is going on in your personal system.

DaltonST

08-22-2018, 04:53 PM	#10
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	ersion 1.0.1 - 22 Aug 2018 Version 1.0.1 - 22 Aug 2018 Added option to load Tags, Publisher, Languages, Date, Date Modified. New Filter by Tags. New Context Menu actions. Miscellaneous other enhancements. An image is attached to the Original Post showing in yellow what has changed since Version 1.0.0. The next version will pick up with any issues reported today (my today), such as one about Open Path. DaltonST

08-23-2018, 12:46 PM	#13
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	Version 1.0.2 Version 1.0.2 - 23 Aug 2018 New Filter by Publisher. New Context Menu actions. Miscellaneous other enhancements and technical changes. A new image was uploaded to the OP. Still in work: some things that BR has requested but have not yet been implemented. Next release(s). DaltonST

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] FanFicFare	JimmXinu	Plugins	11417	Yesterday 11:24 PM
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	527	08-15-2025 01:36 PM
[GUI Plugin] Save Virtual Libraries To Column (GUI)	chaley	Plugins	14	04-04-2021 05:25 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 12:48 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

08-21-2018, 01:20 PM	#2
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	Reserved for future use.

08-21-2018, 05:16 PM	#3
BetterRed null operator (he/him) Posts: 22,010 Karma: 30277294 Join Date: Mar 2012 Location: Sydney Australia Device: none	Should the second Book Other Program Path be Book Other Program Icon Path? Currently I run multiple version 2 calibre servers to achieve a similar result. It works but having and running multiple versions of calibre is something I'd rather not do, and I really don't like browser based applications. But, I need to have multiple calibre libraries open simultaneously with the Library Management (LM) program (calibre.exe), except the one that it has open. I appreciate there'd be an issue if the LM switched to another library - maybe that could be disabled if CalibreSpy was active. Is that something you'd consider? And if so, would it be feasible to display additional columns. And a word of thanks for providing yet another innovative and useful addition to calibre. BR

08-21-2018, 08:55 PM	#5
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	CalibreSpy tells you what metadata.db says. Series Index is 1.0 in table Books as a default, so that is why you see 1.0. All books without a series are assigned a series of "" (Null or None are too ugly) so they can be easily filtered upon and the table sorted quickly. I guess I could change the Series Index to "0.0" for those rows, but the sort sequence would change if a "" were used, as numbers sort before empty strings. The counts are the counts of the choices in the Filter ComboBoxes, as the label says: "choices: 2". Series Filter Count is 2 because there are 2 choices: Series-Any/All and "", meaning no series. CalibreSpy does not do any searches, but rather does only filtering. Regarding global preferences, having them by metadata.db rather than globally would add some to the startup time, which I have optimized to be as fast as it is now. I guess I can make that decision a global preference, so if you do not want to wait for it on tiny tablets that have little juice to begin with, you won't have to. Personally, I would never use that functionality, which is why CalibreSpy does not have it, as I originally conceived it to solve my own issue with using the GUI on my 8" Win10 Tablet. As with all of my plugins excluding Zotero Metadata Importer (which I originally wrote as a favor, since I don't use Zotero (such software didn't exist back when I could have really used it), I originally wrote them for their primary user: me. As for adding "new" columns, I was waiting for that. Again, startup time will increase depending on quantity of new columns and the datatypes (normalized custom columns with 2 tables that must be joined are much slower than unnormalized ones with only 1 table to load). Also, loading "comments" that are data-length-long translates to using much more bandwidth to load the data, translating into potentially much longer startup times. That would have to be a per-metadata.db choice, which is another reason CalibreSpy 1.0 does not have that functionality (all choices are global in 1.0). Your comments about disabling Quick Switches in the GUI if CalibreSpy is running are confusing to me. CalibreSpy is meant to be run via a command file so it is entirely independent of the GUI. If the ability to run CS from the GUI is enabled by a user, then that instance of CS would die as soon as the GUI shuts down. Also, multiple instances of CS cannot be run simultaneously from the GUI. Another reason to use the command file. DaltonST

08-22-2018, 08:13 AM	#9
BetterRed null operator (he/him) Posts: 22,010 Karma: 30277294 Join Date: Mar 2012 Location: Sydney Australia Device: none	If I use the Open Path option, the book folder opens in File Explorer, so far so good. If I select and double click (or hit return) on any file I'm thrown back to CalibreSpy, rather than the default handler for the file type kicking in. If I use the File Explorer->Open context menu option the file is opened by the default handler. BR

08-22-2018, 07:04 PM	#11
BetterRed null operator (he/him) Posts: 22,010 Karma: 30277294 Join Date: Mar 2012 Location: Sydney Australia Device: none	Looks good - I am out for most of my day, so it'll be a while before I can have a look at new version. Suggestion, provide a Copy Metadata (Ctrl+C) feature that is conformant with the one in the calibre itself, this would enable copying of metadata from Book One in Library Alpha to Book Two in Library Beta (or Alpha even). Would need to honour the Exclude fields when copy/pasting metadata Tweak. BR