Library Documentation

[Gary_M_Mugford]

I've asked for this feature from Kovid off and on for years. Crickets. I'm probably the only person interested in it. BUT maybe I've been asking in the wrong location.

As many participants in this sub-forum have, I've got a fair number of libraries, basically split off from the original holding pot when it over-flowed to the point where some upkeep was needed. Unfortunately, WHERE those sub-libs came from have differed over the years, as well with updated sources now now being fully reflected in the sub-libs' settings. Specific to me, the searches. But it even extends to the options (such as the OnKindle being replaced by an OnPW4 awhile back in my WhereIs custom field)

What I need is something in Report form (CSV, Text, HTML, XLS, whatever makes the most sense) of the settings for a/each Lib. That would at least let me note differences and ideally paste them into disparate libraries. The HOME RUN would be a dialog that lets me homogenize across all selected libraries. I realize my settings for comic books are a WHOOOOOLE lot different than the rest of the books, so I wouldn't wish for a nuclear option.

But the bottom line is documenting the settings on a per library basis. As little or as much as can be done. Is this a reasonable ask? What am i overlooking that makes asking for it not generate any sort of enthusiasm?

Thanks for listening to my biennial request. GM

[theducks]

Gary
It is a bit hard to read, but metadata_db_prefs_backup.json contains the per library settings that would be used to restore the metadata.db structure (with settings).

[DaltonST]

Some documentation and homogenization tools already exist. See the attached image. Clearly exactly what you are asking for does not exist in a unified tool.

n.b. JSON files can prettified at many sites for such that purpose, including http://jsonprettify.net/



DaltonST

[Gary_M_Mugford]

Thanks TheDucks and to DaltonST. You've got me moving in a smarter direction. Time to put on my Big Boy Pants and do some JSON breaking down. I initially did a global file find on the machine using Everything and found the JSON files in question, although NOT EVERY library had its own. Not sure why. Guess it comes from being calved off from another library and then not having anything unique done to it. But even with those "same as the first" library, I still had a dozen to work with.

First try was using WinMerge to examine the two oldest, the original big pot and then the original holding pot. (where I would process incoming material, before sending it to the big pot). Well, WinMerge ended up demonstrating JUST HOW DIFFERENT those JSON files were. I took a run through prettify and the result wasn't different to any great degree (but it was worth a shot).

So, I looked at my JSON library in Delphi and I've decided I'm going to take a run at a documentor that satisfies my needs. And if it does that, then the right thing is to provide it as a utility for other Calibre users. Won't be an add-in because I won't be doing it in python, and it will be Windows-specific, but at least it will be available. Did this with a Catalog prettifier years ago and I don't think I got a single download. But maybe years of experience (albeit not with JSON files) will result in something semi-useful.

Thanks both of you for the directional hints. There isn't any basketball just now. Time to program.

[kovidgoyal]

library specific preferences are stored in metadatadb from where you should be able to easily extract them using any sqlite library. If you want to do it using calibre, all you need to do is:

calibre-debug -c "import sys, pprint; from calibre.library import db; db = db(sys.argv[-1]); pprint.pprint(db.prefs)" "/path/to/library/folder"

If you prefer json output replace pprint.pprint(db.prefs) with

import json; print(json.dumps(db.prefs, indent=2, sort_keys=True))

[Gary_M_Mugford]

Originally Posted by kovidgoyal

library specific preferences are stored in metadatadb from where you should be able to easily extract them using any sqlite library. If you want to do it using calibre, all you need to do is:

calibre-debug -c "import sys, pprint; from calibre.library import db; db = db(sys.argv[-1]); pprint.pprint(db.prefs)" "/path/to/library/folder"

If you prefer json output replace pprint.pprint(db.prefs) with

import json; print(json.dumps(db.prefs, indent=2, sort_keys=True))

Kovid,

The batch file I used, using your one-liner spit out a log that looked bereft of carriage returns. And given the arcane listing, I'm not sure that it was in an order of hierarchy that would lead to something useful. And as for your JSON code, it seems with the file mentioned above, I already have a XML-Like code right there.

So, back to my utility I go. I have a four-paned window with two JSON documents being loaded in the top two panes. A pane of buttons to do things and a results pane. Right now, I'm cherry picking the JSON code for just the user stuff. I'm recreating a query string from the User search preferences and that's the MOST valuable to me.

For instance in L1, I have Danny's user search preferences as ((authors:"=J.K. Rowling) AND (tags:"Sports") AND tags:"Baseball")) but in L2, Danny's search preferences are ((tags:"Sports") AND tags:"Baseball")).

With the outputted text file for the combined results I'm getting, I have this bit of output:

User: Danny
Search
L1: ((authors:"=J.K. Rowling) AND (tags:"Sports") AND tags:"Baseball"))
L2: ((tags:"Sports") AND tags:"Baseball"))

Now, with Calibre open, I can now SEE the differences, go to L2 and add Rowling to Danny's search preferences (although I wish I could just paste one search string over the other. Still, the documentor will let me see this.

(I'd actually hoped to affect Calibre's settings by fiddling with the JSON file, but I realize the word BACKUP is there in the title for a reason. Stilllllllll, it would have been great.)

Now, for the time being, because I have lots of memory, I'm running the whole shooting match off in-memory tables and are thus limited to two tables documented (in my useful fractions). But turning the useful but larded JSON code into my version of cleaned up data remains something I have to do outside of your command-line example. THANKS for providing that. Good to see that you can get (and understand) settings when you want.