[GUI Plugin] Consolidate All Library Metadata (CALM)

[DaltonST]

Originally Posted by Beltonmike

Hi again
Thanks for your quick reply but I'm still struggling. Whenever I hit consolidate I am still getting the same message. It is when I hit consolidate not when I am in the library tab. CALM seems to create the library but doesn't put the book metadata in. The address that is created is c\users\....... \CALM\metadata.db There are also metadata_tools, metadata_backup and an empty text file

@Beltonmike:

You need to start over from the beginning. Read and follow the instructions embedded in the Target Library Tab.

You should first "Refresh" the Target Library. Then, specify the Source Libraries, which are your "real" Calibre libraries that have real books in them. Be sure to "save". Then, you must "Generate" your Source Custom Columns. You have never used the word "Generate", so I suspect you have not done so. It is required. Read the ToolTips on that Tab. After a successful Generation, you click the "Execute Consolidation" pushbutton on the Target Library Tab.

Nuclear Option: If the above does not work, delete this file: C:\Users\Beltonmike\AppData\Roaming\calibre\plugin s\Consolidate All Library Metadata.json which is the configuration file that has been saved as a result of everything you have done so far. Then, manually delete the CALM target library (everything, including /CALM/ itself). Then start over from scratch.

If you still have problems, you must attach to your next post "Screen Snips" (using the Windows 10 "Snipping Tool" located at "C:\WINDOWS\system32\SnippingTool.exe") of the first 3 Tabs. Also, attach the entire Job Log from the Consolidation Job after you copy it into a Notepad text file.


DaltonST

[Beltonmike]

Again thank you, I"'ll go for the nuking option as I now have libraries a bit all over the place. Then try again ..... slowly Hopefully the next message will be a thankyou

[Beltonmike]

Sorry Mike again I actually managed to get through to the calibre job box this time but get sqlerror: no such table: Source.languages.


Calm is only finding one of my libraries though calibre can read 2 and I typed the address wrong for the second one which I presume will be the "no such table" error

No more time this evening (france) so will renuke and start over tomorrow when I am CALMer.

I have attached the files but if I am right about the error not a lot to see

Thanks for your trouble and sorry for being dumber than usual

Mike

[DaltonST]

@Mike:

Attaching to New Source Library: C:/Users/MIKE/Desktop/metadata.db
Consolidating Source Standard Metadata: Mon Feb 15 19:06:17 2016
Job: "CALM Consolidation" failed with error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\threaded_jobs.py", line 83, in start_work
File "calibre_plugins.consolidate_all_library_metadata. calm_consolidation", line 94, in main_calm_consolidation
File "calibre_plugins.consolidate_all_library_metadata. calm_consolidation", line 193, in CALM_Control
File "calibre_plugins.consolidate_all_library_metadata. calm_consolidation", line 374, in source_control
File "calibre_plugins.consolidate_all_library_metadata. calm_consolidation", line 392, in source_copy_control
File "calibre_plugins.consolidate_all_library_metadata. calm_consolidation", line 450, in add_new_language_codes
File "c:\cygwin64\home\kovid\sw\build\apsw-3.8.2-r1\src\cursor.c", line 990, in APSWCursor_execute.sqlite3_prepare
File "c:\cygwin64\home\kovid\sw\build\apsw-3.8.2-r1\src\statementcache.c", line 386, in sqlite3_prepare
SQLError: SQLError: no such table: SOURCE.languages

The cause of your problem is very clear. Your Source Library is "C:/Users/MIKE/Desktop/metadata.db" which has no Standard Calibre Table languages. The languages table is the very, very first thing that CALM copies from a Source Library into the Target Library. Your Source Library is not a valid Calibre Library.

By the way, in Windows, the "Desktop" folder is a very special folder. I would not recommend using it as a Calibre library itself.

You need to use Calibre to create a 'real' Calibre Library using its proper menu path for doing so: Right Click the Library Icon> Switch/Create Library > Choose the parent directory which must already exist, and go from there.

Example: "C:\Users\Mike\Documents\eBooks\MyLibraryName" .

Then, switch to that new Library and test it. Add a book to it, and check its functionality.

Click the CALM icon. Refresh the Target Library. Change the Source Libraries to what they should now be, and that does not include C:/Users/MIKE/Desktop . Generate the Source Custom Columns. Then, run the Consolidation.

Note that you can Consolidate a particular Source Library only once for any single incarnation of the CALM Target Library, but that does not mean that you have to Consolidate all of the Source Libraries at once. The checkboxes on the Source Library Tab control which to consolidate at any particular time. The ToolTips say that too. You could try Consolidating only the OneDrive library to see what happens. Based on your attachments, it should work fine. If it does not, apply the Nuclear Option again, and start over from scratch again but using "C:\Users\Mike\Documents\eBooks\MyLibraryName" instead of "C:/Users/MIKE/Desktop/metadata.db".

The reason that only one (1) of your Source Libraries was "Generated" is that you have only one (1) Source Library with any Custom Columns whatsoever.

In summary, the problem is not CALM, but C:/Users/MIKE/Desktop/metadata.db.

Good luck.


DaltonST

Edit: I just noticed that your CALM DB path is C:/Users/MIKE/Desktop/CALM/metadata.db . The erroneous Source Library is C:/Users/MIKE/Desktop/metadata.db . That means that your CALM Target Library is a "child" of your Source Library! That is not ever going to work as far as CALM is concerned. Your CALM Target Library path should be something like: "C:\Users\Mike\Documents\eBooks\CALM\metadata. db"

[Beltonmike]

Thanks again. I've been playing around this morning and by just selecting the libraries that CALM finds it works wonderfully. My problem seems to have come from the test library that I created. Calibre reads it fine but CALM doesn't 'see' it at all and when I type it in I get the CALM library error. I installed Job Spy to try and get a log of what happens but nothing shows up. I recreated a test library ticking the copy current library structure box then importing a book and that went fine, CALM found it and everything worked

If you can tell me how to get a file to show what is going on I will be very happy to send it to you.

Also for information, working on my rather small monitor I didn't see the save button on the source libraries screen (see clip) so that may not have helped the first run through although I would still have got the error message due to the library that CALM wasn't seeing.

Many thanks again for a great program

Mike

[tekeek]

So I came across an issue that causes my Preference - Toolbar to act unusual.

When I mouse over CALM it causes the Available Action to expand

Then even if I expand the Preference - Toolbar it still causes it to expand

[BetterRed]

Originally Posted by tekeek

So I came across an issue that causes my Preference - Toolbar to act unusual.

When I mouse over CALM it causes the Available Action to expand

Then even if I expand the Preference - Toolbar it still causes it to expand

@tekeek - far as I can see, you're seeing what's known as a 'Tooltip'. They've been a standard feature of graphical user interfaces for last 30+ years, hover over almost anything and you'll see something similar. The CALM Tooltip text looks to be longer than most, but that's better than some applications which have none or merely repeat the name.

That it's also shown at bottom of the lists may be a recent innovation - saves the user the bother of hovering

[DaltonST]

Version 2.0.15 -2016-03-04 New: Option to consolidate the 'Multi-Column Search' plug-in 'Word-Book Index' into the CALM Target Library so that MCS Word-Book Queries may be executed against all consolidated Source Libraries simultaneusly.

So, you can do a Word-Book Index Query in MCS while in the CALM Target Library so that you search every book you have consolidated into it simultaneously.

By the way, the MCS 'Full-Text' Query cannot be used in CALM because CALM has no book formats since it is a snapshot of the Source Libraries. However, the Word-Book Index Query does not use any book formats, so it can be used in lieu of a 'Full-Text' Query if a long sentence is broken into individual words (with or without the LIKE wildcards % or _) separated by the bar symbol when the 'bar means AND not OR' checkbox has been selected. It can be orders of magnitude faster too, since it uses an index.



DaltonST

[Terisa de morgan]

I have a question (perhaps it's answered but I haven't found it): If I add books to one of my source libraries and I run the consolidation job again, are those books added to the CALM library? Because it isn't happening to me.

[DaltonST]

Originally Posted by Terisa de morgan

I have a question (perhaps it's answered but I haven't found it): If I add books to one of my source libraries and I run the consolidation job again, are those books added to the CALM library? Because it isn't happening to me.

@Terisa de morga:

Absolutely not. The CALM Target Library is a "snapshot" that is transient. If you try to consolidate a Source Library a second time, your consolidation job log will tell you "no". See the example job log below. I guess you didn't read your latest consolidation job log.

To "add books" to the CALM Target Library, you must Refresh, Generate and Consolidate yielding a new temporary and transient snapshot CALM Target Library.

Starting job: CALM Consolidation Starting CALM Consolidation CALM DB: X:/CALM/metadata.db Sat Mar 26 13:19:50 2016 Python: Windows CPython 2.7.9 SQLite Version: 3.8.4 [APSW] PRAGMA main.locking_mode = EXCLUSIVE; Beginning 'Consolidate All Library Metadata' ══════════════════════════════════════════════════ ══════════════════════════════════════════════════ ═════════════════════════════════════════════ Number of books for this Source Library found in Target Library: 32 ERROR: Source Already Has Books in This Target Library. Ignoring: S:/Calibre/CalibreEnglishNounFrequency/metadata.db Library SQLite memory usage (in bytes) -- Currently: 2,778,544 Job Maximum: 182,188,496

DaltonST

[Terisa de morgan]

No, I don't usually read the log unless there's a problem. I guessed something like this but I prefer to confirm, thank you.

[DaltonST]

Version 2.0.17 -2016-03-26 Attempt to solve OSX subprocess issue.

I still do not have OSX to test the changes, but Windows 10 still works perfectly after a reasonable recommendation found in Stackoverflow was implemented.

Good luck.



DaltonST

[Tanjamuse]

I was wondering, can I read the books from the various libraries directly from the Target Library?

[DaltonST]

Originally Posted by Tanjamuse

I was wondering, can I read the books from the various libraries directly from the Target Library?

@Tanjamuse:

Nope. The Target Library is metadata-only. No books. No formats. No underlying folders. Nothing but three files: metadata.db, metadata_tools.db, and the preferences backup file.

The CALM Target Library is designed to be used to standardize your metadata among all of your Source Libraries, to shuffle books among those libraries if needed, and then to be deleted and later recreated when appropriate.


DaltonST

[DaltonST]

Version 2.0.19 -2016-06-24 Compatibility improvement for use with Linux and OSX; now does not optionally delete any Identifiers that start with a 'z', nor 'doi'.





DaltonST