[GUI Plugin] Consolidate All Library Metadata (CALM)

[petzi]

Today I tried your plugin and experienced similar problems as the user saluli – a OSX user like me – in this thread. Maybe there is some problem with OSX and your plugin?

Now the details: I'm using OSX 10.5.5 on a MacBook Air with Calibre 2.63.0. I have applied the plugin with real libraries (not test libraries).

1. I call CALM and start with the tab "Target Library. As you can see from CALM-screenshot #1 everything is clean. I removed the /CALM folder and also the preference file from previous trials. See screenshot # 1 attached.
2. I choose a parent directory for the Target Library and get the following message (Screenshot #2). At the same time I see in the operating system that the folder is created.
3. I confirm with "OK" and can see that the path to folder is correct. (Screenshot #3).
4. I change to the tab "Source Libraries" and click on "Load all Libraries". (Screenshot #4) But this is already wrong! Instead of six pathnames I get only three and three lines with numbers (00,01,02).
5. I click on the tab"Check All" and all lines are checked. (Screenshot #5)
6. After clicking the tab "Save" I get the message that currently three source libraries are checked. And this three numbers are still visible. (Screenshot #6)
7. I change to tab "Source Custom Columns" and click the first button "List Current Source Libraries/Custom Column Activation Statuses" and get a message which seems correct. (Screenshot #7)
8. I click on the tab "Get All Source Library/Custom Columns Combinations" and get the message "There are no specified Source Libraries. Nothing to do." This seems again wrong to me. (Screenshot #8)
9. I do not know what to try next and abort my trial.

I have restarted from scratch, deleted everything – including the preference file – and re-installed the plugin. Then I have restarted calibre in debug-mode and done all the above mentioned steps again. You will find the log-file as attachment.

I hope you can figure out what's the problem as I think you plugin could be very useful for me.

[DaltonST]

@petzi:

1. Exit from CALM.
   
2. CALM gets the library list from the history that Calibre keeps. So, if there are "bad" libraries, in Calibre go to: Library icon > Remove Library > and remove any spurious library names such as 00, 01, 02, etc.
   
3. Run CALM
   
4. Go to the Source Libraries Tab
   
5. Check the good libraries, and uncheck the bad libraries.
   
6. Erase all "unchecked".
   
7. Then save.
   
8. Then manually add whatever libraries you are missing, if any. You can key in whatever you want. The automated population from Calibre's history is just a convenience. You can delete whatever you do not want simply by unchecking it and clicking "erase all unchecked". Or, manually, blank them out. Save.

Until your list of Source Libraries is good, everything else will be wrong. Do not leave the Source Library Tab until it is correct. The first step is the Source Library Tab. That it why it is the first Tab.

The Original Posts mentions OSX as regards calibredb. Be sure to read that and to install the Calibre command line programs per those instructions. CALM will fail quickly if calibredb has not been installed.


DaltonST

[DaltonST]

@petzi:

Attached are pictures of what your Source Library Tab would look like before generation and then afterwards.

Before generation, you can make any changes you wish. Keep only the few libraries that you are interested in consolidating, and erase the rest.

After generation, the library names are read-only, although you can check and uncheck library names since that controls which libraries get consolidated by the next consolidation execution.



DaltonST

[petzi]

I never had libraries with names 00,01,02 but I do have views from the View Manager plugin called 00-xxx, 01-yyy, 02-zzz. I mention this as I've got messages form CALM referring to View Manager as the current Target Library.

I deleted the View Manager plugin but could not find its preference file (I have to re-index my hard disk, but this will take some time).

Whenever I start from scratch (without View Manager this time) I will get the same situation as I have already described. I should add that those two libraries (from my six libraries) which I am interested to consolidate metadata do not appear in the automatic load up with the CALM tab. I added theses paths manually and deleted all paths form libraries I am not interested and clicked save. Again the result was screenshot #6 from my previous message.

I removed all libraries from calibre created a new directory without spaces in its name ("Meine_Calibre-Books" instead of "Meine Calibre-Books) moved the calibre libraries into this new directory and attached them to calibre again. It didn't help.

Whenever I click on "Extract a Fresh Copy of the Target Library" CALM is referring to "View Manager" as the supposed library and I get the attached sequence of CALM messages.

Any idea what I can try out next?

[DaltonST]

Delete the CALM .json file from the Calibre configuration directory for plugins where the .zip files are. Preferences > Behavior has a button to take you there. Also delete the entire target library folder.

CALM has nothing to do with VM. Really odd...

[petzi]

Always when I started a new trial I deleted everything: CALM folder and preference file, deleted and re-installed CALM plugin, sometimes I deleted all cache files on my machine as well. And I deleted the VM plugin too.). But all these precautions didn't help or change anything.

Do you know of any Mac user using your plugin successfully? I think the problem could be maybe related to differences in the OS.

BTW: The preference file for CALM on OSX is at:
/Users/xxxxxx/Library/Preferences/calibre/plugins/Consolidate\ All\ Library\ Metadata.json

At the moment I have no idea what to try next.

[petzi]

I tried a complete new install on the machine of my wife. (Also OSX and MacBook Air) Neither calibre nor View Manager were installed. It was a complete virgin calibre installation where I created some new libraries.

The first three libraries get the names 00,01,02 and starting with the fourth library clicking at the button "Load All Libraries" their names appear in the window. This is exactly the same behaviour as I already have reported!!! Even when I click on "Extract a Fresh Copy…" I got the same error messages referring to View Manager – even this plugin was never ever installed.

I have no clue what to try next…

[DaltonST]

I will take it from here. Very, very, strange...

[DaltonST]

Actually, I think the 00,01 etc. is because you have so few Calibre libraries. I have ~40, so there is no way I could possibly have tested that. Try creating a few new Calibre libraries so the total is at least 5, and see if it disappears.

I will fix CALM assuming that is what it is.

[DaltonST]

Version 2.0.20 -2016-07-28 Fix Source Libraries Load-All-Known-Library logic when a user has no Calibre libraries except the current one, and excluding any CALM Target Libraries.


DaltonST

[petzi]

Nope, I tried it with the new version 2.0.20. Actually I think it is the other way: The first three libraries get 00, 01, 02. More libraries than the three stabilises the behaviour. When I have more libraries and choose 3 or more of those that were generated later (e.g. those that already appear with their pathnames) then the result is 00,01,02. When I choose only 2 of these valid pathnames than the result is 01,02. When I choose only 1 (I know this does not make sense, but just for test purposes) then the result is 02.

[DaltonST]

Did you first delete your corrupt CALM .json file in the Calibre plugin configuration directory? That is a critical question.

[petzi]

Thanks for reminding me. Checking all the time different things one could forget the essentials. But yes, I did remove the .json file.

I came up with another idea: Let's assume it is a OSX-related issue. What could be happened that it didn't work? (I am saying this as it came to my mind, that on the machine of my wife, I did not run the necessary sym link to install the command line tool and then I noticed I could not run the sym link as newer OSX are protected by SIP (System Integrity Protection). I will try to disable it and see what happens.

[DaltonST]

No, it is not an OSX issue.

I assume that you deleted the .json file while Calibre was not running?

[petzi]

SIP does not apply for me. It starts with OSX El Capitan (10.11) and I have Yosemite (10.10) installed.

And yes: I deleted the .json file while Calibre was not running. I can see if the file is there, because I have opened up a window where I can see all the files of calibre preferences in real time. You are right: deleting the .json file while Calibre is running does not work. I noticed that after I did some trials.

My suggestion that it is maybe an OSX issue comes from the fact that I had the same problem with the machine of my wife. I thought on paths issues or something like that. So, if you thing it is not an OSX issue, what then? What could I try next?