calibredb set_custom(etc) GUI refresh?

[JimmXinu]

I'm looking through the source for calibredb and noticed that while some commands (add/remove/add_format/etc) send an IPC message to the running GUI to refresh, other commands (set_custom/add_custom/etc) do not. And I can see the difference in the running GUI.

Is using calibredb while the GUI is running: allowed, not allowed, or only allowed for certain commands?

The only reference on the calibredb manual page is the --dont-notify-gui option which implies it is allowed.

The only other reference I've found says "don't do custom columns while GUI is running", but it is quite old.

[kovidgoyal]

You can use calibredb while the GUI is running, but it is not a good idea and it's likely the ability will be removed in the future in favor of a direct IPC interface to the db the GUI is using. Adding/modifying custom columns requires a GUI restart, which is why calibre db does not notify the GUI on those actions, while adding/deleting/etc. works fine witha simple refresh in the GUI.

[JimmXinu]

So add_custom_column and remove_custom_column are definitely GUI-unsafe--not surprising.

Judging by which commands update the DB but don't call send_message, the commands set_metadata and set_custom are also GUI-unsafe?

(If calibredb as it is now is going to be around for a while, I'd suggest adding notes to the manual page about which options aren't safe while the GUI is running.)

[DaltonST]

Originally Posted by kovidgoyal

You can use calibredb while the GUI is running, but it is not a good idea and it's likely the ability will be removed in the future in favor of a direct IPC interface to the db the GUI is using. Adding/modifying custom columns requires a GUI restart, which is why calibre db does not notify the GUI on those actions, while adding/deleting/etc. works fine witha simple refresh in the GUI.

Kovid,

I know that you put a great premium on backward compatibility of fundamental changes to Calibre, so I would like to state that I have several plug-ins that use calibredb to automatically create new Custom Columns for the user in order for them to continue.

• CALM - Consolidate All Library Metadata
• ZMI - Zotero Metadata Importer
• LC - Library Codes

ZMI creates its standard set of 27 Custom Columns using calibredb, then automatically restarts Calibre.

CALM creates (potentially hundreds of) Custom Columns using calibredb in a CALM Target Library that is never the current Library, although of course the GUI is running for the current Library since CALM is being used. No restart is needed when calibredb updates an "offline" Library even if the GUI is running for a different Library.

LC creates user-selected Custom Columns, then does what Calibre itself does in Preferences > Add your own columns: the user is told that they must manually restart Calibre for the changes to take effect.

Thousands of Calibre Users would be impacted by any non-backward-compatible changes to calibredb.


Thanks for your consideration.


DaltonST

[kovidgoyal]

@JimmXinu: set_custom and set_metadata dont operate ont he db so there is no question of notifying the GUI

@DaltonST: The correct way for a plugin to access/modify the db is to use the python db apis, not calibredb. However, as I said in my original post, the current design of calibredb which opens the database separately will be replaced by one that connects to a running instance of the calibre GUI/calibre server via IPC to perform its actions. So adapting your plugins would be as simple as passing in an extra parameter such as

--use-running-server

to calibredb.

[eschwartz]

Originally Posted by DaltonST

Kovid,

I know that you put a great premium on backward compatibility of fundamental changes to Calibre, so I would like to state that I have several plug-ins that use calibredb to automatically create new Custom Columns for the user in order for them to continue.

And as has been observed multiple times, don't do that.

Why are you using the command-line-shell wrapper around the db interface, rather than using the currently open db interface directly from python?

You are adding a great deal of complexity, and additionally making your plugins unreliable or outright broken for some of your users.
With zero benefit whatsoever.

Although you have invented a novel way to misuse calibre.

[JimmXinu]

Originally Posted by kovidgoyal

@JimmXinu: set_custom and set_metadata dont operate on the db so there is no question of notifying the GUI

??? They don't change the schema, but they certainly change values in the db. And I apparently misread the code--set_metadata does send the GUI the refresh message.

It looks to me more like set_custom should also send the refresh IPC message to the GUI. The change is quite small, I can make a PR if you like.

diff --git a/src/calibre/library/cli.py b/src/calibre/library/cli.py
index 759c775..a996806 100644
--- a/src/calibre/library/cli.py
+++ b/src/calibre/library/cli.py
@@ -1065,6 +1065,8 @@ def command_set_custom(args, dbpath):
         return 1
     do_set_custom(get_db(dbpath, opts), args[0], int(args[1]), args[2],
             opts.append)
+    write_dirtied(db)
+    send_message()
     return 0

Not that the change will make it perfect--the same sort of race conditions that require GUI PIs to use proceed_question will still be possible.

[kovidgoyal]

Sorry, I was thinking of something else (handling too much piled up email yesterday

Yes, they should be notifying the GUI.