[GUI Plugin] Kobo Utilities

[DrChiper]

Just a thought: I'm using just 7 basic additional plug-ins, you seem to use a lot more, can they influence each other?

[DNSB]

I've seldom seen any interactions between plugins. Might be interesting if you disabled the other plugins and just tried Kobo Utilities to see what happens.

Last time I ran into an interaction between plugins was fun with. I disabled all installed plugins and then re-enabled one at a time until the issue happened. I think I sent about a megabyte of debug logs to davidfor while doing this.

[DrChiper]

Well, "user interface action" plug-ins cannot be disabled, it seems. So when the only other option is to remove them, I refrain from trying that.

[Majutsushi]

The error I fixed was the KeyError that DrChiper reported seeing in 2.19.1. The fix was a simple one-line change to a button in that dialog, so it shouldn't have any effect on anything else in Calibre. None of the other changes in that version touch anything else in Calibre itself, certainly not its database, so I have no idea about the library oddities that DNSB seems to be seeing.

@DrChiper, you said that the new version "prevents Calibre from starting". Are you getting any error message? The only way I could imagine that happening is if the "Kobo Utilities.json" config file in the plugins directory has become corrupted somehow. Could you maybe try removing that to see if that fixes it? And if yes, could you post it here so I can have a look at it? There seem to be a few more bugs in that annotation feature as I also got an error when trying to change the selection in the dialog.

[DNSB]

I tried spinning up a new VM and installing calibre 8.0.1 and KU 2.20.0 (no other plugins) No issues noted with adding and then removing an annotation and the In library column does not show the UUID or AUTHOR entries after removing an annotation. A small library of 40 books was used.

[DNSB]

Checked and vacuumed the KoboReader.sqlite database and restarted the Kobo (used SQLite Expert Personal for the cleanup). Used calibre to check and vacuum it's databases. Reconnected my Kobo and the UUID/AUTHOR have disappeared from the In Library column.

[Majutsushi]

Thanks for the investigation, DNSB. It looks like the "remove annotations" dialog got broken by the migration from Qt5 to Qt6, since Qt6 removed a deprecated API that that dialog used. My fix was the first step in making it work again but it's not complete. I'll take a look at that, although since that switch happened a while ago and no-one has reported an issue before I wonder how many people actually use that feature.

I still can't see how that change could have broken anything else though, so I would need more information from @DrChiper about what he's seeing.

[Majutsushi]

Originally Posted by DNSB

Checked and vacuumed the KoboReader.sqlite database and restarted the Kobo (used SQLite Expert Personal for the cleanup). Used calibre to check and vacuum it's databases. Reconnected my Kobo and the UUID/AUTHOR have disappeared from the In Library column.

I don't quite understand what you mean by this. In fact I'm not sure what the UUID in the "In Library" column even means, I never paid much attention to it. I certainly can't see how this could be caused by the plugin, much less my recent changes.

I actually just tried double-clicking that column for a book and got this exception:

calibre, version 8.0.1
ERROR: Error: Error communicating with device

stat: path should be string, bytes, os.PathLike or integer, not BufferedReader

Traceback (most recent call last):
  File "calibre/gui2/device.py", line 111, in run
  File "calibre/gui2/device.py", line 713, in _view_book
  File "calibre/devices/kobo/driver.py", line 762, in get_file
  File "shutil.py", line 235, in copyfile
  File "shutil.py", line 212, in _samefile
  File "genericpath.py", line 100, in samefile
TypeError: stat: path should be string, bytes, os.PathLike or integer, not BufferedReader

So this looks like a bug in Calibre itself to me.

[DNSB]

Originally Posted by Majutsushi

I don't quite understand what you mean by this. In fact I'm not sure what the UUID in the "In Library" column even means, I never paid much attention to it. I certainly can't see how this could be caused by the plugin, much less my recent changes.

I've attached an image of what I saw. Either UUID or AUTHOR showed in the In library column when in device view where normally all I see is a checkmark. After I couldn't duplicate it on a new calibre install in a VM, I tried a few things and the check/vacuum of the Kobo device and calibre databases seemed to have cleared up the issue. Though that might be a dancing widdershins around an oak at midnight with a full moon coincidence.

Edit: I've attached an image of my "normal" view. When I double clicked on the column, it opened the book in calibre's ebook viewer.

[chaley]

When running in debug mode calibre puts how the book on the device was matched to a book in the library. UUID is the most precise. If there is no UUID match IIRC calibre tries various forms of title and author matching. The matching algorithms are in gui2.device.py.

[Majutsushi]

I assume that the information in the "In Library" column tells you how Calibre matched the books on the device with the books in the library, which is probably different between the initial copy and subsequent connections. I think I've only ever seen UUID there, at least if the book is already on the device. As far as I understand it the plugin isn't involved in this at all, so any weirdness that you're seeing should be due to Calibre itself. Odd that double-clicking on it works for you though while I'm getting an exception.

Edit: Of course @chaley posts the explanation just a minute before me

[DNSB]

Originally Posted by chaley

When running in debug mode calibre puts how the book on the device was matched to a book in the library. UUID is the most precise. If there is no UUID match IIRC calibre tries various forms of title and author matching. The matching algorithms are in gui2.device.py.

Mea culpa! My screwup! I had changed my shortcut on my taskbar for launching calibre to "calibre-debug -g" and forgotten that I had done so. I then launched calibre by clicking on the calibre shortcut on the desktop which launched in normal mode. When I read your message...

[Majutsushi]

I think I fully fixed the "Remove Annotations" feature now. I'm still not sure how the issues could have caused any corruption, so it would be great if @DrChiper could give this a try to see if you're still having problems.

[DrChiper]

I installed the plugin manually, and tested it. No GUI problems anymore, and I could select all 5 radio-buttons for the given selection options.

One remark though:
when the last annotation file is removed of a sub-folder, that sub-folder still remains and hence, could be removed as well. The same applies when all sub-folders are removed within the main folder ANNOTATIONS, that latter folder could be removed also (as is done when selecting the "All" option)

Great work

[DrChiper]

Originally Posted by Majutsushi

[...]although since that switch happened a while ago and no-one has reported an issue before I wonder how many people actually use that feature.

The simple answer is: nobody as it was broken for a long time. I remember mentioning this to DavidFor some years ago that I noticed that the Kobo sw did not cleaned the annotations after removing the books. He mentioned that "when he had time, perhaps something could be done about it", which sadly never materialized. Until now.