calibre 5: Beta testers needed

[DrChiper]

Originally Posted by DNSB

I was thinking more of a Rudyard Kipling poem with its lines "once you have paid him the Danegeld/ You never get rid of the Dane". Or was that give them a centimeter and they'll demand a kilometer?

Wasn't it more something like "you gave them one finger and they took your whole body"?

[sebastost]

Originally Posted by kovidgoyal

OK this is endless. No more feature requests. I am really beginning to regret adding annotations to calibre.

We are all excited and thankfull for this tremendous work. So please, please do not regret it. Is one of the feature most requested for the epub readers in windows and is such a joy that is finally coming to Calibre.

[nixsee]

I have to say that I haven't found people to be rude, ungrateful or even overly picky. To the contrary, people have been very thankful and respectful of Kovid's time and patience throughout the process.

Perhaps, then, I'm specifically the sort of person who is meant by these comments - if so, I'm disappointed to hear that as I've simply been a passionate user sharing feedback, right from day 1 of the beta, to make this a better tool - often with the qualifier that it, of course, can be taken or left at his discretion.

Fortunately most of the feedback - for both bugs and features - from the more vocal users has been incorporated, and I think we can all say we're absolutely thrilled with the tool. Now that Calibre has the best annotation feature of any tool that I've tried, there's absolutely nothing that comes close to Calibre for Windows, or probably any platform, and I've gotten rid of the rest of the tools that I had installed.

Apologies Kovid if I've overstepped my bounds at all, and thanks again for your hard work and patience. Looking forward to v5!

[0me]

thank you! calibre is awesome works so well i love it!

[JimmXinu]

Originally Posted by JimmXinu

FYI, I think that my DB update issue may well have been caused by my threading issue.

<ThreadingViolationError>

...

I have new information on this.

I was trying to duplicate it and was finally able to by assigning a couple hot keys to add/remove from Reading List and just hammering them for a while.

I got the same error as before a couple times, but a more couple times now, instead of a ThreadingViolationError in plugin code, this spit out in the debug output:

Traceback (most recent call last):
  File "C:\Users\retief\Desktop\nook\calibre\src\calibre\db\backup.py", line 63, in do_one
  File "C:\Users\retief\Desktop\nook\calibre\src\calibre\db\cache.py", line 63, in call_func_with_lock
  File "C:\Users\retief\Desktop\nook\calibre\src\calibre\db\cache.py", line 1102, in check_dirtied_annotations
  File "C:\Users\retief\Desktop\nook\calibre\src\calibre\db\backend.py", line 1940, in dirty_books_with_dirtied_annotations
apsw.ThreadingViolationError: You are trying to use the same object concurrently in two threads or re-entrantly within the same thread which is not allowed.

Is there a background thread somewhere that's auto-updating annotations? I did not have server running at the time.

ADDED: I'm running v4.99.16 win10 64bit binaries from source as of 9ec29bb5485b56c7024edd9648ec07fadb7ff28c

ADDED: backup.py is specifically for class MetadataBackup(Thread), so yeah, it looks like there's a BG thread running that can conflict with other DB access.

What happens with backend.py:124 vs backend.py:1940 both `with conn` when conn() is a property function? I don't know Python with well enough to know if that's a problem.

[kovidgoyal]

See the line

 File "C:\Users\retief\Desktop\nook\calibre\src\calibre\db\cache.py", line 63, in call_func_with_lock

in the traceback, it indicates the the dirty_books_with_dirtied_annotations() function is being called with the DB lock held. So it cannot conflict with anything else, unless that something else is calling some DB code without holding a lock. Is there some code in ReadingList that does this?

[kovidgoyal]

And looking at the code, there is indeed such. The various calls to db.prefs.set are not protected by the lock. Doing them like this should eliminate the issue:

with db.new_api.write_lock:
    db.prefs.set

[JimmXinu]

ReadingList calls (a few levels down) self.gui.current_db.prefs.set_namespaced(...).

As do all my plugins -- I thought that was the correct way to save config settings.

Are you saying that all plugins that use set_namespaced() need to change?

[kovidgoyal]

And this should fix it:

https://github.com/kovidgoyal/calibr...91bea18296764b

[kovidgoyal]

Oops, forgot to protect set_namespaced()

https://github.com/kovidgoyal/calibr...7707712dab34c4

[JimmXinu]

Originally Posted by kovidgoyal

Oops, forgot to protect set_namespaced()

https://github.com/kovidgoyal/calibr...7707712dab34c4

Thanks!

I see another error when quitting:

Traceback (most recent call last):
  File "C:\Users\user\Desktop\nook\calibre\src\calibre\gui2\ui.py", line 937, in quit
  File "C:\Users\user\Desktop\nook\calibre\src\calibre\gui2\ui.py", line 1002, in shutdown
AttributeError: 'ThreadSafePrefs' object has no attribute 'write_serialized'

[kovidgoyal]

Should be fixed now.

[JimmXinu]

Confirmed.

I pounded the keys again too and didn't see any more issues. Thanks for addressing this so quickly.

[kovidgoyal]

Released 4.99.17 with the following changes:

1) Fix for JimmXinu's plugin issue, the fix was somewhat invasive, so
please install and run this beta for a few days.

2) Add a context menu for the viewer highlights panel

3) The way calibre's PyQt modules are built has changed to use
sip-build. This should have no observable effects, but who knows.

4) Show a dot next to highlights with notes in the viewer highlights
panel as well

5) Fix removal of multi-line highlights leaving handles in incorrect location

6) Fix a bug in the raw_inlined() template language function

I am aiming to make the final release in a few days, provided this beta
has no more serious bugs.

[capink]

When I click the customize plugin button in the find duplicates plugin, I get the following error:

calibre, version 4.99.17
ERROR: Unhandled exception: <b>AttributeError</b>:'ThreadSafePrefs' object has no attribute 'to_raw'

calibre 4.99.17  embedded-python: True is64bit: True
Linux-4.15.0-33-generic-x86_64-with-glibc2.2.5 Linux ('64bit', 'ELF')
('Linux', '4.15.0-33-generic', '#36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018')
Python 3.8.5
Interface language: None
Successfully initialized third party plugins: Count Pages (1, 10, 0) && && EpubSplit (2, 9, 0) && Extract ISBN (1, 5, 0) && Favourites Menu (1, 0, 5) && Find Duplicates (1, 7, 6) && Import List (1, 5, 0) && Manage Series (1, 2, 9) && Modify ePub (1, 4, 2) && Open With (1, 5, 12) && Quality Check (1, 10, 1) && View Manager (1, 5, 7)
Traceback (most recent call last):
  File "calibre_plugins.find_duplicates.common_utils", line 721, in _current_row_changed
AttributeError: 'ThreadSafePrefs' object has no attribute 'to_raw'