View Single Post
Old 07-05-2021, 06:51 AM   #2549
anacreon
Guru
anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.anacreon ought to be getting tired of karma fortunes by now.
 
anacreon's Avatar
 
Posts: 978
Karma: 3475832
Join Date: May 2012
Location: France
Device: Elipsa, Sage, Libra 2
Quote:
Originally Posted by davidfor View Post
There have been a few reports recently of errors when sending very large books using this driver. The error occurs in the code that is starting the threads to process each internal file. The released version starts one thread for each internal file. This is causing problems with very large books when using the 32bit version of calibre. I am not sure of the exact size of book that will cause a problem, but, the test book I am using has over 5000 internal files.

The attached version of the plugin solves this problem by only starting a smaller number of threads that do the work. At the moment, it is starting up to 100 threads. That should cover most books, but, also prevent larger books from causing issues. There might be some performance impact, but, my testing is inconclusive. The comparisons have been done while other processes have been running that would have affected the time taken. With the large file mentioned, both the released and beta versions are taking between 5 and half and 6 minutes.

Please test this and report any problems. Most of the testing is with the 64bit current release of calibre. I have done very quick tests with the portable versions of 3.48 and 4.23 and it appears to be working with them.

This problem also affects the output plugin for conversions. I have attached a version of this as well if anyone needs it.
I have tested this with Elipsa / calibre portable 5.22. Since this morning I was in a hurry, and made an error that necessitated a rough exit, I had to restore the last data base saved, 03 July late.
First I sent the three "ordinary" epubs I had sent to Elipsa yesterday 4 July, and it worked. I ejected/reconnected, then sent the metadata and reading positions, checked the DB (ok) and saved it. They all landed as kepubs on the Elipsa. I then disabled the "silent" check box on KoboTouch Extended, to have the eventual error message to send you.

I tried a test copy of one of the 11 books which necessitated a preliminary conversion with Calibre 64bit 5.22.1, "Dictionnaire historique de la langue française", 1114 html files, 20.10 Mo, and it didn't work. Here is the error message:
Spoiler:
calibre, version 5.22.0
ERREUR : Erreur: Erreur pendant la communication avec le périphérique

Failed to process Dictionnaire historique test kobotouchextended by Rey, Alain: can't start new thread

Traceback (most recent call last):
File "calibre_plugins.kobotouch_extended.device.driver" , line 276, in _modify_epub
File "calibre_plugins.kobotouch_extended.common", line 246, in modify_epub
File "calibre_plugins.kobotouch_extended.container" , line 465, in convert
File "calibre_plugins.kobotouch_extended.container" , line 458, in __run_async_over_content
File "calibre_plugins.kobotouch_extended.container" , line 447, in __run_async
File "threading.py", line 852, in start
RuntimeError: can't start new thread

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\gui2\device.py", line 89, in run
File "calibre\gui2\device.py", line 595, in _upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 406, in upload_books
File "calibre\devices\kobo\driver.py", line 2137, in upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 302, in _modify_epub
File "calibre_plugins.kobotouch_extended.device.driver" , line 276, in _modify_epub
File "calibre_plugins.kobotouch_extended.common", line 246, in modify_epub
File "calibre_plugins.kobotouch_extended.container" , line 465, in convert
File "calibre_plugins.kobotouch_extended.container" , line 458, in __run_async_over_content
File "calibre_plugins.kobotouch_extended.container" , line 447, in __run_async
File "threading.py", line 852, in start
RuntimeError: Failed to process Dictionnaire historique test kobotouchextended by Rey, Alain: can't start new thread


I am of course at your disposal for further tests whenever I have the time - not that much presently since I go on holidays Thursday 8, but I'll take my Surface, Elipsa and Libra and can continue from the 11th.
anacreon is offline   Reply With Quote