MobileRead Forums - View Single Post - Application slowness after 1000+ chapters

KevinH · 12-30-2021, 01:08 PM

I have run some tests.

If you have selected the "Text" folder in BookBrowser before using Add Blank HTML it just appends the file to the end which requires parsing the complete manifest and spine order of the OPF once to create the new opf.

If you instead have any other xhtml file selected in BookBrowser and use Add Blank HTML it will needs to create the file, update manifest and spine of the opf to add the file, and then move it to after the selected xhtml file and that ends up parsing the opf three times more - once to get the spine order up to the selected file, once to get the spone order after the selected file, and once to move and recreate the spine. Note: both the manifest and spine will have over 1000 entries to parse in each case.

All parsing of the OPF is done via EmbeddedPython calls. I could rewrite all of that code in Cpp for more speed but in all honesty the rebuilding of the BookBrowser icons (the order is important as it represents spine order) and all of this takes less than 2 seconds even with over 1000 chapter files and that time grows linearly with size as long as you just want to add the blank file to the end (because you have Selected the Text folder in BookBrowser before adding anything).

So I simply do not see a dire need to rewrite the entire opf parser into Cpp just to cut this time down below 2 seconds. The thought of the bugs that would be introduced by the rewrite of long running and debugged python code is just not worth it.

So I will add rewriting the current debugged new_opfparser.py into Cpp to my LONGTERM to-do list but as I have repeatedly said - it is a very very low priority. Web serials are meant to be read on the web with none of the epub structure and overhead. Imagine trying to repaginate in Word over 1000 chapters (or even load all of the chapters) and you can see that Sigil is doing a reasonable job as is.

12-30-2021, 01:08 PM	#14
KevinH Sigil Developer Posts: 8,804 Karma: 6000000 Join Date: Nov 2009 Device: many	I have run some tests. If you have selected the "Text" folder in BookBrowser before using Add Blank HTML it just appends the file to the end which requires parsing the complete manifest and spine order of the OPF once to create the new opf. If you instead have any other xhtml file selected in BookBrowser and use Add Blank HTML it will needs to create the file, update manifest and spine of the opf to add the file, and then move it to after the selected xhtml file and that ends up parsing the opf three times more - once to get the spine order up to the selected file, once to get the spone order after the selected file, and once to move and recreate the spine. Note: both the manifest and spine will have over 1000 entries to parse in each case. All parsing of the OPF is done via EmbeddedPython calls. I could rewrite all of that code in Cpp for more speed but in all honesty the rebuilding of the BookBrowser icons (the order is important as it represents spine order) and all of this takes less than 2 seconds even with over 1000 chapter files and that time grows linearly with size as long as you just want to add the blank file to the end (because you have Selected the Text folder in BookBrowser before adding anything). So I simply do not see a dire need to rewrite the entire opf parser into Cpp just to cut this time down below 2 seconds. The thought of the bugs that would be introduced by the rewrite of long running and debugged python code is just not worth it. So I will add rewriting the current debugged new_opfparser.py into Cpp to my LONGTERM to-do list but as I have repeatedly said - it is a very very low priority. Web serials are meant to be read on the web with none of the epub structure and overhead. Imagine trying to repaginate in Word over 1000 chapters (or even load all of the chapters) and you can see that Sigil is doing a reasonable job as is.