Sigil 2.8.6 crashes on Windows11 when adding a cover to a large EPUB3

[rinsay]

Sigil 2.8.6 crashes on Windows when I use Add Cover on a large EPUB3 with
no existing cover. I can reproduce it by opening the EPUB, choosing Add
Cover, and selecting an image. The crash occurs even when I skip Mend
and Prettify.

I'm using Sigil 2.8.6 on Windows x64, build 26100. The book was exported
by txt2epub, an open-source browser-based converter.
Source: https://github.com/RinSayX/txt2epub
Live instance used for export: https://rinsay.com

I tested the same cover image in each case:

- Blank EPUB3 created in Sigil: works.
- Small demo novel exported by txt2epub: works.
- First 2 chapters of the affected novel: works.
- First 1,200 chapters of the same novel: crashes.
- Same 1,200-chapter EPUB with the TOC in nav.xhtml reduced to one entry,
while keeping the chapter files and reading order: still crashes.
- Full novel with 2,226 reading-order entries: crashes.

The original full EPUB passed EPUBCheck 5.4.0 with zero errors and zero
warnings. I was able to add the same cover and save the book in Sigil
2.8.1 on macOS.

Windows Event Viewer and WinDbg showed these failures on separate runs:

- Event Viewer: ntdll.dll, exception 0xc0000374 (heap corruption).
- WinDbg capture 1: 0xc0000409, FAST_FAIL_STACK_COOKIE_CHECK_FAILURE.
The stack includes sigilgumbo!gumbo_string_buffer_append_codepoint
calling ucrtbase!realloc_base.
- WinDbg capture 2, after reducing the TOC to one entry: 0xc0000005,
attempting to read address 0x8 in python314!PyEval_EvalFrameDefault.

I have the full WinDbg stack traces. Has anyone seen similar behavior
when adding a cover to a book with many XHTML files? Is there a debug
build or additional logging that would help narrow down the cause?

Thanks!

[KevinH]

You debug info shows the bug has heap corruption when sigil gumbo tries to reallocate memory on the stack. My guess is the size of the epub is using up the available stack on one thread but I am not sure why that would cause heap corruption instead of code or stack corruption.

My guess is your Windows box does not have enough memory ... or maybe a recursion (which is capped in Sigil) during parsing is happening that is resulting in a stack limit being reached. On macOs or unix you can use ulimit to set appropriate runtime stack sizes. I am not sure how to control the default C++ stack size on Windows. You could also be reaching a limit on the number of open file descriptors. These is nothing specific to the Sigil codebase that should be limiting you.


Please zip up your epub, then post it someplace, and PM a web link to KevinH (me) here on Mobileread. I will use it to try to recreate what you are seeing on Linux and MacOS.

Having over 2000 chapters/files in a single epub does seem like a bit extreme. Is there some reason for so many files?

[DiapDealer]

I'd love to confirm that the same thing happens using the same epub with Sigil 2.8.1 and earlier on Windows.

[DiapDealer]

Would one of our embedded Python routines even be involved in adding a cover image?

- WinDbg capture 2, after reducing the TOC to one entry: 0xc0000005,
attempting to read address 0x8 in python314!PyEval_EvalFrameDefault.

Normally I would suspect a flawed C extension with that kind of Python error. But regardless, that error indicates segfault/crash of the Python interpreter itself.

[KevinH]

Yes python is used by the lxml python pure xml parser. So adding and removing metadata and manifest properties for the cover would need to use lxml to parse the OPF for a sanity check both before reading and after writing for sanity.

My guess is this is not a general Sigil 2.8.6 issue but a Windows machine issue with limited stack space, limited memory, or limited file descriptors. For speed, gumbo can be invoked by QtConcurrent with multiple chapters being parsed at the same time. Ths can eat up open file descriptors like crazy.

[KevinH]

I have one testcase with like 4000 images so I will try with that. I do not think I have any epubs with over 1000 chapters to test with though. Hope the OP provides his epub as a test case.

[KevinH]

Okay I found one test epub with over 15000 images and one test epub with over 4000 images and adding a cover works just fine (slow at times but just fine) on MacOS.

I do not have even one test epub with over 1000 chapters.

[KevinH]

Okay, I found one BeckyEbook once posted. It has 2500 files in it. It is epub2, but I was able to convert it to epub3. Then it does not have any images to make a cover with. So I added one and I have attached that test epub here:

[KevinH]

Now, when I try to add a cover to this epub3, it happens quite quickly but I noticed that the opf spine was in a random order!

So something is funny, even on MacOS.

Not sure what is going on.