Oddity in e²->e³ conversion

[DNSB]

I converted an ePub2 to ePub3 using the e²->e³ plugin. When I opened the converted ePub, epubcheck complained about a missing title and an issue with an missing li in the nav document. epubveri, OTOH, complained about a missing </spine> element.

OEBPS/Text/Section0001.xhtml	6	158	Col: 10: ERROR(RSC-005): Error while parsing file: Element "title" must not be empty.
OEBPS/nav.xhtml	23	631	Col: 12: ERROR(RSC-005): Error while parsing file: element "ol" incomplete; missing required element "li"

OEBPS/content.opf	17	998	FATAL RSC-016: OPF is not well-formed XML: unclosed <spine> at line 15; expected </spine> but found </package>

The weird item is that when I look at the content.opf file in Sigil, the last lines are:

  </manifest>
  <spine toc="ncx">
    <itemref idref="Section0001.xhtml"/>
  </spine>
</package>

and the </spine> element is present.

If I simply save the document, open the ePub in 7-Zip and open content.opf in Notepad++ from there.

</manifest>
<spine toc="ncx">
<itemref idref="Section0001.xhtml" />
</package>

OTOH, if I place the cursor in the last line below </package> and hit a backspace, epubveri is now happy and the </spine> element is present in the saved file.

[DiapDealer]

Are you talking about Kevin's Epub3-itizer plugin or something else?

[DNSB]

Originally Posted by DiapDealer

Are you talking about Kevin's Epub3-itizer plugin or something else?

That's the plugin I was using.

[KevinH]

Does your epub2 actually pass epubcheck before trying to convert it? Specifically if the content opf is broken, the plugin will stand no chance of working.

So before running ePub3-itizer, running epubcheck and fixing errors is highly recommended. Please also make sure you are using the very latest version of ePub3-itizer.

[KevinH]

After eyeballing the code, there is one code path that could result in a possible missing spine close tag. So would you please copy and paste just the epub2 original content.opf here so I can try to see how that code path actually gets triggered and find the bug if one exists.

That is assuming that epubcheck is okay with the original epub2 opf. Otherwize the plugin is bound to have issues with a broken opf.

[philja]

I've converted an epub2 to epub3 a few days ago using the epub3itizer plugin. The epub2 was in good shape and passed F7 and epubcheck ok.
The only problems noted by epubcheck after the conversion were expected ones concerning inline width=x% for images, which epub3 wants in the CSS stylesheet.
There was nothing about a missing </spine> tag.

[KevinH]

Yes, not a common bug if a bug at all but I would like to figure out how triggered to prevent any future issue.

Thanks!

[DNSB]

I triggered it with Sigil set to create a blank ePub2 on open while testing epubveri. I then ran the ePub3-itizer plugin. When I opened the generated ePub3 file in Sigil and looked at the content.opf, the </spine> item appeared to be there and epubcheck did not complain though epubveri did complain about the missing </spine> item. If I opened the ePub3 file in 7-Zip and opened the content.opf in Notepad++, the </spine> item was not present. Any change made seems to fix the now you see me, now you don't </spine> item.

I've attached three files. The content_epub2.opf.txt is the original from Sigil's created blank ePub2, content_epub3_converted.opf.txt is from the converted to ePub3 file opened with 7-Zip and the content_epub3.opf.txt is what I see in Sigil when viewing the ePub3 content.opf.

[DiapDealer]

I can repeat your results with the bog-standard default Sigil epub2 (which passes epubcheck with flying colors before converting). After conversion with the Epub3-itizer plugin and reopening the new epub3, epubcheck complains about the missing title and the required missing (required) li element. I can see the empty ol tags in the nav.xhtml.

Those errors seem perfectly normal given the input. I'm just trying to figure out how there could be a missing closing spine tag that Sigil itself seems to ignore (and actually shows as being there), and Epubcheck would ignore as well.

I'm not ready to jump into the epubveri game just yet. So I can't offer any input there.

However: I CAN confirm that the closing spine tag in the opf is indeed missing when I open the converted epub outside of Sigil. The question is why Sigil opens such an epub with no complaints and pretends the closing tag is there?! It seems to be silently fixing it, but not flushing it to disk.

And another question is why the epubveri plugin can see the missing spine closing tag while Sigil and epubcheck cannot.

Just saving the epub in Sigil after opening it is enough for it to silently add the closing spine tag to the opf.

[BeckyEbook]

I found the cause of the missing </spine>.

The existing if end_spine and not "spine" in prefix: block works when guide follows spine, but it is never reached when spine is the last element in the OPF. This can happen with a minimal EPUB2 that has no guide section (no semantics = no cover, no TOC etc.).

PR #9 adds a final if end_spine: check to handle this case and close the spine properly.

[DiapDealer]

That may make sure the plugin doesn't overlook the closing tag, but at this point, I'm more worried why Sigil opens an epub with a malformed opf without complaint.

[BeckyEbook]

File opf_newparser.py and rebuild_opfxml. Here is our </spine>

[KevinH]

Yes, anytime Sigil itself has to parse the opf, it will add back in any missing tags to keep the opf happy so things work in Sigil.

The plugin has the bug. A closing package tag or the end of parsing itself should close the spine first after adding the nav to the spine. As for the missing nav labels I can put in placeholder values.

I will try to fix things when I return in a week or so.

Thanks for the bug report!

PS: Thank you BeckyEbook for your PR fix!

[DiapDealer]

I'm still a bit unclear, though. Forgetting the plugin for now: I can open an epub with a malformed opf (missing closing tag) and Sigil will silently fix it. I can run epubcheck and Sigil will tell me everything is fine. And I can then close that epub again (without saving--with no warning that anything even needs saving) and the opf will still be malformed in that epub with me being none the wiser. That seems wrong to me.

[KevinH]

The internally parsed opf is written only to the opf resource and just like css and xhtml resources, it must be flushed to disk before saving and before any plugin run. That explains why epubcheck never complains. The builtin opf parser used to prep for any plugin, also tries to self-repair.

So epubcheck may be using the plugin interface to create the opf from its parsed form where as epubveri is trying to work around the parsed opf to use the input opf, or visa versa? But that is a guess.

We could add code to flush any opf change to the disk after saving it as a string in the resource itself but that should not be needed and will slow Sigil down since many Sigil operations touch the opf.

Maybe we need to check the "save all resources to disk" routine actually includes flushing the opf and any ncx as well. But that should already work.

Really not sure how the opf is read (ie by recreating it from parsed form; or by sneaking around the wrapper to directly access the opf file) but epubcheck and epubveri plugins could have made different choices here.