[Plugin] ePub3-itizer - epub3 output plugin for Sigil

[KevinH]

Hi Alpe,

I pushed a "minimal" set of changes on the "momin" branch: https://github.com/pettarin/ePub3-itizer/tree/momin

The changes are:

1. tests/mo/ : directory containing the epub trees to test MO support;

2. README.md : changed date

3. src/plugin.xml : bumped to v0.3.3

4. src/plugin.py : added code to detect and patch SMIL files

5. src/opf_converter.py : added code to generate the <meta...> elements and media-overlay="..." attributes required when MO are defined.

Other notable changes:

A. I believe this was a bug:

https://github.com/kevinhendricks/eP...e12a271c48L259
[/QUOTE]

Yes, that was a definitely a bug. Thank you!

B. refactored several open(...).write(...) calls into:

https://github.com/kevinhendricks/eP...406fd63772R114

Yes that saves lots of repetition. Thank you!

C. there was a potential bug in the creation of the guide landmark in nav.xhtml:

https://github.com/kevinhendricks/eP...406fd63772R209

I have never seen that issue. Would you please provide a specific example of when you have seen this or is this just theoretically more correct?

So it all looks good. Please generate a pull request and I will merge it tonight when I get home from work.

Thanks!

Kevin

[AlPe]

Originally Posted by KevinH

I have never seen that issue. Would you please provide a specific example of when you have seen this or is this just theoretically more correct?

I discovered this issue because the tests/mo/epub2_base tree triggers it, as

https://github.com/pettarin/ePub3-it...ontent.opf#L88

references toc.xhtml, which is not in the spine. If you just add it to the "guide" <nav> in the nav.xhtml file, without adding toc.xhtml to the spine, you will trigger the RSC-011.

(And yes, toc.xhtml comes from the epub3_base from which epub2_base has been "backported". Nevertheless epub2_base is a valid EPUB2 publication.)

The issue stems from the fact that EPUB2 did not mandate that the <guide> reference item points to an item in the spine (?), while EPUB3 requires:

All EPUB Content Documents linked to from the EPUB Navigation Document must be listed in the spine, as well, regardless of whether the Navigation Document has been included in the spine.

(from http://www.idpf.org/epub/301/spec/ep...sec-spine-elem ). See also this epubcheck issue: https://github.com/IDPF/epubcheck/issues/545 .

Note that the same problem should not occur when "translating" the NCX into the corresponding <nav> in the nav document, since the referenced content documents were required to be in the spine by EPUB2 already. (Hence, adding a similar filter block for the TOC <nav> would be redundant.)

Originally Posted by KevinH

So it all looks good. Please generate a pull request and I will merge it tonight when I get home from work.

Done.

[AlPe]

Hi Kevin, I see you merged the code on GitHub, thank you.

Are you going to add other functions before "releasing" v0.3.3 of this plugin?

I am working on the "prepare for MO" plugin I mentioned earlier, and I plan to be done with it before the end of the year. Once done it, and with v0.3.3 of ePub3-itizer out, Sigil could output EPUB 3 files with Media Overlays.

[KevinH]

Hi,
Yes, Doitsu gave me a list of small changes he would like to see: hiding the landmarks, and pagelists of the nav, copying the guide into the new opf just to help for backwards compatibility, and something else we are debating. Once I get those changes merged in, we can make a release.

My focus has been on fixing bugs in Sigil and trying to prevent confusion with SanityCheck vs Flightcrew and how the terms PrettyPrint and Clean have changed their meaning. I hope to have a new Sigil-0.9.2 release out there before the holidays and then follow that up with a epub3-itizer release during my holiday break.

Is that okay, or do you need an official epub3-itizer out there sooner?

KevinH

[KevinH]

Hi Alpe,

Went ahead and made hopefully all of my additional changes and just pushed them. I have asked Doitsu to test them (he is a much bigger user of this plugin than I am and has better test cases). If all looks good (in other words ... if I haven't broken anything), I will go ahead and get a v033 out there asap in case you need it to move forward with your plugin.

Also, when you get a chance, would you create a pull request to add your new contributions to the main README.md and include with it a link to your website and contact information that you prefer people to see. That way all users will be able to easily find you if they run into difficulties with the SMIL part of the code as well as acknowledging your major contribution.

Thank you!

KevinH

[AlPe]

Thanks for the explanation.

I was asking just to know your time frame, and allocate my FLOSS time accordingly. I have no deadline, I do not want to "push" you: feel free to release v0.3.3 as you are comfortable.

About my contribution: not a big deal at all. Since you are going to touch the repo anyway, I suggest that you can phrase the ack as you prefer, e.g.: "... code for handling Media Overlays metadata by Alberto Pettarin (please contact him for issues related to SMIL files)" or something similar.

[KevinH]

FYI:

Just updated the first post for ePub3-itizer_v033.zip. Please give it a try and let me know of any difficulties.

[AlPe]

Seen, thank you. (Also the README.md.)

I have done some quick tests. The only problem I see is the following: the current code in opf_converter.py outputs an empty

<guide>
</guide>

if the original (EPUB 2) OPF does not have a <guide> element.

This produces the following validation error:

ERROR(RSC-005): test.epub/OEBPS/content.opf(41,9): Error while parsing file 'element "guide" incomplete; missing required element "reference"'.

If I remember correctly, the previous code simply commented the guide out, as it is deprecated by EPUB 3. You can either comment it out again, or add a check that outputs the begin/end guide tags only if len(self.guide) > 0.

[KevinH]

Hi Alpe,
So a guide can be optional but not empty. Strange set of specs!

The guide was put back in because of a request so that epub3 ebooks can actually be read and function properly on many epub2 based e-readers if they do not use anything special epub3-wise.

I will include that fix for the next release.

Kevin

ps. FWIW, This tool requires a validated (flightcrew or epubcheck) epub2 before conversion (as input) so the guide should always at least exist.

[AlPe]

Originally Posted by KevinH

So a guide can be optional but not empty. Strange set of specs!

I stopped trying understanding the rationale behind the EPUB specs choices...

Originally Posted by KevinH

ps. FWIW, This tool requires a validated (flightcrew or epubcheck) epub2 before conversion (as input) so the guide should always at least exist.

I do not understand this comment: the <guide> is optional in EPUB 2:

Within the package there may be one guide element, containing one or more reference elements.

(from http://www.idpf.org/epub/20/spec/OPF...htm#Section2.6 ), so an EPUB 2 file whose OPF file does not have a <guide> is still a valid EPUB 2, right?

[KevinH]

Hi Alpe,

If an epub2 has no guide that is legal. But can an epub2 have an empty guide? If not, then this condition means we are okay (not a bug). As the epub3 should only get the guide element start and end tag if they exist in the the epub2.

So the only way a bug exists is if a validated epub2 can have an empty guide element so that it passes an empty guide through to the epub3 opf.

If the epub2 has no guide element at all, it should not create an empty one in the epub3 opf.

Also checking self.guide will not work as we are passing through the guide in the res list and it is built on the fly. So we would have to build up the guide res piece separately in order to prevent an empty guide set from being passed through.

KevinH

[KevinH]

The spec you quote has one or more reference tags for the guide tag in epub2 but our own FlightCrew will happily accept an epub2 with an empty guide tag. I have no idea if epubcheck will accept an empty guide for epub2.

So it looks like I will have to build up the guide section to pass through separately from res and then add it in only if at least one reference item exists.

[AlPe]

Originally Posted by KevinH

If an epub2 has no guide that is legal. But can an epub2 have an empty guide? If not, then this condition means we are okay (not a bug). As the epub3 should only get the guide element start and end tag if they exist in the the epub2.

So the only way a bug exists is if a validated epub2 can have an empty guide element so that it passes an empty guide through to the epub3 opf.

If the epub2 has no guide element at all, it should not create an empty one in the epub3 opf.

I did this test just now: I opened in Sigil 0.9.1 an EPUB 2 with no <guide> in the OPF, validated with epubcheck (no errors or warnings). Sigil added an empty <guide> (i.e., <guide></guide>) to the OPF, and therefore running ePub3-itizer exported the empty <guide> to the EPUB 3 file.

I repeated the test with Sigil 0.9.2 (built from GitHub src), same result.

If I just open Sigil (hence, I get a blank file), I see the empty <guide> in the OPF.

So, I guess the issue is with Sigil, not with the export plugin.

Originally Posted by KevinH

Also checking self.guide will not work as we are passing through the guide in the res list and it is built on the fly. So we would have to build up the guide res piece separately in order to prevent an empty guide set from being passed through.

You are right on this.

[AlPe]

Detaching from the above:

From the previous quote of the specs, I would say that the <guide> is optional, but it cannot be empty.

EDIT: the schema ( http://www.idpf.org/epub/20/spec/OPF...m#AppendixTOCA ) seems to confirm:

<define name="OPF20.guide-element">
<element name="guide">
<ref name="OPF20.optional-id-attribute"/>
<ref name="OPF20.guide-content"/>
</element>
</define>

<define name="OPF20.guide-content">
<oneOrMore>
<ref name="OPF20.reference-element"/>
</oneOrMore>
</define>

Interestingly, epubcheck complains about the empty <guide></guide> if you validate an EPUB 3, but not if you validate an EPUB 2, I have just tried.

[KevinH]

Hi AlPe,

Figures!
I have a fix for this I will test tonight when I get home from work and commit it to the tree. I will also post a ePub3-itizer_v034.zip with the fix here on this site.

Thanks for catching that bug and checking with epubcheck.

KevinH

Originally Posted by AlPe

Detaching from the above:

From the previous quote of the specs, I would say that the <guide> is optional, but it cannot be empty.

EDIT: the schema ( http://www.idpf.org/epub/20/spec/OPF...m#AppendixTOCA ) seems to confirm:



Interestingly, epubcheck complains about the empty <guide></guide> if you validate an EPUB 3, but not if you validate an EPUB 2, I have just tried.