[Plugin] ePub3-itizer - epub3 output plugin for Sigil

[AlanHK]

Originally Posted by KevinH

It will not be changed, especially when doing it inside Sigil would be much faster than doing it via python in a plugin. No reason at all to duplicate that code.

All right, you don't think it's worth your time, but come on: "doing it inside Sigil would be much faster than doing it via python in a plugin".
If the plugin does it it will take microseconds. About 1 million times faster than me doing it manually. If I remember to do it.

It's not pure laziness. Years ago I used to hack epubs and create my own NAV file to get a pseudo epub3 that would work with iBooks. So I know how to do it manually.

But I still get tripped up. A few days ago I converted an epub2 to epub3, loaded it into Apple Books. Worked OK.
Went on to make some tweaks. Did an epubcheck and so changed the filenames, which I had not noticed earlier. Deleted the first version on the iPad and loaded the new one. Books crashes out. Tried over and over and eventually found that it is a "feature" that it caches bookmarks and other metadata, even if you delete the book; so when it could not find any of the original filenames, it blew up. A stupid way to handle the anomaly but it's the target platform. Apple don't care if your homemade epubs screw up. The solution was to change the title metadata so it appeared to be a different book.

I know you have no sympathy for Apple users, but many of us are compelled to use them and as I don't make epub3s that often, remembering to fix this by hand can slip my mind.

[BeckyEbook]

@KevinH: So maybe to avoid the problem after conversion, give a warning about it? It's not about laziness either, but a reminder that there is still "something" to be done before starting the plugin so that everything goes according to plan.

Something like that:

Spoiler:

    file_list = list(bk.text_iter())
    for (html_id, href) in file_list:
        filename = os.path.basename(href)
        if bk.launcher_version() >= 20190927:
            filename = bk.id_to_bookpath(html_id)

        if not '.xhtml' in filename:
            print("Error: ePub3-itizer requires .xhtml extension for html files")
            print("Change the extension for all html files to .xhtml and run the plugin again")
            return -1

[DiapDealer]

@AlanHK: Patches welcome. If someone wanted to port the Sigil C++ code that updates every link in every file, manifest, index, toc, guide-reference, footnote, and css url in every epub to Python to be used in this one Sigil plugin, I'm sure adding it to this one would be considered. But for most, porting something that complicated and invasive is not something that lends itself well for use in individual plugins. Especially when it would have to be kept in sync with any future C++ bug-fixes/behavior-changes for the rest of time.

But if someone were to volunteer their time to mirror and maintain Sigil's inherent file renaming, and link-parsing/updating routines in a safe, reusable Python module that could be added to Sigil's plugin framework, I'm sure many plugin devs would be appreciative of the contribution. It would have to be limited to input/output plugins, though, since there is no file renaming ability for Sigil edit plugins.

Hell of a lot of work to undertake for someone, though. Just to take care of something someone could easily remedy by RTFM and then jotting down their own pre-conversion checklist.

[KevinH]

And doing it in python would be much slower unless you design a set of concurrent worker threads to walk each html file, the ncx, nav, and opf just like it exists in C++ code used now and the support code around it. It is just not worth doing.

Just run epubcheck after conversion to catch any issues. And as DiapDealer says ... RTFM.

[KevinH]

They really should be running epubcheck after conversion as well and epubcheck will issue the same warning.

Originally Posted by BeckyEbook

@KevinH: So maybe to avoid the problem after conversion, give a warning about it? It's not about laziness either, but a reminder that there is still "something" to be done before starting the plugin so that everything goes according to plan.

Something like that:

Spoiler:

    file_list = list(bk.text_iter())
    for (html_id, href) in file_list:
        filename = os.path.basename(href)
        if bk.launcher_version() >= 20190927:
            filename = bk.id_to_bookpath(html_id)

        if not '.xhtml' in filename:
            print("Error: ePub3-itizer requires .xhtml extension for html files")
            print("Change the extension for all html files to .xhtml and run the plugin again")
            return -1

[odamizu]

Originally Posted by KevinH

They really should be running epubcheck after conversion as well and epubcheck will issue the same warning.

I agree. I regularly forget to change html to xhtml before epub3-itizing. But I always run epubcheck after I'm done editing my new epub3. It flags my error. I fix the filenames in Sigil, run epubcheck again, and all is good. I don't need the plugin to issue a warning.

[KevinH]

Or exit since the rename to .xhtml could technically come before or after ePub3-itizer is run.

[odamizu]

Well, usually what happens is I run ePub3-itizer, edit book, run epubcheck, then forgot to rename to .xhtml!

In that case, it's easier to rename to .xhtml than to start all over since I've already done some editing.

Though the more this happens, the more I remember to rename to .xhtml without being prompted

Originally Posted by KevinH

Or exit since the rename to .xhtml could technically come before or after ePub3-itizer is run.

[AlanHK]

Originally Posted by DiapDealer

if someone were to volunteer their time to mirror and maintain Sigil's inherent file renaming, and link-parsing/updating routines

I had assumed that a plugin could call the file rename function in Sigil, making it a trivial operation, rather than having to reimplement it from scratch.
But from your reply it seems that is not the case.

Perhaps then it could just check if there are any .html files and warn the user these are non compliant after conversion, or even before conversion and ask if it should proceed.

Maybe you don't need such a reminder, some of us do.

[KevinH]

Then code it up and add it to your version. BeckyEbook's code shows the correct way to handle it.

As I have already explained, the instructions are clear on how to use the plugin, and running epubcheck afterwards is much better than a single check for xhtml file extensions.

[BKh]

Hello, hope it's ok to open this thread back up...

I'm running the plugin on a very old (~2013) epub that I made. First running the epub check plugin returns no errors. However when I run the ePub3-itizer I get the following error after all the xhtml files say they are converted:

Traceback (most recent call last):
  File "C:\Program Files\Sigil\plugin_launchers\python\launcher.py", line 142, in launch
    self.exitcode = target_script.run(container)
  File "C:\Users\user\AppData\Local\sigil-ebook\sigil\plugins\ePub3-itizer\plugin.py", line 182, in run
    data, mprops, sprops, etypes = convert_xhtml(bk, mid, bookhref)
  File "C:\Users\user\AppData\Local\sigil-ebook\sigil\plugins\ePub3-itizer\plugin.py", line 534, in convert_xhtml
    qp.setContent(bk.readfile(mid))
  File "C:\Program Files\Sigil\plugin_launchers\python\outputcontainer.py", line 137, in readfile
    return self._w.readfile(id)
  File "C:\Program Files\Sigil\plugin_launchers\python\wrapper.py", line 610, in readfile
    raise WrapperException('Id does not exist in manifest')
wrapper.WrapperException: Id does not exist in manifest
Error: Id does not exist in manifest

I'm not clear on what that error message is telling me what to do. Not sure which Id is not existing. Other than the xhtml files, there is a toc.ncx file and a content.opf file.

Can anyone offer any suggestions as to what I should be trying to do. I have about 20 of these old epubs and was going to upgrade them to epub 3 just for the sake of ... well ... I don't know. Seems like a good idea since they are almost 10 years old.

Thanks!

[Doitsu]

Originally Posted by BKh

Can anyone offer any suggestions as to what I should be trying to do.

Theoretically, you should never see this error message, unless the .opf file contains invalid id values.
What was the last status message before you got the error message?

For example:

..converting: file.html with manifest id:  id

The id value shown after "with manifest id:" is the id that the plugin has problems with.

[BKh]

Ah, that makes perfect sense. I thought it had gotten to the end of the book so I didn't pay close attention.

Somehow a file name without a diacritic mark ended up in the opf with one.

Now it's working. Thanks!

[KevinH]

Perhaps a change in case for the id? Or a missing id? One way we can help is to see the opf. You can use BookBrowser in Sigil and save it to a separate file. Then zip that file up and post just it here and I would be happy to take a peak at it.

[AlanHK]

Originally Posted by BKh

I have about 20 of these old epubs and was going to upgrade them to epub 3 just for the sake of ... well ... I don't know. Seems like a good idea since they are almost 10 years old.

Only reason I've found to do this is that it makes Apple (i)Books happier.
If you update an epub2 and replace a file, it won't update any new metadata, like TOC. So it's impossible to use epub2 for a "work in progress" or one you modify or add sections to.

Otherwise, I don't need any of the features of epub3, so conversion just makes the files harder to work with.

But if you do update to epub3, one thing you can do is use the NAV file conversion creates as the TOC page as well, delete any existing TOC page.
The default formatting is unattractive, so I paste this in the header:

<style type="text/css">
h1 {
  font-style: italic;
  text-align: center;
  font-weight: normal;
  font-size: 1.8em;
}
ol { list-style-type: none;}
li {
  font-weight: normal;
  text-align: left;
  margin-left: 0em;
  font-size: 1rem;
  text-indent: -2em;
}
a {  text-decoration: none;}
    </style>

Modify to taste.
When updating the TOC in Sigil it preserves the file header so this will be kept.