Bug: duplicate manifest file entry

[un_pogaz]

So, I enconter a malfomerd OPF that duplicate manifest file entry (two <item> for the same file with different ids) with one "orphan" (not referenced into the spine).

Example:

  <manifest>
    <item id="DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000.html" media-type="application/xhtml+xml"/>
    <item id="DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_001" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_001.html" media-type="application/xhtml+xml"/>
    <item id="DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_002" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_002.html" media-type="application/xhtml+xml"/>
    <item id="id0" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000.html" media-type="application/xhtml+xml"/>
    <item id="id1" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_001.html" media-type="application/xhtml+xml"/>
    <item id="id2" href="Text/DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_002.html" media-type="application/xhtml+xml"/>
    <item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>
    <item id="titlepage.xhtml" href="Text/titlepage.xhtml" media-type="application/xhtml+xml"/>
  </manifest>
  <spine toc="ncx">
    <itemref idref="titlepage.xhtml"/>
    <itemref idref="id0"/>
    <itemref idref="id1"/>
    <itemref idref="id2"/>
  </spine>

As it stands, this is fairly harmless, Sigil work fine wihout enconter any issue.
Unless you launch a plugin that iterates through all the text files with "text_iter()". The plugin will fail because try to read the "orphan" entry that is not referenced into the spine.

Statut : failed

file: titlepage.xhtml
html_id: titlepage.xhtml
file: DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000.html
html_id: id0
file: DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_001.html
html_id: id1
file: DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_002.html
html_id: id2
file: DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000.html
html_id: DeVilliersGerard-SAS-110TUEZRIGOBERTAMENCHU_split_000
Traceback (most recent call last):
  File "/usr/share/sigil/plugin_launchers/python/launcher.py", line 142, in launch
    self.exitcode = target_script.run(container)
                    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/mnt/home/un_pogaz/.local/share/sigil-ebook/sigil/plugins/CleanMSO/plugin.py", line 16, in run
    text = bk.readfile(html_id)
  File "/usr/share/sigil/plugin_launchers/python/bookcontainer.py", line 176, in readfile
    return self._w.readfile(id)
           ~~~~~~~~~~~~~~~~^^^^
  File "/usr/share/sigil/plugin_launchers/python/wrapper.py", line 625, in readfile
    raise WrapperException('Id does not exist in manifest')
wrapper.WrapperException: Id does not exist in manifest
Error: Id does not exist in manifest

And no, its not me who play with the OPF, I found the epub as it.

[KevinH]

The first 3 entries have unique ids but identical file paths inside the epub according to the manifest.

Does epubcheck detect the duplicate manifest entries?

Also will you please copy and rename the epub to .zip and run unzip -t on it to see if these duplicates are identical files listed inside the epub zip itself (maybe with different capitalizations) or just maybe symlinks to the same file?

Really the only way to detect this error during the load is to walk the manifest in advance of the load and see if a file with that exact same path has already been seen and if so generate a opf read warning of some sort? Then if a duplicate is an xhtml file we would have to see if it was part of the spine or not to decide whether to keep it.

I will try to code up a load warning that detects duplicate filepaths and at least manually warn the user to verify/fix its opf manifest to remove identified duplicates for the next release.

[KevinH]

Also would you please post this epub someplace and pm (KevinH) me with a link to it. Alternatively, add a password to that zip, post the password protected one as an attachment to this thread as a zip and pm me with just the password.

That way I will have a test case to run and test my changes against it.

Thanks

[un_pogaz]

Originally Posted by KevinH

The first 3 entries have unique ids but identical file paths inside the epub according to the manifest.

oops, their mean to be differents (fix but you get the idea).

Originally Posted by KevinH

Does epubcheck detect the duplicate manifest entries?

Internal check show nothing. epubcheck will report this:
Col: 101: ERROR(OPF-074): The package resource "texts/Section0001.xhtml" is declared in several items of the manifest.

Originally Posted by KevinH

Also will you please copy and rename the epub to .zip and run unzip -t on it to see if these duplicates are identical files listed inside the epub zip itself (maybe with different capitalizations) or just maybe symlinks to the same file?

Check it, their is no duplicate file or case issue. Only malformed OPF.

Originally Posted by KevinH

Also would you please post this epub someplace and pm (KevinH) me with a link to it. Alternatively, add a password to that zip, post the password protected one as an attachment to this thread as a zip and pm me with just the password.

That way I will have a test case to run and test my changes against it.

Thanks

Damn, I already manualy fix the ePub. And useless, since it's rather trivial to recreate: just open Sigil on a empty ePub (2 or 3), duplicate a entry and boom, error (tested).

Originally Posted by KevinH

I will try to code up a load warning that detects duplicate filepaths and at least manually warn the user to verify/fix its opf manifest to remove identified duplicates for the next release.

Thanks.

[KevinH]

I just pushed a change to master to add a load warning when the epub's opf manifest contains duplicate file paths. This change was pushed to master and will appear in the next release.

Thank you for your bug report.