Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Sigil

Notices

Reply
 
Thread Tools Search this Thread
Old 04-17-2016, 05:31 AM   #1
Klecks
Member
Klecks doesn't litterKlecks doesn't litterKlecks doesn't litter
 
Posts: 19
Karma: 212
Join Date: May 2010
Location: Stuttgart, Germany
Device: Kobo H2O, PocketBook 623 Touch Lux, Tolino Shine
Sigil 0.9.5 breakes some of my epubs

Hi all,

I just stumbled on a problem using sigil on some of my epubs.
Please - can someone look into this and tell me if it's only me or a bug in sigil?

Sigil 0.9.5
Windows 10 (64)
I use both - sigil and calibre.

Usually I edited my books with sigil first and they got the sigil-wanted internal folder structure.
If I later used calibre on these books, calibre didn't care about sigil's wishes and sometimes inserted additional files and folders outside the sigil-given folders. But that's OK, because everything is linked properly and the books validate with epubcheck.

Problem: if I now open these books with sigil 0.9.5 again, sigil does it's usual I-sort-everything-my-way thing and rearranges the calibre-inserted files in it's own folder-structure; but it apparently misses to update the links! So I get broken epubs ...
I managed to produce an example-book. It's one of my problem-books (obfuscated for copyright-reasons).

By the way: on my older computer I use win 7 and sigil 0.7.2.
Sigil 0.7.2 doesn't have that problem - it updates all links. I tested my example-book with both sigil-versions:
open with sigil 0.7.2 - book OK.
open with sigil 0.9.5 - book broken.

For the moment I stick with using sigil 0.7.2 on these books - but maybe someone can look into that problem?

Thanks
Klecks.
Attached Files
File Type: epub test.epub (263.9 KB, 24 views)
Klecks is offline   Reply With Quote
Old 04-17-2016, 06:37 AM   #2
Notjohn
hanging on by fingernails
Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.
 
Posts: 829
Karma: 344490
Join Date: Dec 2012
Device: Kindle
Well, the obvious solution would be to keep Calibre out of the loop. I love Calibre for its intended purpose, manipulating my library of e-books, but long ago I was so horrified by the mess it made of my html that I resolved never to let it near my own books. If 0.9.5 otherwise works for you, surely that would be best. Why mix wine and beer?
Notjohn is offline   Reply With Quote
 
Advertisement
Old 04-17-2016, 07:25 AM   #3
Klecks
Member
Klecks doesn't litterKlecks doesn't litterKlecks doesn't litter
 
Posts: 19
Karma: 212
Join Date: May 2010
Location: Stuttgart, Germany
Device: Kobo H2O, PocketBook 623 Touch Lux, Tolino Shine
I can see that now, but harm is already done.
I have several hundreds of books like that in my calibre library.
Klecks is offline   Reply With Quote
Old 04-17-2016, 08:09 AM   #4
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 12,813
Karma: 69182683
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
No need for a Sigil vs calibre throwdown. If everything is working correctly, there shouldn't really be anything one does that the other can't handle without "breaking" the epub. I'll take a look at this as soon as I can (if Kevin doesn't beat me to it ).
DiapDealer is online now   Reply With Quote
Old 04-17-2016, 08:49 AM   #5
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 12,813
Karma: 69182683
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Yeah, the test epub is a bit of a mess structurally, but it is valid, so Sigil should be able to handle it. But it doesn't. The files are being moved to Sigil's mandated structure OK, but the files' entries in the opf manifest whose hrefs start with the relative "../" are not being updated to the their new locations during Sigil's normal Source Updates. Will have to check into that.

As a workaround in the meantime, using the "Tools->Arrange into folders" feature of calibre's editor (on any epubs you edited with calibre) to make sure files are arranged into Text/, Styles, Images/, Fonts/ folders (OPF and NCX fields empty) will ensure that Sigil will be able to open the epub and successfully update the links. You must, of course, save the epub after making those changes in calibre's editor.

Thanks for pointing out the problem.
DiapDealer is online now   Reply With Quote
Old 04-17-2016, 09:30 AM   #6
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,967
Karma: 660198
Join Date: Nov 2009
Device: many
Yes, having something be above the content.opf (in a higher directory) has probably never been tested. I have seen equal and of course in subdirectories but I don't think I have ever seen one where you go up a directory from the opf. That said, the way we build up the absolute url from the relative should properly handle this case unless the filenames don't match exactly in case.

I will try to look at this today if I get some free time.

KevinH
KevinH is offline   Reply With Quote
Old 04-17-2016, 11:58 AM   #7
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 12,813
Karma: 69182683
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
As far as the OPF is concerned, part of the problem is that paths like "OEBPS/../cover.jpg" passed in the keylist parameter to xmlprocessor.performOPFSourceUpdates() are not matching the ../cover.jpg href in the OPF manifest ... so no updates are performed on those entries (even though the new/correct path is available in the valuelist parameter).

os.path.normpath-ing the keylist entries will allow the match to happen and the OPF file will be properly updated, but that does nothing for the NCX entries, or any of the links in the xhtml, or the xref:link parameter in the SVG cover code. I have a feeling there's a more universal change to be made that will handle the overall situation better that I'm missing.
DiapDealer is online now   Reply With Quote
Old 04-17-2016, 12:03 PM   #8
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,127
Karma: 72218367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by Notjohn View Post
Well, the obvious solution would be to keep Calibre out of the loop. I love Calibre for its intended purpose, manipulating my library of e-books, but long ago I was so horrified by the mess it made of my html that I resolved never to let it near my own books. If 0.9.5 otherwise works for you, surely that would be best. Why mix wine and beer?
Mainly because you are spreading FUD?

calibre does not make a mess of your HTML unless you foolishly decide to use the conversion feature instead of the Editor.
Which is of course entirely appropriate for end-users, but you aren't an end-user so why on earth would you use the conversion feature?


I really don't understand why some people insistently continue to fraudulently compare the Sigil ebook editor to the calibre automated ebook converter.
Rather than comparing the Sigil ebook editor to the calibre ebook editor.

...

As DiapDealer said, it might be a good idea to use calibre's Arrange Into Folders tool -- which I specifically requested so people can manually set up a standardized folder structure like Sigil does automatically.

Also, this is quite obviously a Sigil bug (and not dependent on using calibre, even if calibre usage is likely to trigger the bug) -- so why bash calibre when Sigil will soon work perfectly and they will all play nice together anyway?

Last edited by eschwartz; 04-17-2016 at 12:06 PM.
eschwartz is offline   Reply With Quote
Old 04-17-2016, 12:18 PM   #9
Klecks
Member
Klecks doesn't litterKlecks doesn't litterKlecks doesn't litter
 
Posts: 19
Karma: 212
Join Date: May 2010
Location: Stuttgart, Germany
Device: Kobo H2O, PocketBook 623 Touch Lux, Tolino Shine
Peace - I like both: Calibre and Sigil.
And apparently I managed - by using both - to create epubs, that are "unique".

Thanks for your help.
Klecks.
Klecks is offline   Reply With Quote
Old 04-17-2016, 12:18 PM   #10
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,967
Karma: 660198
Join Date: Nov 2009
Device: many
Wow, that test.epub structure is a real mess. The epub spec in section 2.2 of the standard has only this to say about things outside of the META-INF:

"It is *recommended* that the contents of the EPUB Publication be stored within its own dedicated directory under the container's root."

Notice it says "recommended" not "required". This test.epub has some pieces both inside and outside the OEBPS folder (ie. jpegs outside of the folder) which typically is the "recommended common folder" to hold all of the publication pieces. But this is only a "recommendation" and not a "requirement", so Sigil should properly handle it.

So I will take a closer look at it.

Thanks for the head's up.

KevinH
KevinH is offline   Reply With Quote
Old 04-17-2016, 12:50 PM   #11
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,967
Karma: 660198
Join Date: Nov 2009
Device: many
Now hopefully fixed in master.
Thank you for your bug report.

KevinH
KevinH is offline   Reply With Quote
Old 04-17-2016, 01:02 PM   #12
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,967
Karma: 660198
Join Date: Nov 2009
Device: many
Hi DiapDealer,
IMO, the only sane way to do this is to build the original path from the ebook root (normalized) to the file's original location in the epub and use that as a lookup key linked to its new position inside Sigil's ebook structure.

Then for each file that has links that need updating, take its original position relative to root, add it to whatever href or url that is linked, then normalize it (QDir::cleanPath or similar functions) to generate a string to match against the the set of updates key value pairs generated when the epub was imported by walking the manifest.

This way works but we never ran into a case where the key itself was not properly normalized as most paths relative to the the ebook root were OEBPS/something without any "../" . So your solution of normalizing the key is correct and works as long as we normalize to ebook root, the value we look up to match how the key was designed.

So unless I am messed up, I think your approach is the same as mine, it just means we need to normpath both the key and the lookup value based on the original file location plus the decoded url as well.

The bug was we never ran into a case where the key itself had an ../ in the path (ie. other epubs followed the epub spec recommendation) but now we handle that and normalize the key itself given some manifest links now had "../" in them.

Take care,

KevinH

Quote:
Originally Posted by DiapDealer View Post
As far as the OPF is concerned, part of the problem is that paths like "OEBPS/../cover.jpg" passed in the keylist parameter to xmlprocessor.performOPFSourceUpdates() are not matching the ../cover.jpg href in the OPF manifest ... so no updates are performed on those entries (even though the new/correct path is available in the valuelist parameter).

os.path.normpath-ing the keylist entries will allow the match to happen and the OPF file will be properly updated, but that does nothing for the NCX entries, or any of the links in the xhtml, or the xref:link parameter in the SVG cover code. I have a feeling there's a more universal change to be made that will handle the overall situation better that I'm missing.
KevinH is offline   Reply With Quote
Old 04-17-2016, 01:20 PM   #13
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 12,813
Karma: 69182683
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
So unless I am messed up, I think your approach is the same as mine, it just means we need to normpath both the key and the lookup value based on the original file location plus the decoded url as well.
The "same" except that yours seems to work for everything instead of just the OPF file (which is all the farther my experimentation/debugging had taken me).

I knew the problem probably needed to be addressed farther back along the trail, somewhere, I just hadn't located where best (and exactly how) to do it.

Thanks for the explanation. It (and your solution) made perfect sense.

Last edited by DiapDealer; 04-17-2016 at 01:26 PM.
DiapDealer is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Merge a collection of epubs with Sigil portersprings Sigil 2 08-14-2014 09:39 PM
Opening EPUBs in Sigil takes ages [Windows] Wasserpulle Sigil 4 12-13-2013 01:05 PM
No-Frills-Sigil-Epubs webe Sigil 7 11-01-2011 12:03 PM
Sigil stripping covers from epubs Antioch Sigil 3 06-26-2011 12:09 AM
Advice on using Sigil to standardise all epubs for my Kobo digital_steve Sigil 3 08-10-2010 09:29 PM


All times are GMT -4. The time now is 12:08 PM.


MobileRead.com is a privately owned, operated and funded community.