[Bug?] epub3 - nav destroyer

BeckyEbook · 06-12-2018, 11:35 AM

Step-by-step to destroy nav file.

1. Get any epub3 file with nav file.
2. Open nav file in Code View.
3. Go to <head> section.
4. Try adding the <title> or <style> inside the head section.
Just start typing <title or <style

The nav file is destroyed and the default nav (from the NavProcessor.cpp file) appears instead.

While the title in nav file does not make too much sense, declaring styles is useful.

KevinH · 06-12-2018, 12:08 PM

When you exit the nav (move focus to outside the nav document) did you leave it in a legal state? Or must only a partial tag in head be present?

BeckyEbook · 06-12-2018, 12:20 PM

Quote:

Originally Posted by KevinH

When you exit the nav (move focus to outside the nav document) did you leave it in a legal state? Or must only a partial tag in head be present?

I do not want to leave the nav file.
I want to continue writing ... <style type="text/css"> etc.

Exactly when I write "title" or "style" in an open tag the nav file is reset.

ADDED:
When I paste:

Code:

  <style type="text/css">
    nav#landmarks, nav#page-list { display:none; }
    ol { list-style-type: none; }
  </style>

All is OK.

But when I paste (or write) only the opening tag:

Code:

<style type="text/css">

nav file is reset.

KevinH · 06-12-2018, 01:25 PM

Yes this is a bug. Every edited xhtml file is saved saved by focus moving or even pausing long enough.

The problem is that the nav must remain machine readable at all time as it provides the navigation toc. If you look at NavProcessor.cpp, it tries to parse the current nav and if for any reason it can not find a nav tag with epub:type equal to toc, it assumes the nav is "garbage" and tries to rebuild or replace it.

The problem is something is making focus move out of the nav's tab during intermediate typing and anything that is incomplete is making the Nav fail parsing and causing it to reject the nav and replace it.

This is obviously a bug. I either have to find out what is causing focus to move which in turn makes the incomplete file be saved and then the NavProcessor fail in its parse attempt.

I will look into either more robust parsing methods, or using gumbo to fix things first on the fly before parsing by NavProcessor to try to prevent this from happening.

Thanks for the bug report.

KevinH

KevinH · 06-18-2018, 07:36 PM

Okay, after some investigation, here is what I learned:

1. The bookbrowser uses the nav to generate mouseover strings that describe semantics and this model is auto regenerated like the toc window itself causing the nav to need to be parseable at almost any time.

2. After testing firefox, and safari browsers, and parsing errors in title or style tags in the head have to follow the rules for parsing html and gumbo does at well. The rules of parsing will cause the main body to hide inside head and result in an extra empty body being generated (gumbo does this too)

3. the NavProcessor will therefore take the gumbo/browser based code and fail to find any nav elements in the now empty body tag. This causes the NavProcessor to abandon the current nav as unvalid and start from scratch thereby doing the destroying.

So I will have to create valid nav detection code (probably using regular expressions) to be tried if gumbo/browser rule parsing ever fails before having it give up and throw in the towel.

I will try to rewrite the nav parser to be more robust and tolerant since it can be autoparsed at almost any time.

I will alert you when a fix is pushed to master. In the meanwhile, I do not recommend playing around inside the head tag of the nav, until this is fixed.

KevinH · 06-20-2018, 07:40 AM

If you get a free moment, would you please test something for me. In NavProcessor in the constructor, would you please try removing the entire if (!valid) segment of code and see if simply ignoring the nav when broken for parsing is feasible.

Here are the lines to delete or ifdef out from NavProcessor.cpp

Code:

     if (!valid) {
          SettingsStore ss;
          QString lang = ss.defaultMetadataLang();
          QString newsource = 
            "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"
            "<!DOCTYPE html>\n"
            "<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:epub=\"http://www.idpf.org/2007/ops\" "
            "lang=\"%1\" xml:lang=\"%2\">\n"
            "<head>\n"
            "  <meta charset=\"utf-8\" />\n"
            "  <style type=\"text/css\">\n"
            "    nav#landmarks, nav#page-list { display:none; }\n"
            "    ol { list-style-type: none; }\n"
            "  </style>\n"
            "</head>\n"
            "<body epub:type=\"frontmatter\">\n"
            "  <nav epub:type=\"toc\" id=\"toc\">\n"
            "  </nav>\n"
            "  <nav epub:type=\"landmarks\" id=\"landmarks\" hidden=\"\">\n"
            "  </nav>\n"
            "</body>\n"
            "</html>";
          newsource = newsource.arg(lang).arg(lang);
          QWriteLocker locker(&m_NavResource->GetLock());
          m_NavResource->SetText(newsource);
    }

Does that help? Do you notice any strange behaviour? In BookBrowser if you mouseover or hover on a file that had been marked with semantics (existed in nav landmarks), what happens?

Thanks,

KevinH

BeckyEbook · 06-20-2018, 11:16 AM

Quote:

Originally Posted by KevinH

If you get a free moment, would you please test something for me. In NavProcessor in the constructor, would you please try removing the entire if (!valid) segment of code and see if simply ignoring the nav when broken for parsing is feasible.

Hmmm…
Ignoring is a nice idea.
File is not destroyed and I see message with error on Preview window.

IMAGE

Quote:

Originally Posted by KevinH

Does that help? Do you notice any strange behaviour? In BookBrowser if you mouseover or hover on a file that had been marked with semantics (existed in nav landmarks), what happens?

I did not notice any problems while jumping (mouseover, hover, click, add/remove semantics) in BookBrowser. I will watch this closely.

KevinH · 06-28-2018, 01:01 PM

Okay, I think I have this one fixed without have to ignore the entire nav. I was able to use regular expressions to try harder to extract the toc, landmarks, and page-list just in case a parser error exists when asked to parse the nav someplace else. So it should be more robust to parsing errors even just temporary ones.

I have pushed the fixes to master.

If you get a free moment, please give this a try.

BeckyEbook · 06-28-2018, 02:37 PM

Is OK.
Thank you.

06-12-2018, 11:35 AM	#1
BeckyEbook Guru Posts: 692 Karma: 2180740 Join Date: Jan 2017 Location: Poland Device: Misc	[Bug?] epub3 - nav destroyer Step-by-step to destroy nav file. 1. Get any epub3 file with nav file. 2. Open nav file in Code View. 3. Go to <head> section. 4. Try adding the <title> or <style> inside the head section. Just start typing <title or <style The nav file is destroyed and the default nav (from the NavProcessor.cpp file) appears instead. While the title in nav file does not make too much sense, declaring styles is useful. Last edited by BeckyEbook; 06-12-2018 at 12:04 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Free (Kindle) Dark Destroyer by John Glasby [Village vs Supernatural Mystery Horror]	ATDrake	Deals and Resources (No Self-Promotion or Affiliate Links)	2	09-22-2017 01:39 PM
Free (Kindle) Destroyer of Worlds by E. C. Tubb [Vintage Rediscovered Sci-Fi Novel]	ATDrake	Deals and Resources (No Self-Promotion or Affiliate Links)	2	09-21-2016 03:22 AM
Bug Report: Mobi Conversion puts TOC Nav in wrong spot	therealjoeblow	Conversion	4	12-02-2011 09:56 AM
Mystery and Crime Stevenson, Burton E.: The Destroyer. v. 1.1, 23 Feb 2010	bobcdy	ePub Books (offline)	1	02-22-2010 08:53 PM
History Jones, Ken: Destroyer Squadron23, v1, 29 Sept 2008	mrmikel	BBeB/LRF Books (offline)	0	09-29-2008 07:33 PM

06-12-2018, 12:08 PM	#2
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	When you exit the nav (move focus to outside the nav document) did you leave it in a legal state? Or must only a partial tag in head be present?

06-12-2018, 01:25 PM	#4
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Yes this is a bug. Every edited xhtml file is saved saved by focus moving or even pausing long enough. The problem is that the nav must remain machine readable at all time as it provides the navigation toc. If you look at NavProcessor.cpp, it tries to parse the current nav and if for any reason it can not find a nav tag with epub:type equal to toc, it assumes the nav is "garbage" and tries to rebuild or replace it. The problem is something is making focus move out of the nav's tab during intermediate typing and anything that is incomplete is making the Nav fail parsing and causing it to reject the nav and replace it. This is obviously a bug. I either have to find out what is causing focus to move which in turn makes the incomplete file be saved and then the NavProcessor fail in its parse attempt. I will look into either more robust parsing methods, or using gumbo to fix things first on the fly before parsing by NavProcessor to try to prevent this from happening. Thanks for the bug report. KevinH

06-18-2018, 07:36 PM	#5
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Okay, after some investigation, here is what I learned: 1. The bookbrowser uses the nav to generate mouseover strings that describe semantics and this model is auto regenerated like the toc window itself causing the nav to need to be parseable at almost any time. 2. After testing firefox, and safari browsers, and parsing errors in title or style tags in the head have to follow the rules for parsing html and gumbo does at well. The rules of parsing will cause the main body to hide inside head and result in an extra empty body being generated (gumbo does this too) 3. the NavProcessor will therefore take the gumbo/browser based code and fail to find any nav elements in the now empty body tag. This causes the NavProcessor to abandon the current nav as unvalid and start from scratch thereby doing the destroying. So I will have to create valid nav detection code (probably using regular expressions) to be tried if gumbo/browser rule parsing ever fails before having it give up and throw in the towel. I will try to rewrite the nav parser to be more robust and tolerant since it can be autoparsed at almost any time. I will alert you when a fix is pushed to master. In the meanwhile, I do not recommend playing around inside the head tag of the nav, until this is fixed.

06-28-2018, 01:01 PM	#8
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Okay, I think I have this one fixed without have to ignore the entire nav. I was able to use regular expressions to try harder to extract the toc, landmarks, and page-list just in case a parser error exists when asked to parse the nav someplace else. So it should be more robust to parsing errors even just temporary ones. I have pushed the fixes to master. If you get a free moment, please give this a try.

06-28-2018, 02:37 PM	#9
BeckyEbook Guru Posts: 692 Karma: 2180740 Join Date: Jan 2017 Location: Poland Device: Misc	Is OK. Thank you.