03-27-2014, 02:55 AM | #1 |
Wizard
Posts: 2,608
Karma: 3000161
Join Date: Jan 2009
Device: Kindle PW3 (wifi)
|
About UTF-16 parsing mistake
Hi
I did a curious and fairly reproductible experiment using the broken EPUB I presented to you yesterday. As I am more careful today, I'll let you decide if this is an Editor bug or an advanced feature... I unchecked this box in the Editor preferences (screenshot 1). This means, if I understand correctly, that no UTF-16 character will be created to replace a named entity like nbsp. (of which there are 611 in the book). Then I just modified one word on chapter 2 with the Editor (just change one word and back) and saved the file. 1. - If I open this same chapter 2 file with the Editor, there will be no reading problem but if I check the book, the Editor reports now an error for this modified chapter: "Parsing failed: Document labelled UTF-16 but has UTF-8 content, line 1..." (scr 4 - far right). 2. - Opening this file with Sigil 0.7.4, things are even more gloomy: Sigil gives a warning (scr 2). Looking at the files, I observed that the DOCTYPE and the nbsp have indeed been logically maintained but the modified chapter 2 file is declared unreadable on Sigil without any reason given (in fact it's unreadable because it's declared as UTF-16). If I try to open the chapter 2 xhtml file, it will look a little like Chinese but written by me (scr 3). Changing UTF-16 with UTF-8 in the declaration solves all problems for both editors. If the Editor cannot parse, if Sigil is bewildered by this change, then why do it? Proposal. When the user unchecks the preferences checkbox alluded above (scr 1), not only the nbsp and DOCTYPE should be preserved like now, but the file should stay declared as UTF-8. Last edited by roger64; 03-27-2014 at 03:25 AM. |
03-27-2014, 03:00 AM | #2 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
That option only control replacement of entities as they are typed. The editor never uses UTF-16 or generates UTF-16 encoded files, ever. The editor always outputs UTF-8.
|
03-27-2014, 03:31 AM | #3 | |
Wizard
Posts: 2,608
Karma: 3000161
Join Date: Jan 2009
Device: Kindle PW3 (wifi)
|
Quote:
So since the Editor outputs UTF-8, and using this option maintains DOCTYPE and nbsp (in this particular checkbox case), there should be no reading problems anymore with Sigil. Excellent. Sorry for this. It's again my mistake. One day my EPUB will be unbroken...I learnt another thing today. And here to conclude a final -but not perfect- EPUB with subsetted fonts and 'traditional' nbsp. All other reported mistakes suppressed. Going to and from Sigil without problem. (Sigil used namely for splitting two chapters). Last edited by roger64; 03-27-2014 at 06:08 AM. |
|
03-27-2014, 09:34 AM | #4 |
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Good Info, but...
One way to get your post seemingly ignored is to completely rewrite a post 2.5 hours after you initially post it. No one is pinged by the board that you updated a post. It is possible Kovid read your initial post and will never read your completely rewritten post because there is no mechanism to let anyone know you completely rewrote the post. It is best to just write an additional new post. That way folks following the conversation will have an opportunity to read what you learned. |
03-27-2014, 09:49 AM | #5 |
Wizard
Posts: 2,608
Karma: 3000161
Join Date: Jan 2009
Device: Kindle PW3 (wifi)
|
@DoctorOhh
Thank you for following the thread. The main point of my message is that I found the reason of this behaviour. It was quickly given and made a second reply useless. Then I added some minor edits and I finally posted much later the "unbroken" EPUB. I put it in the same message because I did not wish to disturb people with a new post. But, I agree that I may be wrong about it. |
03-27-2014, 09:56 AM | #6 | |
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
Thanks for providing updated info. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Parsing data from feed | atordo | Recipes | 1 | 01-23-2014 03:50 PM |
HTML5 parsing | nickredding | Conversion | 8 | 08-09-2012 09:50 AM |
Parsing Index | Steven630 | Recipes | 0 | 07-06-2012 04:53 AM |
iPad PageList parsing using Javascript. | Oh.Danny.Boy | Apple Devices | 0 | 05-17-2012 05:24 PM |
Parsing Titles | cgraving | Calibre | 3 | 01-17-2011 02:52 AM |