Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Sigil

Notices

Reply
 
Thread Tools Search this Thread
Old 11-19-2019, 11:33 AM   #16
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,943
Karma: 2514398
Join Date: Nov 2009
Device: many
The very first time an epub file is loaded we run it through the following code to figure out its encoding, convert it to utf-8, and change the line endings ...

See: Sigil/src/Misc/HTMLEncodingResolver.cpp

Code:
// Accepts a full path to an HTML file.
// Reads the file, detects the encoding
// and returns the text converted to Unicode.
QString HTMLEncodingResolver::ReadHTMLFile(const QString &fullfilepath)
{
    QFile file(fullfilepath);

    // Check if we can open the file
    if (!file.open(QFile::ReadOnly)) {
        std::string msg = file.fileName().toStdString() + ": " + file.errorString().toStdString();
        throw (CannotOpenFile(msg));
    }

    QByteArray data = file.readAll();

    if (IsValidUtf8(data)) {
        data.replace("\xC2\xA0", " ");
    }

    return Utility::ConvertLineEndings(GetCodecForHTML(data)->toUnicode(data));
}
I think this is the culprit. It is what is special casing the nbsp. We could remove this manual conversion and instead pass it through PreserveEntities here instead to always set the files on first input to have only the entities the user specified.

I think it was a holdover from an earlier time that we never saw since we used to always run mend on every file to do the universal updates which always ran things through PreserveEntities.

How do you want to handle this? If we add in PreserveEntities code here instead of manuall setting that one, at least the epub will present itself with the entities the user expects.

KevinH



Quote:
Originally Posted by DiapDealer View Post
Has anyone else been able to duplicate this issue? Is the no-break-space being special-cased here?
KevinH is offline   Reply With Quote
Old 11-19-2019, 11:43 AM   #17
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,943
Karma: 2514398
Join Date: Nov 2009
Device: many
Actually running it through PreserveEntities without gumbo being involved will not help. Anything that was an entity will stay an entity.

So perhaps we simply just delete this forced entity conversion and let the user decide when to run Mend to get only the entities they want in every file.

KevinH
KevinH is offline   Reply With Quote
Old 11-19-2019, 12:01 PM   #18
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 6,535
Karma: 29910991
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Aura One, Aura H2O, Aura HD, Nexus 7 HD, iPad Air, Tolino epos
Quote:
Originally Posted by KevinH View Post
Actually running it through PreserveEntities without gumbo being involved will not help. Anything that was an entity will stay an entity.

So perhaps we simply just delete this forced entity conversion and let the user decide when to run Mend to get only the entities they want in every file.

KevinH
That gets my vote.

What? This isn't a democracy???
DNSB is offline   Reply With Quote
Old 11-19-2019, 12:43 PM   #19
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 21,387
Karma: 116833652
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
So perhaps we simply just delete this forced entity conversion and let the user decide when to run Mend to get only the entities they want in every file.
Works for me, too.
DiapDealer is offline   Reply With Quote
Old 11-19-2019, 12:43 PM   #20
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 21,387
Karma: 116833652
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by DNSB View Post
What? This isn't a democracy???
Sure it is. It's just a very small one!
DiapDealer is offline   Reply With Quote
Old 11-19-2019, 12:53 PM   #21
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,943
Karma: 2514398
Join Date: Nov 2009
Device: many
will do
KevinH is offline   Reply With Quote
Old 11-19-2019, 01:47 PM   #22
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,943
Karma: 2514398
Join Date: Nov 2009
Device: many
change pushed to master
KevinH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Metadata oddities MSWallack Marvin 3 11-20-2014 02:55 AM
Catalog oddities tamhas Library Management 7 07-25-2014 11:55 AM
decimal entities in ePub instead of character entities epub4ever Calibre 4 04-20-2012 03:27 AM
Anachronism or other oddities Hellmark General Discussions 34 05-03-2011 02:28 PM


All times are GMT -4. The time now is 07:27 AM.


MobileRead.com is a privately owned, operated and funded community.