[GUI Plugin] Modify ePub

[chaley]

Originally Posted by BetterRed

Anyone who won't take the credit for something they do is usually hoping they won't be blamed when said something goes belly up

Well, that wasn't my reason but in the end you aren't wrong. TBH I don't want to see this plugin again.

[DoctorOhh]

Originally Posted by chaley

Well, that wasn't my reason but in the end you aren't wrong. TBH I don't want to see this plugin again.

We both know if you put your name down as credit that the saying "no good deed goes unpunished" will certainly come back around at a later date. Like helping out a family member or friend with a computer issue. Every issue with that computer after that "good deed" comes back to you.

[Rev. Bob]

Originally Posted by chaley

V1.5.9 that fixes the black diamonds and, I hope, broke nothing else.

Am I overlooking the link?

[chaley]

Originally Posted by Rev. Bob

Am I overlooking the link?

It is there now. I probably did what I often do, forgot to press the "upload" button after adding the attachment.

[hobnail]

Originally Posted by chaley

TBH I don't want to see this plugin again.

Tar baby; dude, you're stuck with it.

[chaley]

Originally Posted by hobnail

Tar baby; dude, you're stuck with it.

Then I will make the plugin change every sentence in the book to "Please don't throw me in the briar patch," then change my email address.

[Rev. Bob]

Originally Posted by chaley

It is there now. I probably did what I often do, forgot to press the "upload" button after adding the attachment.

Got it. What I've done so far is take two random ebooks - one short story, one novel - and first save them untouched, then clone two copies of each and run my standard script under 1.4.1 and 1.5.9, both under calibre 4.23 on Win10/32. So, as close to identical conditions as I could get. Once I had those, I unpacked the modified ebooks into separate folders and ran a DOS file comparison on the lot, both in binary mode and the looser text mode.

Results, short form:

In this environment, 1.5.9 saves its modified text files with CRLF EOLs, while calibre's editor and 1.4.1 both use CR EOLs. This constitutes a very minor bug, but it is nonetheless a difference in behavior.

Results, long form:

1.4.1 saved the modified files with UNIX EOLs (CR only), while 1.5.9 saved them with DOS EOLs (CRLF). One of the short story sections displayed a phantom text-comparison difference that looked like an extra linefeed, but bringing both modified copies into Notepad++ and changing the 1.5.9 version to use UNIX EOLs made them byte-identical. The same behavior showed up in the longer book as well, and I don't see a pattern. Some chapters display several instances, while others show none at all.

I should note that in the case of at least one file for each book, this extends to a known-static file which my copy of calibre custom-generates: titlepage.xhtml. (I use different stock cover page code, but it's a calibre source file mod; it doesn't come from the books.)

It turns out that loading the beta book into the calibre editor, making and undoing a minimal change to an affected file, resaving the book, and extracting that file results in its EOLs being changed to the CR format, which is why I conclude that this is a quite minor bug in 1.5.9.

[chaley]

Originally Posted by Rev. Bob

1.4.1 saved the modified files with UNIX EOLs (CR only), while 1.5.9 saved them with DOS EOLs (CRLF). One of the short story sections displayed a phantom text-comparison difference that looked like an extra linefeed, but bringing both modified copies into Notepad++ and changing the 1.5.9 version to use UNIX EOLs made them byte-identical. The same behavior showed up in the longer book as well, and I don't see a pattern. Some chapters display several instances, while others show none at all.

FWIW: UNIX EOL is LF. IIRC Mac is CR, but that might be outdated info. DOS (Windows) is CRLF. This BS has been causing me headaches for nigh-on 45 years.

I am not surprised at the difference. The last change I made was to take out all guesswork about modifying the files. Reason: in calibre V4.3 the system method not only corrected line endings but sometimes changed to encoding to 8-bit ASCII, which broke everything. Taking it out made calibre V5 leave use the "system" line ending. I suspect if you ran the plugin on Linux or Mac you would get slightly different behaviour.

This can be fixed if we are willing to fork.

I should note that in the case of at least one file for each book, this extends to a known-static file which my copy of calibre custom-generates: titlepage.xhtml. (I use different stock cover page code, but it's a calibre source file mod; it doesn't come from the books.)

The plugin pays special attention to that file.

It turns out that loading the beta book into the calibre editor, making and undoing a minimal change to an affected file, resaving the book, and extracting that file results in its EOLs being changed to the CR format, which is why I conclude that this is a quite minor bug in 1.5.9.

HTML is supposed to ignore EOL differences. I think the rule is any of CR, CRLF, or LF are treated as a space. The line ending should make no difference at all in what the reader app displays.

That said, I am willing to back the changes out if you are willing to support a fork.

[DNSB]

Originally Posted by Rev. Bob

1.4.1 saved the modified files with UNIX EOLs (CR only), while 1.5.9 saved them with DOS EOLs (CRLF). One of the short story sections displayed a phantom text-comparison difference that looked like an extra linefeed, but bringing both modified copies into Notepad++ and changing the 1.5.9 version to use UNIX EOLs made them byte-identical. The same behavior showed up in the longer book as well, and I don't see a pattern. Some chapters display several instances, while others show none at all.

Minor nit. The UNIX EOL is a LF not a CR. Apple used CR for an EOL up to Mac OS X where they switched to using a LF.

[JSWolf]

I just tested 1.5.9 with 4.23 and no errors and no funny characters in the editor. Everything seems good.

I'll give 5.0.1 a test and report back.

[Rev. Bob]

Originally Posted by chaley

FWIW: UNIX EOL is LF. IIRC Mac is CR, but that might be outdated info. DOS (Windows) is CRLF. This BS has been causing me headaches for nigh-on 45 years.

In my defense, I’ve been awake several hours too long.

As to the rest: It’s a very minor thing, one which as far as I can tell is completely cosmetic, so I have no problem leaving it as is. I simply found it and thought, “he might want to know about that.”

[JSWolf]

Originally Posted by Rev. Bob

In my defense, I’ve been awake several hours too long.

As to the rest: It’s a very minor thing, one which as far as I can tell is completely cosmetic, so I have no problem leaving it as is. I simply found it and thought, “he might want to know about that.”

As for 4.23, it has no bother with the editor under Windows 10.

[Rev. Bob]

Originally Posted by JSWolf

As for 4.23, it has no bother with the editor under Windows 10.

Yeah, I reported as much:

Originally Posted by Rev. Bob

It turns out that loading the beta book into the calibre editor, making and undoing a minimal change to an affected file, resaving the book, and extracting that file results in its EOLs being changed to the CR format, which is why I conclude that this is a quite minor bug in 1.5.9.

Basically, if the calibre editor has flagged a file as changed, then it converts its EOLs to UNIX format when saving the book.

[hobnail]

Originally Posted by DNSB

Minor nit. The UNIX EOL is a LF not a CR. Apple used CR for an EOL up to Mac OS X where they switched to using a LF.

My recollection was that Macs used CRLF (and Windows used LFCR as chaley points out). I remember opening Mac files on my Unix system and having CRs at the end of each line (and Windows files had a CR at the beginning of each line). If it was only CR then it would be one long line with no line breaks.

But then I was using emacs and maybe it was doing something reasonable/smart about the end of lines.

[JSWolf]

I just tested 1.5.9 with 5.0.1 and no errors and no funny characters in the editor. Everything seems good with both 5.0.1 and 4.23 with version 1.5.9. Unless anyone else finds something wrong, I think we are good to go.