LF or CRLF line ending?

[CarFMI]

I'm new to the Sigil ePUB editor and tried it out today (v2.8.1 on Windows), and noticed that when saving, in was converting the line endings from LF to CRLF.

Is this the norm for this application?

Is it not supposed to be using only LF (Unix style) for proper EPUB conformity?

I did run a search on the forum first with the keywords “CRLF” and “CRLF ending” and “LF ending” but did not seem to find any relevant posts about the issue.

[KevinH]

The epub spec properly allows and accepts both cr/lf and just lf as will any xhtml or html document where both are considered whitespace and therefore condensed to one. So both are supported by the epub, xhtml, and html specs.

When reading from Windows, Sigil will strip off crlf to just linefeeds for editing in Qt/Sigil's Codeview, and when saving to a file on Windows Sigil puts them back since open-with and other text editors want crlf on Windows.

So whoever said lf only for epub was mistaken. The whatwg living html spec and xhtml parsers all handle both cases properly.

The ⁠EPUB 3.4 specification defines packaging for web content (HTML, CSS, SVG) inside a ZIP container. Line breaks inside these XHTML or text files use standard digital control characters—Line Feed (LF) or Carriage Return + Line Feed (CRLF)—matching general web standards where line endings are handled transparently by XML parsers.

[CarFMI]

Alright, but then why is Sigil forcing an existing LF file into CRLF if it is less space efficient?

Is this only a Windows version issue?

Should it not be standardised independently of the platform, or maybe have a user setting to define which format to use?

PS! All of the editors I have used on Windows have had no issues with working with LF (unix style) files, and have even preserved the format when resaving them. So I don't think it mandatory for the Windows version of Sigil to force CRLF.

[KevinH]

To save space? Huh?

Windows still standardizes on crlf for text. It may now allow others but the native format is crlf. And yes this is something that only happens on Windows. Neither MacOS nor any Unix based alternative needs CRLF.

This has been happening with Sigil on Windows for over 15 years and no one has complained once.

Current Status on WindowsDefault Behavior:
Built-in tools and standard APIs on Windows still output and expect CRLF by default to preserve backward compatibility with legacy software and command-line tools.

Notepad Updates: Windows ⁠Notepad natively parses and maintains LF or CR line breaks in existing files rather than corrupting them, but it still defaults to stamping new files with CRLF.

Is this your way of asking for a feature request or env variable to prevent them from being added back on Windows? If so, you are going about it in a strange way.

Let's see what if anything other longterm users of Sigil on Windows say about this. If they too want this behaviour to change, I will add that is a future version of Sigil.

[DNSB]

A topic on which I am totally neutral.

I have worked on ePubs where both line endings were used and saw no difference in the behaviour when they were opened for reading. At one time, I unzipped the files for 11 ePub/kepubs and used NotePad++ to change the EOL and then re-zipped them. Since ePub files are stored in a zip container, the file size difference was negligible. On my Kobo readers using both renderers, I could see no changes in displayed text or behaviour.

A quote from Macbeth comes to mind: "full of sound and fury, signifying nothing."

[DiapDealer]

I work with epubs in Sigil on Linux and Windows and I can't think of a single time Sigil's behavior RE line endings ever affected me (or crossed my mind for that matter).

[KevinH]

And for what it is worth, Sigil uses Qt's QTextStream for writing out files and it has the following default behaviour:

By default, QTextStream automatically translates line endings depending on the operating system and whether the underlying QIODevice is opened in text mode (QIODevice::Text). On Windows, writing a newline via \n or Qt::endl typically gets converted to CRLF (\r\n), while reading handles both LF and CRLF transparently.

so this is built-in behavior for Qt and their QTextStream class implementation, which means we would have to write files on Windows in binary mode to prevent Qt from doing what it does.

And it is the default for writing text in python3 on Windows as well. So both Qt and python3 both do this by default for text file writing on Windows. Sigil seems to be in good company.

[DNSB]

Originally Posted by DiapDealer

I work with epubs in Sigil on Linux and Windows and I can't think of a single time Sigil's behavior RE line endings ever affected me (or crossed my mind for that matter).

The only reason I even thought about it was that one author whom I did some work for after their book rights had reverted to them had been told that ePubs needed to have LF only as the EOL. After a couple of hours of testing, research and too much coffee, the conclusion was that either could be used. Epubcheck, Amazon's pipeline and Kobo didn't care which was used and neither of us was going to waste anymore time worrying about it.

[CarFMI]

Originally Posted by KevinH

Is this your way of asking for a feature request or env variable to prevent them from being added back on Windows? If so, you are going about it in a strange way.

No, I cannot in way be asking for changes, when I just started using it today. I am just trying to understand how it works.

Basically, I was trying to see if my current work flow would benefit from using it.

Thank you for explaining the philosophy behind how it works.

[KevinH]

If it helps, a Sigil output plugin could easily be created to save any epub to Windows without any crlfs in it.

But be aware if you use git for version control that by default git auto handles crlf conversion on Windows commits and checkouts so you may be in for a surprise if using the default git for revision control.

And fwiw I think there is a Linux subsystem on Windows that now supports gui apps that would of course not need or use crlf.

Hope something here helps.

[CarFMI]

Originally Posted by KevinH

But be aware if you use git for version control that by default git auto handles crlf conversion on Windows commits and checkouts so you may be in for a surprise if using the default git for revision control.

No, I have set "core.autocrlf" accordingly to prevent changes to files irrespective of the format. I usually prefer working in LF no matter the platform.

I have used the Linux Subsystem on Windows before, but found it clunky, but thank you for the additional information.