Starting with EPuB, 4 questions (to start out with).

[ProDigit]

Hi there!

So, I went the way of fully trying to understand EPuB, by generating a book from Notepad++ (in a way it is a more advanced version of Window's wordpad, or coding text editor), rather than have a program compile my epub.

I have a good few tens of pages to read, before I can start my first epub.
I could do a basic epub, but am studying up on some of the classes, XML and CSS, to see if there's a way how I can simplify the whole ebook generating process.

I have written a book before in HTML0, for SONY readers; but now wanted to do one for my Kobo H20.


Questions so far:
1a- What's the difference between encoding in UTF-8 and UTF-16 (or UTF-32)?
1b- And are there benefits going off the UTF-8 format?
1c- I've seen books omit the encoding type in the container.xml. Recommendable, or not?

2- I've read, but want to confirm, if ZIP is the only compression format allowed? 7z, or RAR would not work, correct?

3- If ZIP is, then what parameters should I use? I know at times it is possible to increase compression in some programs, and wondered if it would affect my ebook reader, in trying to open the file?

4- A few years ago, I remember someone posting a link, to a program where mimetype was fused in a zip container (without it being compressed as well).
The program also claimed it compressed far better than Winzip, because it solely focused on text and picture compression, not audio or video, and thus was able to get higher compression ratios.
Which program was this?

5- What program would you think is best to use to generate an .epub file, knowing that all other files are sorted and available? (meaning mimetype, META-INF folder, and the OEBPS folder are complete)?

[eschwartz]

1) UTF-16 is like UTF-8 except it takes up more space. On the theory that if you use a lot of Chinese/Japanese characters it will actually come out to less space.
Seriously, just use UTF-8

2) Correct.

3&4) The only rule is that the mimetype file must be first, and uncompressed.
You can use e.g. advzip after-the-fact to recompress it with zopfli for space-saving -- advzip will NOT try to compress the mimetype, since its a) too small, and b) it doesn't recompress file that were stored without ANY compression.
Whatever other program you heard of probably did more or less the same thing, but with a particular affiliation of brand loyalty.

5) calibre's editor. Or else Sigil, but I am more fond of calibre. They do all the paperwork for you.
Failing that, it's simple with zip.exe -- the appropriate incantations are:

# Add mimetype without compression.
zip.exe -X0 C:\path\to\ebook.epub mimetype
# Add all files other than mimetype and platform-specific crap,
# using zip's maximum compression settings.
zip.exe -rDX9 C:\path\to\ebook.epub * -x "thumbs.db" -x mimetype

[ProDigit]

Thank you for the response.
I think the program was called FB2EPUB, but I'm not sure.

I'm going at a fast pace over the creation of an epub file, since I've made them in the past for Sony books, so now and then I need to put some mental notes down here, that perhaps will help fellow ebook writers in their publications

I'm not fond of Calibre or any other, as I don't want to put any credit to any other program within the book, and neither do I want to have a bloated ebook, with a messed up formatting. I know a lot of time has passed since I last used Calibre, but I didn't find it a very good engine.
I found the best engine is still Notepad++ (aka hand corrected).

The more I read about epub, the more questions I have.

Like:
1- Epub supports jpg, any other?
2- Epub supports .ttf fonts, and I wondered if it does any other?
3- in container.xml there's the option to add: "standalone="no"". What does this mean?
4- Before the rootfile lines in container.xml there are tabs, or spaces, and I wonder if it would pose any problems omitting them to reading devices?
I know they don't for the PC.

.....
 <rootfiles>
    <rootfile .....
 </rootfiles>

All questions I'm sure I'll find an answer for soon!

[eschwartz]

Originally Posted by ProDigit

I'm not fond of Calibre or any other, as I don't want to put any credit to any other program within the book, and neither do I want to have a bloated ebook, with a messed up formatting. I know a lot of time has passed since I last used Calibre, but I didn't find it a very good engine.
I found the best engine is still Notepad++ (aka hand corrected).

I'm not fond of using calibre's converter for ebook creation either.

But I was talking about the Editor.

It is a text editor wrapped around an EPUB unzip-zip functionality.
It also has frills like live HTML preview, smart link renaming (it updates the links in other files), and the ability to run plugins written in python.

Sigil is the same, except it automatically tries to fix parsing errors using gumbo parser (and previously, with HTML Tidy).

Unless Notepad++ supports specific utilities you need for mangling your HTML, I don't see why you should use it. Automatically handling the ZIPping is extremely useful.

And why would you have to put credit for calibre in the book?
Do you currently put credit for Notepad++?

The more I read about epub, the more questions I have.

Like:
1- Epub supports jpg, any other?

Depends on the ereader. I know png is often used, at least on newer devices (some old ones might choke on it? Maybe?)

2- Epub supports .ttf fonts, and I wondered if it does any other?

TTF and OTF.

The EPUB3 spec allows WOFF as well, but find me an EPUB3 E-Ink reader.

3- in container.xml there's the option to add: "standalone="no"". What does this mean?

It is mumbo-jumbo that you only care about if you are an XML processor trying to validate against a DTD.

4- Before the rootfile lines in container.xml there are tabs, or spaces, and I wonder if it would pose any problems omitting them to reading devices?
I know they don't for the PC.

.....
 <rootfiles>
    <rootfile .....
 </rootfiles>

All questions I'm sure I'll find an answer for soon!

It is XML. The spaces don't matter.
whitespace in XML/HTML only matters inside an element, and then it is collapsed down to one space.

[ProDigit]

So far I've seen some inconsistencies between Stylesheets.
Some have it organized in the following files:

- OEBPS\CSS\main.css
or
- OEBPS\Styles\stylesheet.css
or in the main OEBPS dir:
-OEBPS\stylesheet.css

Not sure what to make of it.
I've seen ebooks without stylesheet folders, so I presume a folder is only necessary when applying multiple stylesheet.css files in one book.
I presume for normal books not using a style sheet, or using one for the entire book, the file can be placed in OEBPS.

[ProDigit]

Originally Posted by eschwartz

I'm not fond of using calibre's converter for ebook creation either.

But I was talking about the Editor.

It is a text editor wrapped around an EPUB unzip-zip functionality.
It also has frills like live HTML preview, smart link renaming (it updates the links in other files), and the ability to run plugins written in python.



Depends on the ereader. I know png is often used, at least on newer devices (some old ones might choke on it? Maybe?)



TTF and OTF.

The EPUB3 spec allows WOFF as well, but find me an EPUB3 E-Ink reader.



It is mumbo-jumbo that you only care about if you are an XML processor trying to validate against a DTD.



It is XML. The spaces don't matter.
whitespace in XML/HTML only matters inside an element, and then it is collapsed down to one space.

Thanks for answering some of my questions!
I did not know Calibre now has an editor available! Sounds awesome!
Perhaps some day I'll deepen myself into it.

Right now, I think I've gone over the very basics of an epub (in about 3 hours).
I think I understand the structure, mostly, for basic books, and probably can figure out which files (XML) I can trim of redundant data, and which files need trimming in edit mode.

I saw books have a toc.ncx file in the OEBPS directory.
What benefits this from having a TOC.HTML file in the OEBPS/TEXT directory?

Functionally wise I believe there's little to no difference on a computer based ebook viewer, but I wonder if there are added benefits on using a toc.ncx file system in a modern ebook reader?
I am guessing that toc.ncx is less flexible than a html file in layout?

[eschwartz]

Using folders is a stylistic thing.

Sigil enforces folder structure:
Text/
Styles/
Audio/
Misc/

calibre has a tool to auto-sort them.


It doesn't really matter, just make sure to link them properly. Incidentally, EPUB editors track those links and make sure they are correct.

[eschwartz]

Originally Posted by ProDigit

Thanks for answering some of my questions!
I did not know Calibre now has an editor available! Sounds awesome!
Perhaps some day I'll deepen myself into it.

Sure thing!

I see I was still editing my post when you answered.

I saw books have a toc.ncx file in the OEBPS directory.
What benefits this from having a TOC.HTML file in the OEBPS/TEXT directory?

Functionally wise I believe there's little to no difference on a computer based ebook viewer, but I wonder if there are added benefits on using a toc.ncx file system in a modern ebook reader?
I am guessing that toc.ncx is less flexible than a html file in layout?

the NCX is a semantic ToC, which means the ereader understands it and marks chapters for navigation. (e.g. the Kindle's GoTo menu.)

The toc.html is just a page in the book with hyperlinks to chapters.

Many people use both.

[ProDigit]

Thank you so much for the info!
Very valuable!

Will ebook readers display toc.ncx as a page on screen, or will it only appear in navigation links?


A tough question:
Suppose one has a trilogy, with 90 chapters, 30 in each book, and one absolutely wants to combine the three into 1 book.
How could I format the toc file, to be able to skip between books, before selecting the right chapter? Is this possible? Or will I have to go through 30 chapters, before I can select the second book?
Or perhaps it depends from device to device?

I think I'm almost ready to start creating my first EPuB!
Preps are done, and now the real work begins ^_^

[eschwartz]

Well, you can create a two-level ToC.
The Book-level nodes will display with the chapter-level nodes collapsed.

The NCX usually shows in a special navigation sidebar/popup/dropdown.


This is often seen in omnibuses. Like the one you are asking about.
Nothing wrong with omnibuses. I personally prefer splitting them with EpubSplit (calibre plugin). But some people like squashing them together with EpubMerge!

[Notjohn]

I'm not clear whether your object is to publish the book, but if that is the case, be aware that the vast majority of sales are on Amazon, and that Amazon "requires" both an html (the actual) TOC page and the NCX (virtual) TOC. I say "requires" because books are routinely sold that lack one or the other, but that's the stated policy.

And, for Amazon, I rely entirely on Sigil. The gen on the KDP forums is that Calibre-generated books are sometimes rejected. This complaint is less common now than formerly, but it still crops up from time to time.

I love Calibre for one personal project: my university library stocks increasing numbers of titles in digital format that can be downloaded either as PDF files or as epubs for reading on ADE. I take the epub and convert it to mobi for reading on my Fire tablet. But I don't like the styles that it creates; Sigil does a much finer job of employing my own style sheet.

But I am really a doofus at all this, so FWIW.

[Toxaris]

You might want to check out the tool called ePUBPack. You then just would need to tell where your mimetype is, and it will create the ePUB for you.
Personally I prefer the Sigil editor much more than the Calibre editor, but YMMV. Although there is beauty in handcraft, that is how I started as well, the automatic creation of the opf and ncx are a very useful part of the editors. As you can open and tinker with those files as well in the editors, you can still learn a lot.

You can also use GIF, PNG and SVG. All readers *should* support that, but be aware that not all the options of SVG are supported, nor are animated GIF.

[ProDigit]

Originally Posted by Toxaris

You might want to check out the tool called ePUBPack. You then just would need to tell where your mimetype is, and it will create the ePUB for you.
Personally I prefer the Sigil editor much more than the Calibre editor, but YMMV. Although there is beauty in handcraft, that is how I started as well, the automatic creation of the opf and ncx are a very useful part of the editors. As you can open and tinker with those files as well in the editors, you can still learn a lot.

You can also use GIF, PNG and SVG. All readers *should* support that, but be aware that not all the options of SVG are supported, nor are animated GIF.

Yes, I presume that would be a good way to go; to manually (or hand-)prune the file to the bare basics.

Although generating some of the files is pretty easy, as most of the data is just copy/paste, and modify a few parameters.

My only concern at the moment would be the compression of the text.
I like a very good compression agent to compress the file very small, yet also not compress the mimetype.
And that's a problem with most ZIP/RAR/7Z compressors as they would also compress this file.

[eschwartz]

RAR/7Z is irrelevant, since they compress with an incompatible compression algorithm.

zip compressors need to be played with until they put the mimetype first, in addition to not doing something stupid and compressing it (which should actually lead to it becoming larger).

...

Didn't I mention advzip? It can recompress zipfiles with the zopfli algorithm, and it is confirmed to be safe to use on existing valid EPUBs.

[ProDigit]

Yeah, RAR or 7Z also compress in .zip formats.
I've read about 5 years ago or so, that this one guy optimized the dos based zip.exe compression program to better compress text (for ebooks).
I'm still trying to figure out who that guy was, and what the name of the program was. It also left mimetype uncompressed in the archive, but supposedly his program compressed the text by a lot better than a standard zip program; and was very easy to use in DOS, as it was a one command compression program.

THere's a lot that comes into place, making an epub.(learn HTML, XML, EPuB format, and compression formats), but it sure is interesting!