A few questions after being gone a few years - (formats for kindle/calibre library)

[Blakestr]

I have a 32 gb Kindle Oasis 2017 version and I'm in the process of consolidating my library. I'm a bit of an archivist so I want to have a well organized library and then just leave it alone. (I did that like three years ago but now need to update it)

1. Do I need to keep a epub/mobi copy of every file, if I mainly only read on the Kindle? I use PC and android, but I'm assuming both devices would at least have KFX reader, so there's less reasons to maintain multiple formats for the same book than there have been in the past? Last time I just kept them out of fear of the AZW not converting back, and I was forced to filter calibre for only AZW before sending them to my Kindle.

2. How backwards compatible is the new (new to me) KFX format? Can I go back and forth between files (excluding the problamatic PDF's, which I will probably handle differently, feel free to recommend an updated solution for PDF's layouts, 3 years ago my understanding was simply, "conversion is generally crappy"

3. Has indexing on the kindle hardware changed? Last it was one of those things that can take days for thousands of books, requiring you to keep the device plugged in and in some cases, manually refresh it to speed up the process.

[jhowell]

Originally Posted by Blakestr

1. Do I need to keep a epub/mobi copy of every file, if I mainly only read on the Kindle?

I consider the Kindle formats to be proprietary reader formats. Along with your Kindle output format of choice I suggest archiving a more open publishing format, such as EPUB, for future compatibility.

Originally Posted by Blakestr

I use PC and android, but I'm assuming both devices would at least have KFX reader, so there's less reasons to maintain multiple formats for the same book than there have been in the past? Last time I just kept them out of fear of the AZW not converting back, and I was forced to filter calibre for only AZW before sending them to my Kindle.

KFX is supported on Kindle apps and devices released within the past few years. It is not supported as a format for personal documents that sync across devices associated with your Amazon account.

Originally Posted by Blakestr

2. How backwards compatible is the new (new to me) KFX format? Can I go back and forth between files (excluding the problamatic PDF's, which I will probably handle differently, feel free to recommend an updated solution for PDF's layouts, 3 years ago my understanding was simply, "conversion is generally crappy"

KFX has limited compatibility. Converting either from or to KFX is a lossy process. The conversion process applies "adjustments" to the formatting of the book in an attempt to improve readability. The HTML markup of the original source is also lost because unlike most other e-book formats, KFX is not based on HTML. Converting to KFX from other formats may not always be possible due to the limitations of the KFX format.

KFX enables the Enhanced Typesetting features of the Kindle platform. If compatibility is more important then I would recommend KF8 (azw3) as a more compatible alternative since it is based on HTML.

Originally Posted by Blakestr

3. Has indexing on the kindle hardware changed? Last it was one of those things that can take days for thousands of books, requiring you to keep the device plugged in and in some cases, manually refresh it to speed up the process.

Still the same. Transferring smaller batches is recommended because indexing can fail for some books stopping the process and when that happens finding the book that caused it can be problematic.

[Blakestr]

Originally Posted by jhowell

I consider the Kindle formats to be proprietary reader formats. Along with your Kindle output format of choice I suggest archiving a more open publishing format, such as EPUB, for future compatibility.

I'm fine with keeping a separate batch and just filtering by KFX and transferring that whole lot to my Kindle, some of those typeset features seem appealing. (I'm assuming Calibre's previewer can help me see gross, obvious conversion issues.

Transferring smaller batches is recommended because indexing can fail for some books stopping the process and when that happens finding the book that caused it can be problematic.

How small is small? If I have a library of 15k titles, I know the process is going to take time. Hopefully moving from a Voyage to 2nd gen Oasis will help with this. Thanks again for your advice

[jhowell]

Originally Posted by Blakestr

How small is small? If I have a library of 15k titles, I know the process is going to take time.

I would have recommended a hundred at a time, but with that many books that isn’t practical. So I don’t know.

[Tarana]

I'm converting everything to azw3. Novels convert seamlessly between mobi, epub and azw3 using Calibre. AZW3 has formatting that I prefer to use with my newer ereaders (Paperwhite 3, Voyage) and the format works fine on the older ones (Kindle Keyboard, Touch) even though those font features are usable there. I haven't sufficiently tested ebooks with a lot of links and illustrations to make any claims. I do know I've had problems with some cookbooks, but that is the fault of the original publisher, not Calibre, and I returned those books for a refund rather than keep them.

[Blakestr]

Originally Posted by Tarana

I'm converting everything to azw3. Novels convert seamlessly between mobi, epub and azw3 using Calibre. AZW3 has formatting that I prefer to use with my newer ereaders (Paperwhite 3, Voyage) and the format works fine on the older ones (Kindle Keyboard, Touch) even though those font features are usable there. I haven't sufficiently tested ebooks with a lot of links and illustrations to make any claims. I do know I've had problems with some cookbooks, but that is the fault of the original publisher, not Calibre, and I returned those books for a refund rather than keep them.

What about KFX? Doesn't it let you take advantage of the different fonts, again it's been awhile but I know I have some books whose font can't be changed. But now that I'm thinking about it, every clip on YouTube that mentions KFX format only mentions it in the context of DRM removal, not features like fonts changing.

Does azw3 format allow enhanced typesetting features?

[jhowell]

Originally Posted by Blakestr

What about KFX? Doesn't it let you take advantage of the different fonts...

Yes, both formats support custom fonts and adjustable boldness.

One slight advantage of KF8/azw3 over KFX is that the font-ramp hack can be used with it on e-ink Kindles to give finer control of font sizes. That does not work with KFX.

Originally Posted by Blakestr

Does azw3 format allow enhanced typesetting features?

No, except that you can get it to do hyphenation if you add soft hyphens to each file using the Hyphenate This! plugin in calibre.

Most of the Enhanced Typesetting features are subtle. Not everyone likes them or feels that they are necessary.

[Blakestr]

Originally Posted by jhowell

Yes, both formats support custom fonts and adjustable boldness.

Most of the Enhanced Typesetting features are subtle. Not everyone likes them or feels that they are necessary.

So the one book I'm currently reading doesn't let me change the font, only size IIRC. I'm assuming that it just has an embedded font type and is probably mobi or epub?

Like you said, I've read paper books and never thought, "boy, I wish I could change the font" so I'm most likely going to stick with the awz3 format for most, maybe keep the originals in an archive folder before converting.

It also seems nothing is changed much in terms is tags displayed in Kindle compared to Calibre, I'm going to look into making a TOC book that has all the metadata then use that to search what is on the Kindle, as I generally like to put everything on the Kindle once and then only user Calibre when I'm adding something new.

[ps67]

Originally Posted by Blakestr

So the one book I'm currently reading doesn't let me change the font, only size IIRC. I'm assuming that it just has an embedded font type and is probably mobi or epub?

Like you said, I've read paper books and never thought, "boy, I wish I could change the font" so I'm most likely going to stick with the awz3 format for most, maybe keep the originals in an archive folder before converting.

It also seems nothing is changed much in terms is tags displayed in Kindle compared to Calibre, I'm going to look into making a TOC book that has all the metadata then use that to search what is on the Kindle, as I generally like to put everything on the Kindle once and then only user Calibre when I'm adding something new.

Are You reading a book with some kind of fixed layout? Because otherwise You can always change the font either in moby, azw3 or kfx. The only difference is that moby - at least the old moby format, I did not check the new format - doesn't support embedded font (useful if You want to read using the font embedded by the publisher, which is often terrible) nor custom font that You installed on Your Kindle.

About the best format: for reading in my Kindle I prefer kfx because of Enhanced Typesetting features, but in calibre I prefer to have also the original epub or azw3 because, as said by @jhowell, kfx is a lossy format. But I have no space problems and I don't have 15.000 books.

[Blakestr]

Originally Posted by ps67

About the best format: for reading in my Kindle I prefer kfx because of Enhanced Typesetting features, but in calibre I prefer to have also the original epub or azw3 because, as said by @jhowell, kfx is a lossy format. But I have no space problems and I don't have 15.000 books.

Interesting. I guess the conversion process to KFX works better than it has in the past. Can I get away with doing a bulk conversion of most of them are simple epubs with few or no layout elements like tables or images? I'm fine with treated as a one-way conversion and just putting it on my Kindle.

[hobnail]

Originally Posted by Blakestr

What about KFX? Doesn't it let you take advantage of the different fonts, again it's been awhile but I know I have some books whose font can't be changed.

I can't remember if this was on the Kindle or the Kobo but I've developed this voodoo where I delete all font-family lines in the book's CSS when it's for the global font (i.e., not for monospace for things like typewritten letters). Like you I'm never bothered by whatever font is used in a printed book (but then I've noticed that fiction always uses some reasonable serif font). But with eink ereaders I always want the Cacaelia or Bookerly font.

I can't remember if they specify a specific font family, as in

body {
  font-family: Arial, Helvetica, sans-serif;
}

or just

body {
  font-family: sans-serif;
}

but in either or both cases I'm then not allowed to specify Cacaelia as the font I want to use in the ereader's settings.

I can't understand why people feel the need to set a font for the body/paragraph text; leave it up to the user.

[ps67]

An increasing number of books can be converted in kfx, if there are no particular thing conversion will work, also with images and tables, there was a limit in tables row number but I don't know now if it still exists.

About bulk conversion I don't know how fast (how slow?) it could be, it seems a titanic enterprise with 15.000 books, but I am not so experienced, I only converted one book or two at a time.

@jhowell is the author of this plugin and his advice is more useful than mine.

[ps67]

Originally Posted by hobnail

I can't remember if this was on the Kindle or the Kobo but I've developed this voodoo where I delete all font-family lines in the book's CSS when it's for the global font (i.e., not for monospace for things like typewritten letters). Like you I'm never bothered by whatever font is used in a printed book (but then I've noticed that fiction always uses some reasonable serif font). But with eink ereaders I always want the Cacaelia or Bookerly font.

I can't remember if they specify a specific font family, as in

body {
  font-family: Arial, Helvetica, sans-serif;
}

or just

body {
  font-family: sans-serif;
}

but in either or both cases I'm then not allowed to specify Cacaelia as the font I want to use in the ereader's settings.

I can't understand why people feel the need to set a font for the body/paragraph text; leave it up to the user.

I don't think that this css will prevent You to change the font in a Kindle with Caceilia or Bookerly, I will try because I never encountered that issue.

Edit: a Kobo is a different animal: if I remember correctly with epub it respect very strictly the css, but I patched it to always use my preferred font.