Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Conversion

Notices

Reply
 
Thread Tools Search this Thread
Old 03-05-2012, 03:04 PM   #1
LaurelRusswurm
self publishing novelist
LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.LaurelRusswurm knows the way to San Jose.
 
LaurelRusswurm's Avatar
 
Posts: 30
Karma: 54620
Join Date: Oct 2011
Location: dual citizenship in the interwebz and canada
Device: none
Does Calibre support the new kindle format?

I'm a self publishing author trying to publish via Amazon; situation complicated because I use free software (Ubuntu and Trisquel) which Amazon doesn't much seem to support. Because I want control and to to understand what I'm doing, I formatted my book in Sigil, including font selections as an ePub. It displays exactly as wanted in the Calibre reader. Yay.

Except it doesn't in Amazon's "simple" online previewer. Fonts and css seems to have disappeared. Amazon also has an advanced viewer for Mac and Windows, so maybe it will display correctly there, but I can't tell, because it isn't fot linux.

Amazon allows me to download a converted mobi file which I tried to view in the Calibre reader but it won't open at all, I get this message:
-------------------------------------------------------------------
ERROR: Could Not Open ebook
X All strings must be XML compatible: Unicode pr ASCII, no NULL bytes or control characters
-------------------------------------------------------------------
That is way over my head.

For all I know it will display properly, but I have no way to tell.

Apparently Kindlegen does have a linux version now, so I will try that, but I would prefer to go direct from Sigil or Calibre than to have to reformat in yet another (proprietary) program. I gather Mobi does not support embedded fonts but the new KF8 is supposed to, along with CSS.

Is it possible to use Calibre to convert my Sigil built ePub directly to the new KF8 (?) format that Amazon will accept?
LaurelRusswurm is offline   Reply With Quote
Old 03-06-2012, 08:41 AM   #2
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,442
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
To my knowledge... calibre doesn't yet have a way of dealing with the DRM-free versions of the stand-alone KF8 ebook files that are starting to find their way into many people's libraries (and will only be steadily increasing). The viewer won't handle them properly and they can't be converted.

Nor can calibre convert anything to the KF8 format, yet (that I know of).

I imagine that will eventually change, though.
DiapDealer is offline   Reply With Quote
Advert
Old 03-06-2012, 07:52 PM   #3
wallcraft
reader
wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.
 
wallcraft's Avatar
 
Posts: 6,975
Karma: 5183568
Join Date: Mar 2006
Location: Mississippi, USA
Device: Kindle 3, Kobo Glo HD
The KF8 format is currently only used on the Kindle Fire and (perhaps, reported to work - but undocumented) on Kindle for PC. In the long term it is going to be the best Kindle alternative to ePub, but for now for the majority of Kindle readers the original MOBI format is what you need.

MOBI does not include embedded fonts and has limited CSS support (there is no CSS in the MOBI, but some ePub CSS wil get converted to the equivalent MOBI formatting). So the "simple" online previewer may be accurately showing you the MOBI version.

If I was producing a Kindle ebook today via Linux, I would use Calibre to produce a MOBI from the ePub. If the MOBI looks good in the Calibre viewer it will probably be ok on a Kindle (and in the "simple" online previewer). A "good" MOBI should have a cover and a metadata-based TOC and maybe some "frills" for chapter headings (although this are more limited for MOBI than for ePub). Italics should be maintained from the ePub, but this assumes you used HTML to get the italics (using an embedded italics-only font isn't good practice in an ePub and won't convert to a MOBI).
wallcraft is offline   Reply With Quote
Old 03-06-2012, 08:37 PM   #4
dwig
Wizard
dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.
 
dwig's Avatar
 
Posts: 1,613
Karma: 6718479
Join Date: Dec 2004
Location: Paradise (Key West, FL)
Device: Current:Surface Go & Kindle 3 - Retired: DellV8p, Clie UX50, ...
Given the massive installed base of Kindles that do not, and never will, support KF8 it is important that however you design a book it should look good and function well on non-KF8 devices and software. It will be quite some time before it's safe to ignore these non-KF8 devices.

KF8 files generally contain two versions of the formatted text, one for KF8 compliant readers and one for non-KF8 compliant readers. Both versions need to be tested.

Embedded fonts should be used only for special text (headings, titlepage, ...) and not for the main body copy. Even then, the book needs to still look good when the non-KF8 version is displayed with its fall back to the device's/software's own fonts. The same applies to ePub versions. Not all ePub readers support embedded fonts.
dwig is offline   Reply With Quote
Old 03-06-2012, 09:34 PM   #5
michaelsmith1983
Junior Member
michaelsmith1983 has a complete set of Star Wars action figures.michaelsmith1983 has a complete set of Star Wars action figures.michaelsmith1983 has a complete set of Star Wars action figures.
 
Posts: 4
Karma: 288
Join Date: Mar 2012
Device: None
start with a html file and convert that to .mobi rather than epub to.mobi if there is a problem with that. open file in openoffice and save as html
michaelsmith1983 is offline   Reply With Quote
Advert
Old 03-06-2012, 09:37 PM   #6
michaelsmith1983
Junior Member
michaelsmith1983 has a complete set of Star Wars action figures.michaelsmith1983 has a complete set of Star Wars action figures.michaelsmith1983 has a complete set of Star Wars action figures.
 
Posts: 4
Karma: 288
Join Date: Mar 2012
Device: None
why not open your text in open office and save as html, and convert that to .mobi? you seem to be having difficulty converting epub to .mobi
michaelsmith1983 is offline   Reply With Quote
Old 03-07-2012, 09:07 AM   #7
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,442
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
The KF8 format is currently only used on the Kindle Fire and (perhaps, reported to work - but undocumented) on Kindle for PC.
It's not only purported to work on Kindle for PC and Kindle for Mac (>= v 1.9.0), it does work. And the prompt that comes up asking you if want to upgrade to the latest version says as much.

Re-downloading my past purchases from my Kindle for PC archive reveals many stand-alone KF8 books. The problem is... they look like a MOBI, smell like a MOBI and act like a MOBI (because they're still in a prc container). So unless people know enough to dump the headers of the files and actually check, there's just no way to know for sure.

Which means that a very common workflow is going to result in DRM-free MOBI's being imported into calibre libraries that are, in fact, stand-alone KF8's that calibre can't view properly and can't convert to anything else.

It's going to result in some confusion for a while.

Quote:
Originally Posted by dwig
KF8 files generally contain two versions of the formatted text, one for KF8 compliant readers and one for non-KF8 compliant readers. Both versions need to be tested.
That's only the version that you upload to Amazon. Amazon is splitting the MOBI-only portion from the KF8 portion and delivering the appropriate file to the various devices/apps depending on what they can support. The KF8 ebooks that are being delivered to my Kindle for PC have no fall-back format. They simply can't be viewed at all on the eInk devices (I'm talking about DRM-free versions, of course).

I agree that MOBI is still going to be the more common format for quite some time, but KF8 is no longer a "red-headed stepchild." It's out there... people just don't know it yet.

Last edited by DiapDealer; 03-07-2012 at 09:19 AM.
DiapDealer is offline   Reply With Quote
Old 03-07-2012, 12:34 PM   #8
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
could we create a conversion plugin

Hi,

Looking at the calibre source archive, I see there are conversion plugins much like there are file type plugins. Given the Mobi_Unpack code can convert a kf8 ebook to its original epub, would it be possible to create a new conversion plugin that runs at higher priority than the current mobi_input.py that does the following:

1. works on all mobi/azw ebooks
2. check to see if KF8 (standalone or dual) or just original mobi
3. if just original passes it to the current mobi_input.py plugin
4. if Kf8 (assume the user would prefer the kf8 version since it supports fonts and things and then use the code in Mobi_Unpack to convert the mobi KF8 format piece to its original epub
5. then call the epub_input.py plugin

I looked and one of the input plugins can call another input plugin (this is done in htmlz which invokes the html plugin) so unless I am misundertanding things, this should be possible.

So when people load an original mobi ebook, they get a .mobi, but if they load a KF8 mobi ebook, they effectively get an epub. This would work well for viewing a KF8 inside Calibre, and for converting it (already is an epub so calibre can convert it to whatever format including the original mobi).

The only drawback is that there would be no way to generate KF8 output since creating it takes much more work (and reverse engineering) than simply reading it. But I think something like this would work to get KF8 support (at least input and conversion) into calibre in a very short time window.

For the right person (DiapDealer! ...;-) , this might take only a day or two to create.

My 2 cents ...

KevinH
KevinH is offline   Reply With Quote
Old 03-07-2012, 05:05 PM   #9
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
proof of concept

BTW:

I am working on proof of concept where we replace mobi_input.py with a new version that tries to load classes from the mobi_unpack_v042_plugin.zip and if it succeeds it invokes the epub plugin and if not it does what it used to do.

This requires a new interface class within mobi_unpack.py to work well with calibre approach to using "streams" but it should work.

I will debug it and see if I can get it working and if so, someone can decide if we want to include it in the next release or not as a fill in until actual support becomes available.

KevinH
KevinH is offline   Reply With Quote
Old 03-07-2012, 08:52 PM   #10
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
Hi All,

It is working. With a modified mobi_unpack plugin and a change mobi_input.py, I now have KF8 ebooks showing up and converting to other formats properly in Calibre.

Of course, I still can't create a KF8 ebook with Calibre but we can with KindleGen V2.

If anyone wants these changes, I would be happy to provide them.

Kevin
KevinH is offline   Reply With Quote
Old 03-07-2012, 10:13 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,748
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@KevinH: Sure send me your code and I'll merge it. It can form a good starting point for KF8 input support.
kovidgoyal is offline   Reply With Quote
Old 03-07-2012, 10:19 PM   #12
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
Hi Kovid,

Sent it to user_none but I will e-mail you the revised mobi_input.py and the fixed mobi_unpack plugin asap.

Thanks,

KevinH


Quote:
Originally Posted by kovidgoyal View Post
@KevinH: Sure send me your code and I'll merge it. It can form a good starting point for KF8 input support.
KevinH is offline   Reply With Quote
Old 03-08-2012, 09:28 AM   #13
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 324
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
Quote:
Originally Posted by kovidgoyal View Post
@KevinH: Sure send me your code and I'll merge it. It can form a good starting point for KF8 input support.
Calibre could use kindlegen to do epub->kf8 conversion. I think if the calibre installer downloaded kindlegen from amazon instead of redistributed it there wouldn't be an issue with the license. Then calbire would have a complete KF8 solution.
nickredding is offline   Reply With Quote
Old 03-08-2012, 09:39 AM   #14
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,662
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by nickredding View Post
Calibre could use kindlegen to do epub->kf8 conversion. I think if the calibre installer downloaded kindlegen from amazon instead of redistributed it there wouldn't be an issue with the license. Then calbire would have a complete KF8 solution.
Kindlegen is not open source software (or we would not be having this discussion on how 2 do KF8 from Calibre )

Will/can the Open With PI call Kindlegen?
theducks is online now   Reply With Quote
Old 03-08-2012, 12:35 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,748
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I am never going to use a binary blob like kindlegen for conversion:

1. It only supports 3 platforms, calibre runs on BSDs and ARM as well
2. I have no way of knowing exactly what kindlegen does
3. I have no way of fixing bugs in kindlegen

Producing KF8 doesn't seem to be insuperably difficult, I'll get to it eventually.
kovidgoyal is offline   Reply With Quote
Reply

Tags
amazon, calibre, fonts, kindle, sigil

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Does Calibre support the Kindle FIRE? TomTeaPack Devices 3 11-15-2011 09:02 PM
Why People developed books in mobi or prc format, If Kindle support PDF book tushar Kindle Formats 4 08-11-2011 03:50 AM
Calibre support of new Kindle 3 chilady1 Calibre 7 01-15-2011 04:36 PM
Kindle support in calibre user_none Amazon Kindle 85 09-20-2009 05:26 PM
Does Calibre Support Kindle DX? Sonist Calibre 16 06-20-2009 07:59 PM


All times are GMT -4. The time now is 07:17 AM.


MobileRead.com is a privately owned, operated and funded community.