New mobi format won't sync reading progress on Amazon

[mkgtu]

I always converted to mobi using the "mobi output" option "both" formats. Then uploaded to Amazon. No problem. Of course these books on Kindle can't use the Kindle bold font adjustment like purchased books.

Then I tried converting using only the "new" mobi format. Uploaded. And was pleased to find that these books CAN use the Kindle boldness feature, same as purchased books.

So I reconverted and uploaded a bunch of books in the "new" format, only to find that none of them could sync reading progress between Kindles! I then converted them again using the "both" formats setting, and they all now sync furthest page again.

Must be something missing in the conversion to the "new" mobi format that causes them not to be able to sync reading progress. Would be nice to be able to take advantage of the Kindle boldness options and still sync reading progress.

(And, no, I don't have the Facebook share thing turned on — which I know disables Whispersync.)

Sent from my SM-G975U using Tapatalk

[mkgtu]

Follow up to my first post.

I also tried converting to .azw3. Can't upload azw3 to Kindle, so I renamed the extension to azw before uploading. That worked.

The result on my two Kindle devices is that this format can take advantage of the new boldness adjustments, but unfortunately, just like with the new mobi format, the books will not sync reading progress between devices.

Sent from my SM-G975U using Tapatalk

[jhowell]

There is a trick that might help you. If you start with or convert your book to EPUB format then you can change the extension of the file from .epub to .jpg and email it to your account. Amazon will then convert the book to the appropriate Kindle format to your devices. Doing that may work better for font bolding and synchronization.

[JSWolf]

There is no new Mobi format. There's only Mobi and dual format Mobi. Neither whcih are new.

[jhowell]

Originally Posted by JSWolf

There is no new Mobi format. There's only Mobi and dual format Mobi. Neither whcih are new.

That is the terminology that the MOBI Output conversion plugin uses for "MOBI file type":

"old" - Original MOBI format

"new" - Newer KF8 (aka azw3) format - introduced in 2011, provides more features and richer formatting

"both" - Original MOBI and KF8 combined into a single file - This is similar to but not the same as a Master MOBI produced by the Kindle Previewer

[JSWolf]

Originally Posted by jhowell

That is the terminology that the MOBI Output conversion plugin uses for "MOBI file type":

"old" - Original MOBI format

"new" - Newer KF8 (aka azw3) format - introduced in 2011, provides more features and richer formatting

"both" - Original MOBI and KF8 combined into a single file - This is similar to but not the same as a Master MOBI produced by the Kindle Previewer

But isn't that a bit misleading as KF8 is not Mobi so it's not a new Mobi format? Even if KF8 was once a new Mobi format, it's no longer new. KFX can be called new (but not improved) for now.

[mkgtu]

Thanks.
I'd used that trick in the past, (epubs sent as jpg or png) but it hasn't worked for me in a long time. They just get rejected, whether I put "convert" in the subject line. Seems Amazon is no longer fooled.
Tried it again today. No luck. Rejected.

What does work is converting to .txt or .docx. Tried both and uploaded. Both results can use the boldness adjustments and they sync reading progress.

But one big disadvantage of both formats is that you lose a functioning table of contents. The txt format loses all TOC links — there is a textual TOC, but they don't work as links. The docx has 2 TOC's, one of which has working links. But you'd have to bookmark that page because neither docx nor txt has a TOC that works with Kindle's "Go to" feature. In other words, there is a "content" TOC, but they lose the "metadata" TOC from the original ebook format.

(A working TOC is important to me because I frequently alternate between the printed version and an audiobook. No TOC and I easily get lost!)

And if you have any images or fancy formatting in the original, I think you'd lose all that with a txt format. You might keep most of it with docx, but I won't swear to that.

Bottom line, it just would be nice if the conversion to the new mobi format could be made compatible with some of the newer Kindle features, like fuller font formatting (which it is) and yet not lose the synchronization feature in the process.

Sent from my SM-G975U using Tapatalk

[mkgtu]

Originally Posted by JSWolf

But isn't that a bit misleading as KF8 is not Mobi so it's not a new Mobi format? Even if KF8 was once a new Mobi format, it's no longer new. KFX can be called new (but not improved) for now.

May be misleading, but that's the terminology used in calibre. I think of "new" as KF8 disguised as mobi. After all there is no option to convert to KF8, only KFX. And you can't email or use "send to Kindle" to upload either of those file types to the Amazon cloud. So the KF8 has to look like a mobi.

My point is simply that an uploaded KF8 or "new" mobi won't sync reading positions, but it will be able to use some features that the older mobi format can't.

I've also submitted this as a bug report to calibre. Will see what happens.

Sent from my SM-G975U using Tapatalk

[mkgtu]

Just got a response to my calibre bug report on this issue from Kovid Goyal:

"Yes, it would be nice if it worked. Sadly the reason it doesnt work is
Amazon is trying to nudge people into their walled garden so they
gradually try to remove features from side-loaded books in newer
formats, to try to gradually disincentivise people from maintaining their own e-book collections."

So I guess it is what it is for the foreseeable future ☹️



Sent from my SM-G975U using Tapatalk

[JSWolf]

Originally Posted by mkgtu

May be misleading, but that's the terminology used in calibre. I think of "new" as KF8 disguised as mobi. After all there is no option to convert to KF8, only KFX. And you can't email or use "send to Kindle" to upload either of those file types to the Amazon cloud. So the KF8 has to look like a mobi.

My point is simply that an uploaded KF8 or "new" mobi won't sync reading positions, but it will be able to use some features that the older mobi format can't.

I've also submitted this as a bug report to calibre. Will see what happens.

Sent from my SM-G975U using Tapatalk

There is an option to convert to KF8. Calibre calls it AZW3 which technically is incorrect. As for KF8 not syncing, that's down to Amazon not caring about side loaded content. They put it in place before KF8 and do not want to update. Amazon only really cares about KFX since that's the format they are pushing because they can get away with DRM even when the publisher doesn't want it and they can change the DRM once it's cracked.

Don't expect Amazon do do anything logical. It won't happen.

[jhowell]

Originally Posted by JSWolf

But isn't that a bit misleading as KF8 is not Mobi so it's not a new Mobi format? Even if KF8 was once a new Mobi format, it's no longer new. KFX can be called new (but not improved) for now.

I agree that naming something "new" or "old" is a bad choice in the long run. However, at ths point those names have been in use for ten years. It would just cause more confusion to change them now.

Originally Posted by mkgtu

I'd used that trick in the past, (epubs sent as jpg or png) but it hasn't worked for me in a long time. They just get rejected, whether I put "convert" in the subject line. Seems Amazon is no longer fooled.
Tried it again today. No luck. Rejected.

That is odd. I changed the extension of this EPUB to .jpg and emailed it to my Kindle with no subject and it worked for me just yesterday. Perhaps you could try the same file and see what happens.

Originally Posted by mkgtu

Bottom line, it just would be nice if the conversion to the new mobi format could be made compatible with some of the newer Kindle features, like fuller font formatting (which it is) and yet not lose the synchronization feature in the process.

Amazon's personal document infrastructure is built to support Master-MOBI files created using Amazon's tools, kindlegen and Kindle Previewer. As far as I can tell if those are used the resulting book will support both boldness and Whispersync.

MOBI-new files created by calibre contain only a subset of the Master-MOBI format so it is not totally surprising to me that some features do not work properly when that is used.

[mkgtu]

Originally Posted by jhowell

That is odd. I changed the extension of this EPUB to .jpg and emailed it to my Kindle with no subject and it worked for me just yesterday. Perhaps you could try the same file and see what happens.

I tried your Frankenstein file and it worked. I tried my own file — the one I'd been experimenting with all along — and it failed again and again, no matter which method (email or Send to Kindle). Must be something odd in the file, though it works fine uploaded in its "undisguised" mobi state.

So I tried the jpg trick with several other epub files and they worked — also with boldness and sync working. I tried emailing and using the "Send to Kindle" app on PC. Both worked, but emailing loses the author name and used my email address as author. But since the author name is also in the title of the filename, at least that shows in the title on a Kindle. But then you can't really sort "by author".

Uploading the "jpg/epub" using the Send to Kindle app not only works, but you can add the author name manually so that it shows up properly in the author column on your Kindle. Of course, that means you can't bulk upload a bunch of books with different suthors!

Since I convert and upload a lot of books, I'm not sure all this hassle is worth the trouble just to get the boldness feature. My eyes are old (76yrs) so I almost need the bolder font. But I can always use the Amazon Ember Bold font; it's just that that one gets boring after a while. So it's good to know there are alternatives, awkward as they may be.

Sent from my SM-G975U using Tapatalk

[mkgtu]

FYI
The jpg trick doesn't seem to work if the book in question was originally an Amazon book, ie purchased from Amazon or borrowed as a Kindle book through public library via Overdrive.

If the (now) drm-free azw3 (thank you Messrs Alf and Harper) is converted directly to epub in calibre, then uploaded to Amazon for Kindle, I've found it fails to be converted and delivered..

But if I first convert the azw3 to mobi, then convert the mobi to epub, then the epub can be successfully renamed as a jpeg and sent to Kindle.

So, bottom line, if the book was originally in Amazon's clutches, it needs a double laundering. Thank you, Jeff Bezos!

Epubs that never belonged to Mr Jeff seem to go through as jpeg's just fine on the first try.

Sent from my SM-G975U using Tapatalk

[JSWolf]

You may have to edit the OPF to remove any traces of Amazon. Do that and try again.

[jhowell]

Originally Posted by mkgtu

FThe jpg trick doesn't seem to work if the book in question was originally an Amazon book, ie purchased from Amazon or borrowed as a Kindle book through public library via Overdrive.
...
So, bottom line, if the book was originally in Amazon's clutches, it needs a double laundering. Thank you, Jeff Bezos!

Originally Posted by JSWolf

You may have to edit the OPF to remove any traces of Amazon. Do that and try again.

The issue is not caused by Amazon recognizing that the book originally came from them. It is a consequence of the conversion from AZW3 to EPUB done using calibre.

When you email a file with an image extension it is processed by Amazon using kindlegen. That program recognizes that the format is actually EPUB and processes it accordingly.

However kindlegen is designed to convert books as part of the publishing process. It often intentionally rejects books with content that fails to meet Amazon's publishing guidelines. That makes sense in a publishing environment where the publisher is expected to deliver a properly formatted book and needs to know if that is not the case.

When calibre converts AZW3 to EPUB it tends to create links to HTML ids that are placed on body elements in the XHTML files. Those links, while correct according to the EPUB specification, are rejected by kindlegen if they occur in the table of contents.

One workaround for this is to use the Kindleunpack plugin to convert AZW3 to EPUB instead of using calibre's Convert Books function. That produces an EPUB that better reflects the original content of the AZW3 file and is more likely to be acceptable to kindlegen.