Firmware Update Kindle firmware 5.16.3

[imanub]

Honestly whether the reading percentage isn't perfect (for example, I'm REALLY at 60%, but the Kindle tells me I'm at 50%, or 65%), doesn't really matter to me. The thing that plays a fundamental role in my daily life is having the NEW label on the cover of the book just sent to my kindle from the SendtoKindle application. Instead, from version 5.16.2, instead of the NEW label, I have the percentage of 0%.

[tomsem]

Calibre v6.26.0, just released, fixes the issues with MTP support for Kindle Scribe.

[sricochet]

After I updated the Kindle Scribe to 5.16.3, and connected to calibre (which unfortunately takes much longer now with a large library), azw3 files no longer display thumbnails of the book covers in Library view.

I've tried:
1. restarting the Kindle
2. converting to azw3 and checking the "Share with Facebook etc"
3. converting the jpg of the cover image into png
4. uploading the book and allowing the kindle to load the book, then reconnecting the kindle to calibre
5. opening the book and giving the kindle time to identify the cover
6. turning off wifi and assigning an undefined publication date.

The books display the cover on the sleep screen.

Edit: it seems that the connection to the Scribe from calibre is slowed down by all the ._ files that the mac produces. If they are removed by dot_clean or other method (such as MobileFileSearch v1.46, getting the list of books on device through calibre from the reader is much quicker.

[jhowell]

Originally Posted by sricochet

After I updated the Kindle Scribe to 5.16.3, and connected to calibre (which unfortunately takes much longer now with a large library), azw3 files no longer display thumbnails of the book covers in Library view.

Calibre contains drivers to handle the specific needs of various e-reader devices. Calibre uses the Kindle driver for most e-ink Kindle devices. However Kindle Scribe devices with the 5.16.3 firmware appear to the OS running calibre as MTP (Media Transfer Protocol) devices, causing them to be accessed using the MTP driver within calibre.

The 6.26 release of calibre has some changes to the MTP driver to ignore some of the extraneous files found in the Scribe that the Kindle driver has always ignored automatically. That makes the Scribe work somewhat better when connected to calibre. But there is still a lot of Kindle-specific handling that will no longer be applied to Scribe devices.

As you found one of the Kindle-specific features that will no longer work for Scribes is providing cover thumbnail images for sideloaded AZW3 files. Others are providing page numbers (APNX files) for sideloaded MOBI and AZW3 files, reading metadata from Amazon purchased books in KFX format, and importing Amazon purchased books in KFX format into calibre from the books-on-Device list.

There are most likely additional side effects of this change that I am not yet aware of.

[tomsem]

Originally Posted by jhowell

Calibre contains drivers to handle the specific needs of various e-reader devices. Calibre uses the Kindle driver for most e-ink Kindle devices. However Kindle Scribe devices with the 5.16.3 firmware appear to the OS running calibre as MTP (Media Transfer Protocol) devices, causing them to be accessed using the MTP driver within calibre.

The 6.26 release of calibre has some changes to the MTP driver to ignore some of the extraneous files found in the Scribe that the Kindle driver has always ignored automatically. That makes the Scribe work somewhat better when connected to calibre. But there is still a lot of Kindle-specific handling that will no longer be applied to Scribe devices.

As you found one of the Kindle-specific features that will no longer work for Scribes is providing cover thumbnail images for sideloaded AZW3 files. Others are providing page numbers (APNX files) for sideloaded MOBI and AZW3 files, reading metadata from Amazon purchased books in KFX format, and importing Amazon purchased books in KFX format into calibre from the books-on-Device list.

There are most likely additional side effects of this change that I am not yet aware of.

It's not clear to me what is impacted and how.

On Scribe I do see a Title for everything, include KFX (it's concocted somehow -has punctuation stripped and ASIN tacked on - so not from metadata). Purchased content also does not match up to Library, whereas personal documents do show expected author/title metadata and match to calibre Library (none of this is side-loaded). For personal document KFX, I see expected Title and Author. I can import purchased KFX as KFX-ZIP file. There are no cover images for anything in on-device list.

On Paperwhite 11th gen (which takes much longer to be detected by calibre), again purchased KFX content doesn't show actual metadata, no match to calibre Library, on-device preview shows no cover images, and the purchased KFX cannot be Added even as KFX-ZIP.

So the only difference is you can FWIW import KFX (to KFX-ZIP) with MTP and cannot using USB Drive.

I side loaded azw3 files to both (transfers faster to Scribe), and for third party content, you no longer get cover thumbnail when transferring with calibre (I always use send to Kindle anyway).

How would I check APNX functionality?

[j.p.s]

Originally Posted by tomsem

How would I check APNX functionality?

Tap the lower left corner of the page to cycle through the reading progress display options. If the apnx file is (properly) present, the page number will be present as one of the displays, if you are on a page contained in the apnx file. I have come across a number of books recently that don't display a page number until page 1 is reached. (i, ii, iii, ..., x, etc not present).

[jhowell]

Originally Posted by tomsem

On Scribe I do see a Title for everything, include KFX (it's concocted somehow -has punctuation stripped and ASIN tacked on - so not from metadata).

Purchased content also does not match up to Library,...

On Paperwhite 11th gen (which takes much longer to be detected by calibre), again purchased KFX content doesn't show actual metadata, no match to calibre Library,

The concocted title is the file name of the book as it appears on the Kindle with punctuation stripped out. Calibre uses that as the book title with author "Unknown" when it cannot read the metadata from a book. The incorrect title and missing author prevents matching to the calibre library.

Two things are going wrong with obtaining metadata for Amazon purchases in KFX format. The first applies to some books on all Kindles. The second applies to all KFX books but only on Scribe devices.

In KFX as used for Amazon purchases the book metadata is stored in a metadata.kfx file separate from the main book contents. That file is sometimes DRM encumbered resulting is missing metadata for some books on all Kindles. (Unfortunately fixing this would require calibre to perform DRM removal before accessing metadata from KFX books.)

But on a Scribe the metadata being in a separate file is always a problem for another reason. The MTP driver only accesses the main file for any book. It does not know about the splitting of KFX format across multiple files so the metadata will always be missing even when there is no DRM involved.

Originally Posted by tomsem

...whereas personal documents do show expected author/title metadata and match to calibre Library (none of this is side-loaded). For personal document KFX, I see expected Title and Author.

Personal documents in KFX format on a Scribe always have the metadata in the main KFX file, which is always DRM-free, allowing access by calibre.

Originally Posted by tomsem

On Scribe...I can import purchased KFX as KFX-ZIP file.

On Paperwhite 11th gen...the purchased KFX cannot be Added even as KFX-ZIP.

So the only difference is you can FWIW import KFX (to KFX-ZIP) with MTP and cannot using USB Drive.

On the Scribe it will import the only the main KFX file of the book into a KFX-ZIP. Because it is missing the additional KFX files holding DRM keys, metadata, and images it is not possible to remove DRM or convert that file. It is useless as is.

On other Kindles calibre blocks import of KFX format by default. It considers KFX to be a "virtual format". There is a tweak that can be set to undo this blockage as described in the documentation for the KFX Input plugin. If that tweak is set and the KFX Input plugin and proper DRM removal plugin are installed then it is possible to import KFX books from non-Scribe Kindles.

[sricochet]

Thank you for your response

Your information was enlightening as to the new state of things with the Kindle Scribe.

Thanks to your information, I was able to get book covers uploaded to the Scribe.

This is my procedure

1. Download KFX Output plugin
1a. Update KFX Output plugin to create azw8 (here: Post #90 https://www.mobileread.com/forums/sh...=272407&page=6 - the plugin has updated in the meantime but most of it is still applicable)
1b. go into "C:\Users\[username]\AppData\Roaming\calibre\mtp_devices.json" (open with Notepad) and add "azw8" to "format map" beneath "azw3"
2. Convert books to azw8 (or KFX), selecting "Create personal document instead of book" option in KFX Output menu in options on left side of window
3. Connect Kindle Scribe to pc
4. Send to Device (checking that azw8 or kfx are selected)
5. enjoy thumbnails in calibre-uploaded books



Thanks again.

Edit: Converting from AZW3 to KFX (or AZW8) results in custom fonts not being retained in KFX. Best to convert AZW3 to EPUB or MOBI first, then convert to KFX to get your own fonts in the book.

Also, I can't remember if it's been said, but the KFX books take a beat or two to show up in the Library list, and usually the book files don't have to be indexed to be viewed in the Library. they will appear in list view almost immediately.

[Karamazov]

I may have missed something but with Calibre 6.26 on MacOs Ventura and my Kindle Scribe with the latest firmware, Calibre is not seeing the device. I am getting a "No MTP devices connected to system" on the debug logs. Did I missed a step or is the MTP fix is for Windows only at this stage? (Cable and stuff tested as device is visible with Android File Transfer). If anyone would have a lead, it is driving me crazy!

[DNSB]

Originally Posted by Karamazov

I may have missed something but with Calibre 6.26 on MacOs Ventura and my Kindle Scribe with the latest firmware, Calibre is not seeing the device. I am getting a "No MTP devices connected to system" on the debug logs. Did I missed a step or is the MTP fix is for Windows only at this stage? (Cable and stuff tested as device is visible with Android File Transfer). If anyone would have a lead, it is driving me crazy!

Linux and MacOS devices can only have one connection to an MTP device. You will need to stop AFT before calibre can see your Scribe.

[Karamazov]

Originally Posted by DNSB

Linux and MacOS devices can only have one connection to an MTP device. You will need to stop AFT before calibre can see your Scribe.

I disabled AFT, restarted the MAC, launch calibre and still getting the same No MFT device detected unfortunately.

[DNSB]

Originally Posted by Karamazov

I disabled AFT, restarted the MAC, launch calibre and still getting the same No MFT device detected unfortunately.

Are you using calibre 6.26?

[sricochet]

Uploaded books display by author sort.

Is there a way to fix this?

[JSWolf]

Originally Posted by sricochet

the new update now sorts authors by last name, but newly uploaded books display that way, too.

Is there a way to fix this?

Other then dump your Kindle(s) for a Kobo, nope.

[sricochet]

Thank you JSWolf. You gave me the courage to carry on!

the default function of the KFX Output plugin is to use the calibre Tweaks functions for metadata.

In Preferences >Tweaks > Author sort name algorithm I changed the author_sort_copy_method = 'comma' to author_sort_copy_method = 'copy'

apply changes to tweak, restart calibre then convert to KFX. the metadata will be embedded in the KFX file.

Then the author name shows up as the first name then last name order in Kindle Previewer

This however does not change the author layout on the Kindle. I think Amazon is trying to sort the authors by last name, but in this update version, the kinks haven't been worked out and the author names are displayed as Last Name, First Name. I hope that in future updates this is addressed. Cross my fingers.