Kindle Scribe - Sideloading books with USB?

[stumped]

Originally Posted by tomsem

It might be different with email delivery, which I never use. I've always used the Send To Kindle app (or equivalent feature of iOS app).

I was quoting what it says on the Amazon page. If .MOBI does not mean 'dual format MOBI' i.e. what kindlegen used to generate, and what Tor and other publishers used to make for Kindles, then I don't know what they mean by it.

that amazon help page is a real mess.
the allowed formats for send to kindle APP is not the same as for send via email

could be they have not fully updated the page, or maybe they really are different.

i assumed they had inserted the mobi rules change which I quoted, near top of page, but had not gone all the way down the page to make other rules consistent.

some helpful person could perhaps test and update- i am not that fussed myself, i got my send to kindle working using epub and that was all I needed

[talaivan]

Just converted a small file to kfx with Calibre+plugin and loaded it onto the Scribe -- no luck. Can't edit it with the pencil.

[tomsem]

Originally Posted by talaivan

Just converted a small file to kfx with Calibre+plugin and loaded it onto the Scribe -- no luck. Can't edit it with the pencil.

But you can still annotate the usual way, right? You just can't create Sticky Notes?

We should try sending it via Send To Kindle, but decline to store in Cloud. If that works then at least there's an alternative to side loading.

Another test: Send a copy via S2K, compare the resulting KFX to the side loaded one.

One fun discovery: take the Print replica KFX converted from a PDF off of Scribe and side load it to iOS app. You can mark it up, make text notes. Something broken at the moment on Scribe.

[tomsem]

Originally Posted by talaivan

Just converted a small file to kfx with Calibre+plugin and loaded it onto the Scribe -- no luck. Can't edit it with the pencil.

I side loaded KFX and can add Sticky Note (and as expected the sent-but-not-stored one works as well). That's the only Pen thing you can do with reflow. What isn't working for you?

Why do you want to side load in the first place? If you send ePub you don't need calibre to create KFX, and you don't need to store it.

For side-loading, the missing part is not being able to create Print Replica locally, and ideally one that includes active TOC (to make it more navigable than the one you get via send to kindle, which doesn't generate one from PDF alone). It should be possible, I think.

[Quoth]

Yet other makes of 10″ approx eink tablets just use a plain PDF. The Amazon system is deliberately overcomplicated to take control. A walled garden. They could break, turn off or charge for the conversion.
I was going to get a Scribe due to the 300 dpi. But I won't ever while you have to interact with Amazon's servers for annotation. That's not needed and unreasonable. It's a nasty point of failure (your own internet could be down, or Amazon's server) apart from privacy and security issues.

[DuckieTigger]

Originally Posted by tomsem

For side-loading, the missing part is not being able to create Print Replica locally, and ideally one that includes active TOC (to make it more navigable than the one you get via send to kindle, which doesn't generate one from PDF alone). It should be possible, I think.

So the reason that side loaded PDF does not annotate, but emailed PDF works, is the conversion to fixed format KFX? For reading is there a benefit of using KFX vs PDF on the Scribe?

[Quoth]

Originally Posted by DuckieTigger

So the reason that side loaded PDF does not annotate, but emailed PDF works, is the conversion to fixed format KFX?

Yes, they do a "best effort" conversion to fixed layout KFX. Everyone else annotates the PDF, which PDFs are sort of designed for as you can add extra layers of text, image (pen annotation) or anything, anywhere on the document.

Originally Posted by DuckieTigger

For reading is there a benefit of using KFX vs PDF on the Scribe?

The reverse. Converting PDF at all, ever, degrades it. PDF is an end use print replica. They are trying to convert it to a Fixed Layout version of KFX. Also there are many kinds of PDF content: bitmap images, vector images, postscript (layout, vector and text) etc, which can be in layers. Unlike ebooks / HTML the content in the file may not even be sequential. It's totally stupid system that Amazon has, apparently so they have a copy of the documents and you thus can't annotate except via their server. What if they start to charge, or are hacked, or lose interest and bin the service.

If you just want to read the PDF, just use USB transfer. Same applies to ebooks not bought from Amazon. Convert them to azw3 (if mobi or epub) and transfer via USB. Docx is best epub2 via Calibre, convert to azw3 and transfer via USB. Only use PDF for things already as PDFs. There are tools to pre-crop margins or brighten for scanned to PDFs from such as archive dot org.

[DuckieTigger]

Originally Posted by Quoth

Yes, they do a "best effort" conversion to fixed layout KFX. Everyone else annotates the PDF, which PDFs are sort of designed for as you can add extra layers of text, image (pen annotation) or anything, anywhere on the document.

The reverse. Converting PDF at all, ever, degrades it. PDF is an end use print replica. They are trying to convert it to a Fixed Layout version of KFX. Also there are many kinds of PDF content: bitmap images, vector images, postscript (layout, vector and text) etc, which can be in layers. Unlike ebooks / HTML the content in the file may not even be sequential. It's totally stupid system that Amazon has, apparently so they have a copy of the documents and you thus can't annotate except via their server. What if they start to charge, or are hacked, or lose interest and bin the service.

If you just want to read the PDF, just use USB transfer. Same applies to ebooks not bought from Amazon. Convert them to azw3 (if mobi or epub) and transfer via USB. Docx is best epub2 via Calibre, convert to azw3 and transfer via USB. Only use PDF for things already as PDFs. There are tools to pre-crop margins or brighten for scanned to PDFs from such as archive dot org.

Thank you. I assumed the "print replica" KFX is simply the PDF file embedded 1:1 into a KFX container. That, unfortunately, is not the case.

[tomsem]

Originally Posted by DuckieTigger

So the reason that side loaded PDF does not annotate, but emailed PDF works, is the conversion to fixed format KFX? For reading is there a benefit of using KFX vs PDF on the Scribe?

You can annotate side loaded PDF (if there's text 'layer'), but it's unchanged from what it has always been, so no Pen annotations.

PDFs sent after 12 Dec (or thereabouts) get converted to Print Replica (one of the several fixed layout formats KFX can contain) when they are sent to Scribe (other Kindles and the Kindle apps continue to get PDF).

Right now they should look the same, and depending on the PDF in question, the KFX should have better performance (possibly much better), but you can't search for text, select it, look up words in dictionary or highlight it (we are awaiting a fix).

But you can use Pen for marking up, use highlighting brush to decorate text, and create sticky notes anywhere on the page. The highlighting and sticky notes won't be logically associated with any text, but with careful placement it will be similar visually.

When there's an update to fix it, you should be able to use the other (text markup) methods as well without having to send the PDF in again, or even downloading it again. And you could go through and replace the unattached highlights and notes with ones that are 'attached'.

And the annotations in all forms will be backed up and sharable, whereas side loaded PDF annotations can only be located in my clippings text file.

To me the converted form will be more functional despite not having all annotation options available.

[jhowell]

If you want to create your own print-replica KFX for markup on the Scribe the easiest way is by send-to-kindle of a PDF to a Scribe.

If you want to produce fixed-layout KFX yourself without involving Amazon’s servers you can do that by using Amazon’s Kindle Create to produce a print replica book from your PDF, export that in KPF format, and then use the CLI mode of my KFX Output plugin to convert that KPF to KFX. (Conversion using calibre won’t work because it does not support fixed layout.) That will produce a KFX file that you can sideload to a Scribe via USB.

I doubt that it would be possible to export the annotated file so this would probably not be very useful other than as an experiment.

[DuckieTigger]

Originally Posted by tomsem

You can annotate side loaded PDF (if there's text 'layer'), but it's unchanged from what it has always been, so no Pen annotations.

PDFs sent after 12 Dec (or thereabouts) get converted to Print Replica (one of the several fixed layout formats KFX can contain) when they are sent to Scribe (other Kindles and the Kindle apps continue to get PDF).

Right now they should look the same, and depending on the PDF in question, the KFX should have better performance (possibly much better), but you can't search for text, select it, look up words in dictionary or highlight it (we are awaiting a fix).

But you can use Pen for marking up, use highlighting brush to decorate text, and create sticky notes anywhere on the page. The highlighting and sticky notes won't be logically associated with any text, but with careful placement it will be similar visually.

When there's an update to fix it, you should be able to use the other (text markup) methods as well without having to send the PDF in again, or even downloading it again. And you could go through and replace the unattached highlights and notes with ones that are 'attached'.

And the annotations in all forms will be backed up and sharable, whereas side loaded PDF annotations can only be located in my clippings text file.

To me the converted form will be more functional despite not having all annotation options available.

Ah good. I shall wait until this is fixed before deciding to purchase a Scribe, unless I cave in earlier and use it as a bigger PDF reader. If sideloaded PDF works just as well on the Scribe as it does on the Oasis, I would be happy.

[DuckieTigger]

Originally Posted by jhowell

If you want to create your own print-replica KFX for markup on the Scribe the easiest way is by send-to-kindle of a PDF to a Scribe.

If you want to produce fixed-layout KFX yourself without involving Amazon’s servers you can do that by using Amazon’s Kindle Create to produce a print replica book from your PDF, export that in KPF format, and then use the CLI mode of my KFX Output plugin to convert that KPF to KFX. (Conversion using calibre won’t work because it does not support fixed layout.) That will produce a KFX file that you can sideload to a Scribe via USB.

I doubt that it would be possible to export the annotated file so this would probably not be very useful other than as an experiment.

Wait, what? To create a print replica KFX all I need is to open the PDF in Kindle Previewer, save as KPF, and then use which commands exactly for the KFX output plugin? Do you have a thread about it here on MR? Edit: oops, Kindle Create - off looking for a download.

I don't yet have a Scribe. Would those print replica KFX be compatible to sideload to other Kindles?

[tomsem]

Originally Posted by jhowell

If you want to create your own print-replica KFX for markup on the Scribe the easiest way is by send-to-kindle of a PDF to a Scribe.

If you want to produce fixed-layout KFX yourself without involving Amazon’s servers you can do that by using Amazon’s Kindle Create to produce a print replica book from your PDF, export that in KPF format, and then use the CLI mode of my KFX Output plugin to convert that KPF to KFX. (Conversion using calibre won’t work because it does not support fixed layout.) That will produce a KFX file that you can sideload to a Scribe via USB.

I doubt that it would be possible to export the annotated file so this would probably not be very useful other than as an experiment.

Thanks, at least it's possible to experiment, nice to know of that CLI option. I am curious about the internals.

In theory they could construct one from page links near the start of the book that look like a table of contents. It seems like a little ML could handle that task pretty easily, and it would definitely make the experience of navigating more pleasant than a plain PDF is normally, a selling point and of course consistent with Print Replica from Kindle Store. They do generate a default TOC with Go to page at least.

In some cases creating an ePub fixed layout instead might work. You can get TOC that way, but hyperlinks don't survive import, and no text selection and therefore no annotation whatsoever. A pretty steep tradeoff.

But perhaps they'll find reasons to support this in the future. If you could add alt text to comic bubbles, for example, it would be accessible for screen readers, and maybe provide an anchor for annotation as well.

[talaivan]

All right, I sent a pdf of a 100 pages or so to my Scribe with send-to-kindle. I could annotate it -- no problem, and the pen works well on the screen. Then I sent it from the Scribe it to my email, and it appeared. When I clicked the file to "download" it, it appeared in Safari and I had to export it as a pdf from Safari. I couldn't see any way to download it directly as a pdf, though the pdf produced by Safari was satisfactory. But why so many steps? Send, wait for it to appear, annotate, email it to yourself, save as pdf. Am I missing something?

[jhowell]

Originally Posted by DuckieTigger

Wait, what? To create a print replica KFX all I need is to open the PDF in Kindle Previewer, save as KPF, and then use which commands exactly for the KFX output plugin? Do you have a thread about it here on MR? Edit: oops, Kindle Create - off looking for a download.

Kindle Create is a tool that Amazon makes available for publishers. You double click Create New, select Print Replica format, import your PDF file, and then Export as a KPF file.

To convert the KPF open a Command Prompt window (Windows), switch to the folder where the file was saved and run:

calibre-debug -r "KFX Output" -- "your-file-name.kpf"

That will produce your-file-name.kfx which can be sideloaded to a Kindle device or app.

Not having as Scribe I can't actually test how the results will work. If someone tries it let us know.

Originally Posted by DuckieTigger

I don't yet have a Scribe. Would those print replica KFX be compatible to sideload to other Kindles?

You can sideload them, but they work poorly at present. For example you cannot select text within the document.