Kindle Scribe notebook format

[jhowell]

Originally Posted by j.p.s

I misunderstood post #9 to say that a notebook does not include a copy of the template.

It was a reasonable guess made by tomsem that a notebook didn't include the template. That turns out to be incorrect. We are still learning how it works at this point.

If any other Scribe owner wants to share files from their .notebooks folder, either publicly or sent to me in private, it would help in my decoding how Scribe notebooks are structured with a goal of being able to convert them to a usable format without involving Amazon's export function.

[tomsem]

Sync of Notebooks between Scribes works as well as one can expect: elements added and deleted on each device (offline, for example) get merged once the changes sync up. Of course this will be a fairly unusual situation until notebook editing expands to other platforms (which I think is likely).

[tomsem]

Originally Posted by jhowell

The sample notebook file provided by tomsem references the music_1860x2480 template but it also includes a copy of that template internally. So it appears that once a notebook is created it will not be affected by any future changes made to the template it is based on.

Cool, thanks!

[tomsem]

Originally Posted by jhowell

That is definitely a KDF file, KFX format in an SQLite database with a "fingerprint" to prevent tampering. I tried to convert it using my KFX Input plugin but it uses many new data types that I have not encountered before so very little of it can be decoded at present. It appears to be three pages of content based on the music_1860x2480 template.

I have attached a textual representation of the data contained in that file in Amazon's Ion data format.

Thanks, your findings match up with the content (number of pages etc.), and I will be looking at the .txt file so I understand the structure better.

I think it would be worth the effort to 'decode everything'. I assume to do this we need an example with all of the current entity types (brush, pen, highlight etc.). I'll try to do this.

Once that's in place, I think it would be quite useful to be able to export to some standard vector format so that one could continue editing with some 3rd party tools.

I assume the "DL" in "NMDL" expands to 'Description Language'. Is this some standard DL? Even if it is not, these could probably be mapped to one that is.

And you could collect these NBK extracts in a SQLite database to feed a conversion toolchain.

And same technique could be used to export pen annotations more generally.

Going the other way (side loading notebooks created somewhere else) seems less useful, and more challenging technically.

[tomsem]

Originally Posted by tomsem

It appears you cannot back up and restore older notebook versions using computer.

This worked:
- copy notebook folder to computer
- remove notebook folder from Scribe
- disconnect usb and sync (download is gone)
- connect usb and copy notebook folder back
- disconnect usb
- download is restored

This didn't work:
- copy notebook folder
- disconnect usb
- make a change to notebook and sync
- remove download
- connect usb
- copy previous notebook folder to scribe
- disconnect usb
- no download shown, it ignores older version

So to backup a version, it looks like you need to make a copy that also gets saved to cloud.

So I am thinking about this some more.

It seems Kindles think they are in London. Download something and it displays creation time that's in future with respect to local time (+8h for me).

So maybe to restore previous version you need to change the modify time(s) (mtime) so it is greater than create time (ctime)? So it thinks you have an update? I'll try it.

I also tried changing the folder guid to see if Scribe would at least open it. That alone is not enough, but maybe it is again issue with timestamp.

It would be nice to have some way to have a 'local', non-cloud backup that would allow restore of a specific version, and if it would let you supply your own guid's you could effectively side load notebooks that would not be in cloud.

But I suspect the guid has to be one associated with your account.

Even if that's the case, if it's possible to execute scenario #2 by mucking with mtime, it might be useful for people with privacy concerns: you generate as many empty notebooks as you like (in the folders you want them to reside in), work with those with WiFi off, and if you need to go online just move them off kindle so they never sync to cloud, restore after going offline. A PITA but some automation could help with it.

[tomsem]

Originally Posted by tomsem

So maybe to restore previous version you need to change the modify time(s) (mtime) so it is greater than create time (ctime)? So it thinks you have an update? I'll try it.
.

It works! e.g.:

touch -A +10hh nbk

But it does not like custom guid's.

[jhowell]

Originally Posted by tomsem

I think it would be worth the effort to 'decode everything'. I assume to do this we need an example with all of the current entity types (brush, pen, highlight etc.). I'll try to do this.

More notebook examples would help. Having examples of simple drawings, such as a single dot or straight line, would be useful in decoding how drawing data is stored. Seeing the .notebook files associated with annotated documents would also help fill in the gaps.

Originally Posted by tomsem

Once that's in place, I think it would be quite useful to be able to export to some standard vector format so that one could continue editing with some 3rd party tools.

I am surprised by what we have found so far. I expected the Scribe to store something like SVG (scalable vector graphics). Instead it is keeping lower level data, like pen tilt and pressure, as binary data.

Spoiler:

      'nmdl.stroke_bounds': [3123, 4448, 3447, 4707],
      'nmdl.color': 0,
      'nmdl.brush_type': 7,
      'nmdl.random_seed': 0,
      'nmdl.thickness': 23.625,
      'nmdl.type': 'nmdl.stroke',
      'nmdl.stroke_points': {
        'nmdl.thickness_adjust_factor': {{
          AQFMAQAAFERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERGQ=
        }},
        'nmdl.density_adjust_factor': {{
          AQFMAQAAFERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERGQ=
        }},
        'nmdl.num_points': 332,
        'nmdl.position_x': {{
          AQFMAQAAHV1ubdRERETW1NVERE1dRdReVE1URNXVTVRU1EXV1EREREREREVNXUVNXURdRURF3V1URERd
          1UXVREXUbURERNVEVNREVNRdVE1URE1URNVNVERNXV1FRdRdRU1dVERUTV1ERF1FRE5dVNRF1F1ETVRN
          VERF1ERdRERdXV1NRF1NXV3VTV1NXVRN1U1U1dVF1ERdXV1FRNVETVTU1URF1UVF1UVF1rQ=
        }},
        'nmdl.position_y': {{
          AQFMAQAAFF1ERU1ERERF1NbURF1ERURERE1V1EXV1ERF5V5URNVF1dXURF1ERF1ERdRF5URERERETVRE
          RERERURF1URERE1E1V1dRNVEXUREXURdXdRdRU1F1ERETW1E1dXV1E1dVE1eVEREXUXV5URdRERERdRE
          1dREXURETV1URE1dVERERF1dRNRd1dVETV1ERF1G1FXVREREXVREXW5U1URE1d1F3V5e3RI=
        }},
        'nmdl.pressure': {{
          AQFMAQAAFV1NcZGU3heUSfGRlJ7vYZTZHmkdVJ9FcWdUT01PV5bpGe9E0fVO1HUZGWTX4e5pHfleZpTd
          3V9JcZdeHVXU1fTUbW1OfWfk15Xk1k1k3lQdTX5dTUREQZTZFGQZVPfk5U1uVuRGQZfZHlcZafHm5W8f
          Zd51VOftTd15VZFlVn0UdEkd7f/V9E517fXdVUfU3Wkd1pbxnVnflGndkfT+mZmRkRERGR13PQofJCgE
          FR4cFgUEBAQEBAQEBAUEBAQEBQUEBAUFBAQEBQQFBQQEBAQGBQgGBQYGBQQJBAcFBAUEBAQEBAQEBgQI
          BAUGBQQFPpcXZwYGCwUvLCAH
        }},
        'nmdl.tilt_x': {{
          AQFMAQAAJEREQZRERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERERERERERERERERERERBlEREREREREREREREREREREREgNZGRkZA==
        }},
        'nmdl.tilt_y': {{
          AQFMAQAAJBlEQZRERERERBlERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          REREkURERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERERE
          RERERERERERERERERERERERERERESRRERERERERERERERBlERERERERERERERERERERERGwHZGTIyGRk
          ZGTIyMjI
        }},
      },

I have never worked on software for digitizing drawings so I am unfamiliar with the data structures and APIs typically used in that field. Perhaps someone familiar with drawing devices could suggest a documented data format that would be suitable for exporting Scribe drawings.

I do know that some work has been done to reverse engineer the internal data formats used by reMarkable tablets. That might be worth looking into.

Originally Posted by tomsem

I assume the "DL" in "NMDL" expands to 'Description Language'. Is this some standard DL? Even if it is not, these could probably be mapped to one that is.

I could not find anything relevant online about "NMDL". I am assuming that it is an acronym made up by Amazon. The phrase "note model" appears often in the Scribe firmware. That might be the source of "NM".

[tomsem]

'Note Model Description Language' works for me!

I created a very simple notebook with 5 strokes: Pen, Fountain pen, Marker, Pencil, and Highlight using the blank page template. Except for variations in stroke weight, pressure, tilt I think these are the only entities you can create currently with Pen. The templates must have some others (shapes? text? lines?). I'll pick a few of those and post examples of those next.

I'm going to start reading up on SVG and Python libraries for that. It seems like the way to go as a conversion target.

https://github.com/amazon-ion/ion-python (Python library for Amazon ion)

And yes, interesting that NMDL is so low level.

[j.p.s]

Originally Posted by jhowell

I am surprised by what we have found so far. I expected the Scribe to store something like SVG (scalable vector graphics). Instead it is keeping lower level data, like pen tilt and pressure, as binary data.

Originally Posted by tomsem

And yes, interesting that NMDL is so low level.

I suspect it is related to the spirit of the quote from Blaise Pascal's 1697 The Provincial Letters.

I have only made this letter longer because I have not had the time to make it shorter.

[tomsem]

I don't know how useful these will be, but these are empty notebooks based on a sampling of the templates.

[jhowell]

Originally Posted by tomsem

Here is a zipped up folder for a notebook.

I have been examining the notebooks you uploaded and through trial and error have had some success at decoding the blobs that hold stroke data and converting them to SVG images. It is still a work in progress.

I am attaching a zipped SVG image along with a rasterized JPEG version of the first page of the 61233b77-330b-5427-4581-5f977d998bb5 notebook.

At present the conversion includes only the hand drawn strokes, not the background template. It looks pretty good to me, but I don't have the original to compare it to.

Originally Posted by tomsem

I created a very simple notebook with 5 strokes: Pen, Fountain pen, Marker, Pencil, and Highlight using the blank page template. Except for variations in stroke weight, pressure, tilt I think these are the only entities you can create currently with Pen.

I have been able to decode some more data variations from that, but it isn't making complete sense so far. It is a work in progress.

At this point I am planning that the next release of the KFX Input plugin will be able to convert Scribe notebooks to EPUB format with each page containing an SVG image with a fairly accurate representation of the user's drawing along with the template background.

[tomsem]

Very cool! And no, I don't know how to actually write music.

Let me know if I can help in some fashion.

[jhowell]

Originally Posted by tomsem

Very cool! And no, I don't know how to actually write music.

Let me know if I can help in some fashion.

One thing that would be helpful is to post the PDF that Amazon produces when you export your notebook that shows examples of the various brush types. That would give me a target to work towards.

Also from you or any other Scribe owners more sample notebook files and the associated exported PDF equivalent would be helpful in testing my understanding of the Scribe's notebook format.

Lastly I would like to understand how the Scribe stores annotations. Some sample annotation files from the .notebooks folder would be useful for that.

[tomsem]

This is the PDF export for the Notebook of post #23.

As you will see, it just contains raster images: no objects or layers.

To the extent we can convert the on-device notebooks (to PDF or FXL ePub), it would be an improvement, since we would be able to preserve more of the original properties, and presumably be able to have template and annotation on separate layers (something Amazon should be doing IMO, and maybe they will, some day).

I"ll get some Pen annotation data for documents/books next (with export PDF).

[tomsem]

ebooks: .sdr and .documents folders (content is copy-protected)
kfx_document: public domain book .sdr, .documents folder, .kfx
print_replica_document: documents folder, .sdr, .kfx

Interestingly, Share of personal documents takes two forms: 1) from Notebook (notes and highlights), it Shares only the annotations (both text and Pen), and 2) using the Share icon exports the entire contents with annotations on top.

Didn't notice this before.

Print Replica documents currently do not allow text notes, so Notebook doesn't display any: it shows thumbnail of the annotated page with the Pen annotations.