FYI: new Kindle ebooks have DRM tied to clipping limit and TTS

[KevinH]

FYI:

Since calibre has to work with and possibly modify metadata for Kindle ebooks with DRM, please note the very latest Kindle ebooks (bought in the last 2 or 3 days) have been changed to tie their DRM to additional metadata values. It no longer just uses keys 209, and 208 on Mobi ebooks or keys and thumbnail for Topaz ebooks. Values in "keys" and "209" now point to multiple metadata fields including the "clipping limit" and/or "TTS" metadata values.

So now if software or the user changes one of these other metadata values like the "clipping limit", that modified ebook with DRM will no longer open. If a user wants to change the clipping limit, they will need to actually remove the DRM first.

More specifically, if you look at the "keys" metadata value of the latest Topaz ebooks, you will see something similar to:

thumbnail,ADOK,ClippingLimit

and so changing any of the metadata values for any of those fields will prevent the modified ebook from being decoded/opened by official Kindle software (or having its DRM removed).

Similarly, for mobi ebooks, the code that determines which metadata values are now involved in DRM looks like the following:

rec209 = self.meta_array[209]
data = rec209
# The 209 data comes in five byte groups. Interpret the last four bytes                                             
# of each group as a big endian unsigned integer to get a key value                                                 
# if that key exists in the meta_array, append its contents to the token                                            
for i in xrange(0,len(data),5):
     val,  = struct.unpack('>I',data[i+1:i+5])

So changing any of the metadata values whose keys are given by any in the set of "val" values, will result in breakage that will prevent the ebook from being opened even with official software.

Thought you should be aware that a larger set of metadata must not be touched in the latest Kindle ebooks to prevent breakage.

Of course, removing the DRM before modifying those metadata values will still work and allow those metadata values to be changed.

[kovidgoyal]

As far as I know the calibre MOBI metadata update code does not touch tts/clipping limit fields. But thanks for the heads up, this way if I get reports about calibre rendering DRMed MOBI files un-openable I will know where to start looking.