KindleUnpack (MobiUnpack): Extracts text, images and metadata from Kindle/Mobi files

[DiapDealer]

Does anybody have know of any free/cheap kindle books that have HD images for testing purposes? I've not come across any yet through my normal reading/purchasing.

[KevinH]

Hi,

Check out the HD Image test files tkeo posted here:

https://www.mobileread.com/forums/sho...&postcount=779

Also a few posts earlier than that there was a Doormouse test case that had them as well (when we were fixing the split bug that lost mobi 8 images)

https://www.mobileread.com/forums/sho...&postcount=740

Not sure if the link in that post is still active or not.
KevinH

[AaronShep]

Originally Posted by DiapDealer

Does anybody have know of any free/cheap kindle books that have HD images for testing purposes? I've not come across any yet through my normal reading/purchasing.

I'm not sure what qualifies as an HD image, but my book "Baking with Cookie Molds" is 99 cents and has about 75 photos that are 864 x 1296, with a few smaller or larger.

[NiLuJe]

@DiapDealer: Alternatively, I've just randomly fed an ePub that happened to have large images to KG 2.9, and bam, HDContainer .

[kovidgoyal]

FYI, I finished reversing the full CONT record yesterday (apart from a couple of fields that are always zero and one in all the files I have seen). Feel free to pinch my work.

I also dont know what the bytes 4-12 in the CRES records are as they seem to always be 0000 000C

[KevinH]

Hi Kovid,

Wonderful! Thanks. I will grab and integrate what I can into the next stable release. Luckily we don't ever have to write Kf8 files like Calibre does!

Take care,
Kevin


Edit: I grabbed calibre/src/calibre/ebooks/mobi/debug/containers.py to look at and that will be a big help in decoding the CONT header values actually mean! Thanks!

Originally Posted by kovidgoyal

FYI, I finished reversing the full CONT record yesterday (apart from a couple of fields that are always zero and one in all the files I have seen). Feel free to pinch my work.

I also dont know what the bytes 4-12 in the CRES records are as they seem to always be 0000 000C

[tkeo]

Hi,

It seems kindlegen.exe cannot set EXTH 508,512 and 522.
I also find that Furigana input fields exist in "Enter Your Book Details" page of KDP in amazon.co.jp, but not in amazon.com.

Thanks,

[KevinH]

Hi tkeo,

There are lots of EXTH values that can never be set by kindlegen but still exist in commercial ebooks produced by Amazon. Most but not all of these relate to DRM, text to speech settings, text clipping, the "watermark", the "tamperproof" keys, etc. So I am okay with decoding what these mean and using what we can to recreate the original epub as closely as possible. It just means that passing it back through kindlegen will not recreate the original as it has no way to set these extra MetaData.

I am thinking of writing a full metadata editor in python to allow full editing of metadata items in the mobi 6, mobi 8 and now container headers. I will base it off an improved version of the dual meta editor code along with the new information on all of the possible exth values.

That way people can augment the output of kindlgen with all of the known metadata without having to pass it through KDP or whatever else Amazon is doing now.

Take care,

Kevin

Originally Posted by tkeo

Hi,

It seems kindlegen.exe cannot set EXTH 508,512 and 522.
I also find that Furigana input fields exist in "Enter Your Book Details" page of KDP in amazon.co.jp, but not in amazon.com.

Thanks,

[JSWolf]

Originally Posted by NiLuJe

@DiapDealer: Alternatively, I've just randomly fed an ePub that happened to have large images to KG 2.9, and bam, HDContainer .

I fed Kindlegen Words of Radiance by Brandon Sanderson and because it has large high resolution images, the resulting combo file does have HD images.

I did find one bug. in 0.73 with HDImages. If HDImages exist and you do not check to use them, you get a directory called HDImages with the HDImages. But if you check to use HD images and they exist, you get the HDImages directory, but it is empty. Even when there are no HD images present, the HDImages directory is created and empty.

[KevinH]

Hi,

Yes, that is as designed. There are directories in the output tree for all possibilities for mobi 8.

Why do you consider that a bug? If there are no HDImages you can see that as the directory is empty. If there are HDImages then they will be placed there unless you tell KindleUnpack to instead use them to replace their lower resolution versions.

I consider that a "feature!". ;-)

KevinH

Originally Posted by JSWolf

I fed Kindlegen Words of Radiance by Brandon Sanderson and because it has large high resolution images, the resulting combo file does have HD images.

I did find one bug. in 0.73 with HDImages. If HDImages exist and you do not check to use them, you get a directory called HDImages with the HDImages. But if you check to use HD images and they exist, you get the HDImages directory, but it is empty. Even when there are no HD images present, the HDImages directory is created and empty.

[DiapDealer]

Originally Posted by KevinH

Why do you consider that a bug? If there are no HDImages you can see that as the directory is empty. If there are HDImages then they will be placed there unless you tell KindleUnpack to instead use them to replace their lower resolution versions.

Jon sometimes confuses "bug" with "I don't want it to work that way."

J/K Jon.

[tkeo]

Hi Kevin,

Originally Posted by KevinH

I am thinking of writing a full metadata editor in python to allow full editing of metadata items in the mobi 6, mobi 8 and now container headers. I will base it off an improved version of the dual meta editor code along with the new information on all of the possible exth values.

It is good if the full metadata editor is available.

I am now updating (very slowly though) the processing of RESC to separate metadata into for epub2 and for epub3, and of EXTH 508,512 and 522.

As for adding refine attribute from EXTH 508,512 and 522.
It is difficult to find correspondence between Titles, Creators, Publishers and EXTH 508,512, 522 if they have more than two values since KDP seems not preserve the oders of EXTH 508,512 and 522. So editing manually is required as you mentioned before.

Take care,
tkeo

[JSWolf]

Originally Posted by KevinH

Hi,

Yes, that is as designed. There are directories in the output tree for all possibilities for mobi 8.

Why do you consider that a bug? If there are no HDImages you can see that as the directory is empty. If there are HDImages then they will be placed there unless you tell KindleUnpack to instead use them to replace their lower resolution versions.

I consider that a "feature!". ;-)

KevinH

One of those "its a bug, but since I don't want to fix it, it's now a feature".

[KevinH]

Nope, it is "as designed" so it is a feature, not a bug. Please reread my comments about how it is meant to be used and what information it carries.

Originally Posted by JSWolf

One of those "its a bug, but since I don't want to fix it, it's now a feature".

[davidnwelton]

I am sorry to butt in, but I was just curious if you guys had considered putting the source code for mobiunpack up on github or something like that that makes it easier to collaborate on.

Thanks