KindleUnpack (MobiUnpack): Extracts text, images and metadata from Kindle/Mobi files

[DiapDealer]

I believe process should be exactly the same if it's a stand-alone AZW3 or a stand-alone MOBI. Their header records (and EXTH) should be the same structure. The only time it would be different is if it's raw Kindlegen output ... which contains both MOBI and AZW3 formats (and a header and an EXTH for both).

If you are indeed talking about a dual-purpose file (MOBI/KF8), the thread you linked to wound down with the general consensus that editing the KF8's EXTH (in the second header of the hybrid file) didn't really solve much of anything (with regard to devices considering them Personal Docs and only supporting low-quality covers and such). It seems to be a bit of a wild goose chase.

In my opinion, the hybrid, dual-format output of Kindlegen shouldn't really be considered suitable for distribution. It WILL work on devices. But as you've found out, there are unsatisfactory side-effects.

[EbokJunkie]

Thanks, I need edit only pure mobi8 aka azw3 because I extract azw3 from kidlegen's mobi and just need either swap offsets of thumbnail and cover (201 and 202) + fill 501 or need to fill 113 + fill 501. In each case I need to output separate azw3.
Looking at mobi_split code I see that autor checks if length of content at exth121 (KF8 BOUNDARY Offset) is not zero. When I submit azw3 to mobi_split I see that this length is zero.
Might it be that location of metadata in dual/combo mobi and in azw3 in different?

[quiris]

There is a problem during unpacking dual Mobi files created with the latest calibre 1.31:

Traceback (most recent call last):
  File "/Applications/KindleUnpack v0.64.app/Contents/Resources/kindleunpack.py", line 1684, in <module>
    sys.exit(main())
  File "/Applications/KindleUnpack v0.64.app/Contents/Resources/kindleunpack.py", line 1673, in main
    unpackBook(infile, outdir)
  File "/Applications/KindleUnpack v0.64.app/Contents/Resources/kindleunpack.py", line 1604, in unpackBook
    process_all_mobi_headers(files, sect, mhlst, K8Boundary, False)
  File "/Applications/KindleUnpack v0.64.app/Contents/Resources/kindleunpack.py", line 1429, in process_all_mobi_headers
    if k8resc.getSpineSkelid(index) < 0:
  File "/Applications/KindleUnpack v0.64.app/Contents/Resources/mobi_k8resc.py", line 301, in getSpineSkelid
    return self.spine[i][1]
TypeError: 'NoneType' object has no attribute '__getitem__'

[DiapDealer]

I ran into the same error with one book, but assumed it was just a badly formed exception, so I didn't really investigate. I don't think it was a calibre generated mobi, though, so it may not be limited to that.

EDIT: actually it WAS a calibre generated azw3 (standalone), but it was created with a calibre version much earlier than v1.31.

[tkeo]

Originally Posted by quiris

There is a problem during unpacking dual Mobi files created with the latest calibre 1.31:

Hi,

I have fixed the bug to process the azw3 file you attached. However; I have not checked KindleUnpack v0.64 as a clibre plugin because calibre did not support 'page-progression-direction' and 'page-spread' properties when I wrote it.

In addition, I have modifed another part in 'kindleunpack.py' in order to process a RESC section which does not match the size retrieved from a base32 string to actual bytes.

The modified parts are as follows:

line 1415(originally 1411) fixed.
lines 1241 -1244 inserted.

Thanks,

EDIT
The attached file is removed and reposted with a changelog addition.

[DiapDealer]

Originally Posted by tkeo

Hi,

I have fixed the bug to process the azw3 file you attached. However; I have not checked KindleUnpack v0.64 as a clibre plugin because calibre did not support 'page-progression-direction' and 'page-spread' properties when I wrote it.

I've tried to make the plugin as "plug 'n' play" as possible. Calibre may not support those properties, but you still should be able to drop updated kindleupack files into the plugin's 'kindleunpack' folder with no trouble (provided the __init__.py stays in place). Unless the number/order of parameters in one of the main methods that the plugin's code accesses has changed, of course.

This reminds me that I haven't updated the plugin to the latest KindleUnpack code, yet.

[tkeo]

Originally Posted by DiapDealer

I've tried to make the plugin as "plug 'n' play" as possible. Calibre may not support those properties, but you still should be able to drop updated kindleupack files into the plugin's 'kindleunpack' folder with no trouble (provided the __init__.py stays in place). Unless the number/order of parameters in one of the main methods that the plugin's code accesses has changed, of course.

Hi,

I had confirmed that v0.63 works fine as a calibre plugin (but not satisfy my purpose). I did not changed main methods; it should work well I thought.

I have confirmed v0.64 and posted 'kindleunpack.py.zip' as the calibre plugin with a few mobi files.

Thanks,

[KevinH]

Hi Tkeo,
Please don't forget to increment the version number (or add a letter) and add a changelog entry at the top before posting new versions of files here to prevent version confusion.

I assume you want this to become part of a KindleUnpack version 0.65? If so, once it is verified your fix works, we can go ahead an make a new release.

Thanks,

Kevin

[quiris]

Is there an open github.com repository with KindleUnpack source code? It would be easier to track changes…

[KevinH]

Hi quiris,

Been through that once before. No one used it when we had a google code one. I am not a fan of learning yet another source code repository system (rcs, cvs, subversion, bazaar, mercurial - anyone?). Updates are simply too few in number and typically come months apart with no activity in between.

We just need to be careful to mark anything posted here either with a new version number or label them experimental and add a letter to indicate each alternative version.

KevinH

[KevinH]

Hi quiris,

Does the proposed fix work for you? If so, let us know and we can make a new official release and then DiapDealer could update his plugin to version 0.65 skipping 0.64 to save him some work.

Thanks,

KevinH

[tkeo]

Originally Posted by KevinH

Hi Tkeo,
Please don't forget to increment the version number (or add a letter) and add a changelog entry at the top before posting new versions of files here to prevent version confusion.

I assume you want this to become part of a KindleUnpack version 0.65? If so, once it is verified your fix works, we can go ahead an make a new release.

Hi Kevin,

I'm sorry for my laziness. I add a chenglog and chenge the version to v0.64a.

I am not sure that this modification should be treated as a official version up.
In my consideration, files which are not able to processed with v0.64 are somthing irregular; i.e. K8 format with no resc section and the size difference of resc data. Should they be able to be unpacked?

Thanks,

[KevinH]

Hi,
Given that calibre is a big producer of both azw3 and joint mobis, and calibre does NOT generate a RESC section, then yes we need to make sure we don't barf on calibre generated files.

That is why I thought this fix is needed. Also, if calibre tries to keep an old RESC around, it would have the wrong skelids in the RESC, so we really need to protect against missing or ill-formed RESC sections and ignore them.

Take care,

Kevin

[tkeo]

Originally Posted by KevinH

Given that calibre is a big producer of both azw3 and joint mobis, and calibre does NOT generate a RESC section, then yes we need to make sure we don't barf on calibre generated files.

That is why I thought this fix is needed. Also, if calibre tries to keep an old RESC around, it would have the wrong skelids in the RESC, so we really need to protect against missing or ill-formed RESC sections and ignore them.

Hi Kevin,
OK.

I have updated 'kindleunpack.py' to improve compatibility and reliability. The functional differece from v0.64a is to make cover page creation enable even when RESC is missing.
Please update the offcial release if it has no ploblem.
Thanks,

P.S.
In the future, I might separate the code for the cover page creation from 'mobi_k8resc.py' to an independent file and integrate the code which DiapDealer have written. But I do not plane it.

[tkeo]

Hi,

This is not intend to include an official release.
I have modified 'mobi_opf.py' to convert Amazon defined meta properties to epub3 properties. This makes improve fixed-layout ebooks conversion.
However, I have changed the package version in an opf from 2.0 to 3.0 in order to fulfill epub3 regulations. It may cause serious trouble for reading systems designed to both epub2 and epub3 although I have not encountered yet and it seems the package version does not affect. To fulfill the epub3 regulations completely much modifications are needed.

Thanks,