KindleUnpack (MobiUnpack): Extracts text, images and metadata from Kindle/Mobi files

[KevinH]

Hi tkeo,

Originally Posted by tkeo

Hi Kevin,
I have books in which idrefs have meaningful names, for example: p-cover, p-titlepage, p-caution, p-copyright.
Indeed they do not improve functionality, but is it not enough for reason to keep them?

Thanks,

I found a public wifi site and saw your post.

They need not be meaningful and most are not. That said, as a compromise, how about the following:

1. since their idrefs must be unique for them simply add a very short prefix to theirs such as "x_" or some other short prefix (this is guaranteed to keep theirs unique). This will also guarantee that their idrefs do not overlap with our itemXXXXX idrefs.

2. Remove all of the "set" code and unique id code and simply replace it all with something similar to the following pseudo code.

idref_to_use = "item%05s" % refnum
if their_old_idref[refnum]:
    idref_to_use = "x_" + their_old_idref[refnum]

Or something similar to that in meaning.

This guarantees uniqueness, allows you to see the meaningful names but is so simple no block of uniqueness code is needed making the code as simple as possible.

How does that sound?

KevinH

[tkeo]

Hi Kevin,

Originally Posted by KevinH

They need not be meaningful and most are not. That said, as a compromise, how about the following:

1. since their idrefs must be unique for them simply add a very short prefix to theirs such as "x_" or some other short prefix (this is guaranteed to keep theirs unique). This will also guarantee that their idrefs do not overlap with our itemXXXXX idrefs.

2. Remove all of the "set" code and unique id code and simply replace it all with something similar to the following pseudo code.

This guarantees uniqueness, allows you to see the meaningful names but is so simple no block of uniqueness code is needed making the code as simple as possible.

How does that sound?

It is good.
I am glad for your cooperation.

Take care,
tkeo

[JSWolf]

Can someome please post a new version of KindleUnpack that has the patches applied to 0.27f? Thanks. Yes, I have found .27f failed with an error and I'd like to see if the patches fix this.

[DiapDealer]

Originally Posted by JSWolf

Can someome please post a new version of KindleUnpack that has the patches applied to 0.27f? Thanks. Yes, I have found .27f failed with an error and I'd like to see if the patches fix this.

Post as much detail about the error as you can, Jon. And if you can't post an out-of-copyright file that can reproduce the error, at least point to a commercial book that can.

Also keep in mind that if the drm-free book in question has been altered (after the fact) by any third-party utilities before using KindleUnpack (other than perhaps core calibre features), there's no guarantees that KindleUnpack will be able to accommodate any issues said utilities might introduce.

Remember, KindleUnpack is mainly about reverse engineering kindlegen's drm-free output. A little variance between that and calibre's output is accommodated wherever possible, but if something else has diddled the binary mobi/kf8 (after it became drm-free)... all bets are off.

[JSWolf]

Originally Posted by DiapDealer

Post as much detail about the error as you can, Jon. And if you can't post an out-of-copyright file that can reproduce the error, at least point to a commercial book that can.

Also keep in mind that if the drm-free book in question has been altered (after the fact) by any third-party utilities before using KindleUnpack (other than perhaps core calibre features), there's no guarantees that KindleUnpack will be able to accommodate any issues said utilities might introduce.

Remember, KindleUnpack is mainly about reverse engineering kindlegen's drm-free output. A little variance between that and calibre's output is accommodated wherever possible, but if something else has diddled the binary mobi/kf8 (after it became drm-free)... all bets are off.

But since there are patches made to the version I have, then the version I have is no longer useful for testing. I need a version with the patches applied to test this eBook otherwise, any testing I do is invalid.

[DiapDealer]

Originally Posted by JSWolf

But since there are patches made to the version I have, then the version I have is no longer useful for testing. I need a version with the patches applied to test this eBook otherwise, any testing I do is invalid.

Not having a version with the latest proposed patches applied is not preventing you from stating the nature/details of the error you're getting with the .72f version of KindleUnpack.

[tkeo]

Hi,

Originally Posted by JSWolf

But since there are patches made to the version I have, then the version I have is no longer useful for testing. I need a version with the patches applied to test this eBook otherwise, any testing I do is invalid.

The patchs I have posted do not fix any fatal errors. They are a minor bug fix of cover page renaming and code cleanups. So please test with v0.72f.

Thanks,

[tkeo]

EDIT: Attached files are removed. The changes presented in them will be made after discussions in future version.

Hi,

I have modified KindleUnpack v0.72f. I have made two versions. Both versions include the patches I have posted and do not have functional changes from them if no bugs.
The purpose of them are to reduce the degree of redundancy and to make easier to understand.

The difference of v0.72h from v0.72g is that the spine information retrieved from RESC is passed by a parameter of OPFProcessor() instead of getting correspondences between filenames and idrefs/attribs in the spine inside the OPFProcessor(). To do so, the parameter filenames is changed to fileinfolist which is the list consist of [dir, filename, idref, attribs]. I think it is easier to understand.

Thanks,

[tkeo]

Hi Kevin,

I have moved adding metadataguidetext to guide for print replica books from mobi_opf.py to kindleunpack.py. Please confirm it is proper for simplification.

I have removed and commented out unnecessary code as possible. Further simplification of the code may be possible. I think we will do it after replacing mobi_k8resc.py to simplified version.

Take care,
tkeo

[tkeo]

Hi Kevin,

Sorry, I missed to look at this post.
I will chage in future posts.

Originally Posted by KevinH

In an effort to help track down other mistakes like that, I have changed the entire code base to use the python.org PEP recommendation when testing against None as follows:

I have changed all:
== None
to:
is None

And changed all:
!= None
to:
is not None

Now any testing against 0 vs non-zero using "if variable" will stand out as different and I can track down any such occurrences to make sure they were not using it to test against None but were instead looking at the non-zero case.

I have attached a patch to move us to follow the python PEP recommendation.
See pep_use_patch.txt

[JSWolf]

In order to test the latest KindleUnpack, I need a full version like I've said all along. I've applied the diff files and it stops working. Please post a full version so I can test it with the AZW3 that I have that fails with 0.72f. I was hoping what was posted to make 0.72h would have been a full version. But nope. It just causes KindleUnpack under Windows to break and not work at all.

[Doitsu]

@JSWolf:You keep saying that the latest KindleUnpack version, doesn't work, but you fail to provide any details such as the exact error message displayed by KindleUnpack. It's quite possible that the latest version, which is not intended for end users anyway, might contain a bug, but if you don't provide any details, it won't be fixed.

Have you tried unpacking the same .azw3 file with the stable KindleUnpack version that is included in the current KindleUnpack Calibre plug-in?

It so, what were the results?

[JSWolf]

Originally Posted by Doitsu

@JSWolf:You keep saying that the latest KindleUnpack version, doesn't work, but you fail to provide any details such as the exact error message displayed by KindleUnpack. It's quite possible that the latest version, which is not intended for end users anyway, might contain a bug, but if you don't provide any details, it won't be fixed.

Have you tried unpacking the same .azw3 file with the stable KindleUnpack version that is included in the current KindleUnpack Calibre plug-in?

It so, what were the results?

It works with the 0.67 plugin for Calibre. But 0.72f is not the latest. 0.72h is the latest and that's the version I should be testing. I tried building 0.72h and it didn't work. This is why I'd like 0.73h already built so I can test this AZW3. If it then doesn't work, I will post the error and give over the AZW3 if needed so the error can be fixed if there still is an error.

[DiapDealer]

It's a good plan: keep it a secret until you see if it accidentally/magically gets fixed before giving the devs something concrete they can actually be working on right now.

Here's the skinny: none of the devs are aware of any fatal flaws in 0.72f. If they were, they'd be working on them. So if you HAVE discovered one, the chance that any of the latest PROPOSED changes would accidentally fix that fatal flaw (that no one was aware of in the first place) is slim to none. In short... you're only holding up any potential fix for your issue even longer.

[JSWolf]

Originally Posted by DiapDealer

It's a good plan: keep it a secret until you see if it accidentally/magically gets fixed before giving the devs something concrete they can actually be working on right now.

Here's the skinny: none of the devs are aware of any fatal flaws in 0.72f. If they were, they'd be working on them. So if you HAVE discovered one, the chance that any of the latest PROPOSED changes would accidentally fix that fatal flaw (that no one was aware of in the first place) is slim to none. In short... you're only holding up any potential fix for your issue even longer.

What is the deal with posting a full version of 0.72h? It gets posted, I test, if things fail, I post about it and it gets fixed. Simple really. But as it is, I get blocked at all turns about testing the latest version.