KindleUnpack (MobiUnpack): Extracts text, images and metadata from Kindle/Mobi files - Page 50

R. Scot Johns · 05-26-2014, 03:23 PM

An anomaly is an unknown deviation from the common expectation, and can be caused by any number of random factors along the chain, such as a signal fluctuation during transmission that causes a loss of data, or a user error. It does not inherently imply a bug exists, since a bug is something which generally has a consistent cause that can be repeated, traced and fixed. I use the term anomaly in the sense that there is an unknown variable, which is why I asked if anyone else has seen it occur. Apparently the answer to that question is no, which makes it even more of an anomaly than before. Of course, I'm inherently a philosopher, not a programmer, so I tend to use terms differently than others do quite often.

As for "compressed" versus "uncompressed" the answer should be obvious. Kindlegen applies compression to input images (assuming you do not use the -C0 option in the command line). The resulting "compressed" images are sent to standard resolution devices, and I presume these are what end up in the mobi7/mobi8 images folders on extraction. The original source images remain "uncompressed" (i.e. not resized in either resolution or file size), and these I assume are contained in the .azw3 file as well as the kindlegensrc.zip, deriving now from those stored in the HD CONTAINER if I understand correctly.

I've asked Aaron to post the file in question, but whether he will or not is up to him.

I don't really need the DumpMobiHeader script, as you've answered my question sufficiently. For practical purposes of content creation all I needed to know was where the images exist, and in what form, in order to control the file size for distribution.

Thanks!

KevinH · 05-26-2014, 04:03 PM

Software is not signal processing. If you pass valid input to a software program and it does not do as advertised, it is a bug that needs to be fixed, not something random that just happens. It will have an underlying cause.

So random or anomalous behaviour in a piece of software is a bug by its very definition.

Quote:

As for "compressed" versus "uncompressed" the answer should be obvious.

Kindlegen applies compression to input images (assuming you do not use the -C0 option in the command line). The resulting "compressed" images are sent to standard resolution devices, and I presume these are what end up in the mobi7/mobi8 images folders on extraction. The original source images remain "uncompressed" (i.e. not resized in either resolution or file size), and these I assume are contained in the .azw3 file as well as the kindlegensrc.zip, deriving now from those stored in the HD CONTAINER if I understand correctly.

Well actually based on your definition of "compression" above, it was not obvious at all. Any data can be losslessly "compressed" using a number of different data compression schemes that have nothing to do with image data, image size or image resolution. I thought you were talking about gzip, zip, huffman, etc compression schemes on the data and not downsampling and image conversion. The reason this is important is that the starting bytes in the Mobi sector are used to identify the image type. If the data itself is compressed using a compression scheme like zip, gzip, huffman, xc, etc, these bytes would not match any known for an image and would cause a problem in KindleUnpack.

EbokJunkie · 05-26-2014, 04:18 PM

Guys, I saw cover file (only cover) missing from mobi8 while present at mob7 folder when I tried to unpack azw3. That happened always with Python 2.7.2. Installed 2.7.6 and covers returned to mobi8.

R. Scot Johns · 05-26-2014, 04:20 PM

Okay, yeah, I follow you on the compression issue. From my pov there are simply two different versions of the same images being passed through Kindlegen, one set that is untouched, and the other that is modified. The use of the term compressed seemed obvious to me to describe what has happened to the smaller images, since Kindlegen used this term itself. But I see your point quite clearly. My bad.

But your error with regard to the anomaly of missing images is in assuming that this is occurring in the software, and thus rightly a bug. But that is an unknown. The pipeline follows a long and tortuous route long before it get to KindleUnpack, or even Kindlegen, for that matter. Using my previous example, if a sunspot causes a disruption of data flow during wireless transmission of a file from Amazon to my Kindle device, and that file is then unpacked, this is not a bug, but an anomaly, caused by a completely random event that cannot be replicated. But as we do not know the cause of data loss, it may be a bug in the software, or maybe the images were never in there in the first place.

AaronShep · 05-26-2014, 05:28 PM

I've posted the requested sample file at this location:

http://www.newselfpublishing.com/private/MouseDeer.mobi

This is a preview file downloaded direct from Amazon KDP. When unpacked with KindleUnpack 0.65 (AppleScript version), the folder mobi8 > OEBPS > Images is empty. The same folder is empty in the output from unpacking the AZW3 file.

Also, when the AZW3 file is viewed, the images disappear about a third of the way into the book, replaced by empty boxes. I found these same results whether viewing in Previewer, Kindle for Mac, or Fire HD. By contrast, all images display as expected when viewing the preview file before unpacking.

When a preview file was generated for the same book in Previewer 2.922 on the desktop and unpacked, the same image folders were empty. However, the AZW3 file showed the images as expected when viewing.

I'm on Mac OS X 10.6.8 with Python 2.7.6.

R. Scot Johns · 05-26-2014, 05:44 PM

Okay, Aaron, I've got images in both folders when extracted, and the downloaded file views correctly throughout. But I am now seeing the issue with the AZW3 file showing only borders, beginning with "Tiger ran away" about 9 pages in.

Moreover, double-tapping on the missing image area does nothing, so no file is being referenced at all. Also, the cover does not appear in the carousel with the azw3 version, but does for the complete file.

Extracting the AZW3 produces images in both folders, but only 11 of them, ending with the last one that I saw in the opened book. I also got this error message from KindleUnpack:

Error: Referenced image 12 in style url was not recognized in <img style="width: 100%; height: auto;" width="1440" height="432" id="x0000_i1074" src="kindle:embed:000C?mime=image/jpg" alt=""/>

And then the same error for all the images after that.

AaronShep · 05-26-2014, 05:58 PM

So, it looks like I need to run the script directly to see error messages. Who woulda thunk.

KevinH · 05-26-2014, 06:04 PM

Hi,

Quote:

Originally Posted by AaronShep

I've posted the requested sample file at this location:

http://www.newselfpublishing.com/private/MouseDeer.mobi

This is a preview file downloaded direct from Amazon KDP. When unpacked with KindleUnpack 0.65 (AppleScript version), the folder mobi8 > OEBPS > Images is empty. The same folder is empty in the output from unpacking the AZW3 file.

Also, when the AZW3 file is viewed, the images disappear about a third of the way into the book, replaced by empty boxes. I found these same results whether viewing in Previewer, Kindle for Mac, or Fire HD. By contrast, all images display as expected when viewing the preview file before unpacking.

When a preview file was generated for the same book in Previewer 2.922 on the desktop and unpacked, the same image folders were empty. However, the AZW3 file showed the images as expected when viewing.

I'm on Mac OS X 10.6.8 with Python 2.7.6.

Thanks! Okay, this is one of those newly generated file types that has an additional HD Container used for some images.

Here is some partial output of DumpMobiHeader_v014.py

Code:

Map of Palm DB Sections
    Dec  - Hex : Description
    ---- - ----  -----------
    0000 - 0000: HEADER 5 [8828]
    0001 - 0001: Text Record 0 [1820]
    0002 - 0002: Text Record 1 [1760]
    0003 - 0003: Text Record 2 [1714]
    0004 - 0004: Text Record 3 [1752]
    0005 - 0005: Text Record 4 [1495]
    0006 - 0006: 000000  [3]
    0007 - 0007: Image jpeg [113924]
    0008 - 0008: Image gif [112]
    0009 - 0009: Image gif [112]
    0010 - 000a: Image jpeg [108380]
    0011 - 000b: Image jpeg [71264]
    0012 - 000c: Image gif [29276]
    0013 - 000d: Image gif [24268]
    0014 - 000e: Image jpeg [17764]
    0015 - 000f: Image jpeg [112972]
    0016 - 0010: Image jpeg [23224]
    0017 - 0011: Image jpeg [106344]
    0018 - 0012: Image jpeg [24176]
    0019 - 0013: Image jpeg [109668]
    0020 - 0014: Image jpeg [68288]
    0021 - 0015: Image gif [112]
    0022 - 0016: Image jpeg [114344]
    0023 - 0017: Image jpeg [32340]
    0024 - 0018: Image jpeg [14144]
    0025 - 0019: Image jpeg [118560]
    0026 - 001a: Image jpeg [75340]
    0027 - 001b: Image gif [112]
    0028 - 001c: Image jpeg [106580]
    0029 - 001d: Image jpeg [41892]
    0030 - 001e: Image jpeg [117484]
    0031 - 001f: Image jpeg [22640]
    0032 - 0020: Image jpeg [117996]
    0033 - 0021: Image jpeg [71124]
    0034 - 0022: Image jpeg [107724]
    0035 - 0023: RESC [4112]
    0036 - 0024: Image jpeg [23960]
    0037 - 0025: FLIS [36]
    0038 - 0026: FCIS [44]
    0039 - 0027: Source Archive 0 [7196227]
    0040 - 0028: Source Archive 1 [2012]
    0041 - 0029: BOUNDARY [8]
    0042 - 002a: HEADER 8 [8828]
    0043 - 002b: Text Record 0 [1905]
    0044 - 002c: Text Record 1 [1751]
    0045 - 002d: Text Record 2 [1841]
    0046 - 002e: Text Record 3 [1731]
    0047 - 002f: Text Record 4 [1889]
    0048 - 0030: Text Record 5 [1941]
    0049 - 0031: Text Record 6 [1202]
    0050 - 0032: Text Record 7 [1042]
    0051 - 0033: 0000  [2]
    0052 - 0034: Fragment Index 0 [248]
    0053 - 0035: Fragment Index 1 [388]
    0054 - 0036: Fragment Index CNX [48]
    0055 - 0037: Skeleton Index 0 [244]
    0056 - 0038: Skeleton Index_Index 1 [224]
    0057 - 0039: Guide Index 0 [232]
    0058 - 003a: Guide Index 1 [212]
    0059 - 003b: Guide Index CNX [12]
    0060 - 003c: FDST [28]
    0061 - 003d: FLIS [36]
    0062 - 003e: FCIS [44]
    0063 - 003f: DATP [320]
    0064 - 0040: BOUNDARY [8]
    0065 - 0041: CONT [2296]
Container EXTH Dump

    Key: "K8_Count_of_Resources_Fonts_Images_(125)"
        Value: 0x000f

    Key: "Creator_Software_(204)"
        Value: 0x00c9

    Key: "Creator_Major_Version_(205)"
        Value: 0x0002

    Key: "Creator_Minor_Version_(206)"
        Value: 0x0009

    Key: "Kindlegen_BuildRev_Number_(535)"
        Value: "0730-890adc2"

    Key: "Creator_Build_Number_(207)"
        Value: 0x0000

    Key: "Mimetype_(539)"
        Value: "application/image"

    Key: "Image_Size_(538)"
        Value: "2400x3840"

    Key: "Unknown_(542)"
        Value: "jj4s"

    Key: "Unknown_(543)"
        Value: "HD_CONTAINER"
    0066 - 0042: CRES [483872]
    0067 - 0043: Empty_Image/Resource_Placeholder [4]
    0068 - 0044: Empty_Image/Resource_Placeholder [4]
    0069 - 0045: CRES [610392]
    0070 - 0046: CRES [169152]
    0071 - 0047: Empty_Image/Resource_Placeholder [4]
    0072 - 0048: Empty_Image/Resource_Placeholder [4]
    0073 - 0049: Empty_Image/Resource_Placeholder [4]
    0074 - 004a: CRES [405920]
    0075 - 004b: Empty_Image/Resource_Placeholder [4]
    0076 - 004c: CRES [399504]
    0077 - 004d: Empty_Image/Resource_Placeholder [4]
    0078 - 004e: CRES [455124]
    0079 - 004f: CRES [171112]
    0080 - 0050: Empty_Image/Resource_Placeholder [4]
    0081 - 0051: CRES [498860]
    0082 - 0052: Empty_Image/Resource_Placeholder [4]
    0083 - 0053: Empty_Image/Resource_Placeholder [4]
    0084 - 0054: CRES [335956]
    0085 - 0055: CRES [178168]
    0086 - 0056: Empty_Image/Resource_Placeholder [4]
    0087 - 0057: CRES [631892]
    0088 - 0058: Empty_Image/Resource_Placeholder [4]
    0089 - 0059: CRES [406732]
    0090 - 005a: Empty_Image/Resource_Placeholder [4]
    0091 - 005b: CRES [451288]
    0092 - 005c: CRES [170860]
    0093 - 005d: CRES [1590268]
    0094 - 005e: kindle:embed:0001?mime=image/jpg|kindle:embed:0004?mime=image/jpg|kindle:embed:0005?mime=image/jpg|kindle:embed:0009?mime=image/jpg|kindle:embed:000B?mime=image/jpg|kindle:embed:000D?mime=image/jpg|kindle:embed:000E?mime=image/jpg|kindle:embed:000G?mime=image/jpg|kindle:embed:000J?mime=image/jpg|kindle:embed:000K?mime=image/jpg|kindle:embed:000M?mime=image/jpg|kindle:embed:000O?mime=image/jpg|kindle:embed:000Q?mime=image/jpg|kindle:embed:000R?mime=image/jpg|kindle:embed:000S?mime=image/jpg| [495]
    0095 - 005f: CONTBOUNDARY [12]
    0096 - 0060: EOF_RECORD [4]

The original old mobi typically stores all images here:

0007 - 0007: Image jpeg [113924]
0008 - 0008: Image gif [112]
0009 - 0009: Image gif [112]
0010 - 000a: Image jpeg [108380]
0011 - 000b: Image jpeg [71264]
0012 - 000c: Image gif [29276]
0013 - 000d: Image gif [24268]
0014 - 000e: Image jpeg [17764]
0015 - 000f: Image jpeg [112972]
0016 - 0010: Image jpeg [23224]
0017 - 0011: Image jpeg [106344]
0018 - 0012: Image jpeg [24176]
0019 - 0013: Image jpeg [109668]
0020 - 0014: Image jpeg [68288]
0021 - 0015: Image gif [112]
0022 - 0016: Image jpeg [114344]
0023 - 0017: Image jpeg [32340]
0024 - 0018: Image jpeg [14144]
0025 - 0019: Image jpeg [118560]
0026 - 001a: Image jpeg [75340]
0027 - 001b: Image gif [112]
0028 - 001c: Image jpeg [106580]
0029 - 001d: Image jpeg [41892]
0030 - 001e: Image jpeg [117484]
0031 - 001f: Image jpeg [22640]
0032 - 0020: Image jpeg [117996]
0033 - 0021: Image jpeg [71124]
0034 - 0022: Image jpeg [107724]
0036 - 0024: Image jpeg [23960]

including the cover image, thumbnail, that makes 29 image files.

The new mobi8 (kf8) part does not store any images at all but just references the mobi7 image sectors in the palm file.

Then comes one of the new CONTAINERs with the HD Image files and their own metadata EXTH data:

0064 - 0040: BOUNDARY [8]
0065 - 0041: CONT [2296]

Container EXTH Dump

Key: "K8_Count_of_Resources_Fonts_Images_(125)"
Value: 0x000f

Key: "Creator_Software_(204)"
Value: 0x00c9

Key: "Creator_Major_Version_(205)"
Value: 0x0002

Key: "Creator_Minor_Version_(206)"
Value: 0x0009

Key: "Kindlegen_BuildRev_Number_(535)"
Value: "0730-890adc2"

Key: "Creator_Build_Number_(207)"
Value: 0x0000

Key: "Mimetype_(539)"
Value: "application/image"

Key: "Image_Size_(538)"
Value: "2400x3840"

Key: "Unknown_(542)"
Value: "jj4s"

Key: "Unknown_(543)"
Value: "HD_CONTAINER"

0066 - 0042: CRES [483872]
0067 - 0043: Empty_Image/Resource_Placeholder [4]
0068 - 0044: Empty_Image/Resource_Placeholder [4]
0069 - 0045: CRES [610392]
0070 - 0046: CRES [169152]
0071 - 0047: Empty_Image/Resource_Placeholder [4]
0072 - 0048: Empty_Image/Resource_Placeholder [4]
0073 - 0049: Empty_Image/Resource_Placeholder [4]
0074 - 004a: CRES [405920]
0075 - 004b: Empty_Image/Resource_Placeholder [4]
0076 - 004c: CRES [399504]
0077 - 004d: Empty_Image/Resource_Placeholder [4]
0078 - 004e: CRES [455124]
0079 - 004f: CRES [171112]
0080 - 0050: Empty_Image/Resource_Placeholder [4]
0081 - 0051: CRES [498860]
0082 - 0052: Empty_Image/Resource_Placeholder [4]
0083 - 0053: Empty_Image/Resource_Placeholder [4]
0084 - 0054: CRES [335956]
0085 - 0055: CRES [178168]
0086 - 0056: Empty_Image/Resource_Placeholder [4]
0087 - 0057: CRES [631892]
0088 - 0058: Empty_Image/Resource_Placeholder [4]
0089 - 0059: CRES [406732]
0090 - 005a: Empty_Image/Resource_Placeholder [4]
0091 - 005b: CRES [451288]
0092 - 005c: CRES [170860]
0093 - 005d: CRES [1590268]

It has slots for all 29 images with placeholders for non-HD images and actual data for the full HD ones.

Next comes the kindle internal link and mimetype for all of images in the HD container (there are 15 of them) in a pipe delimited string that I have broken into separate values. My guess it is these images that you are missing.

0094 - 005e:
kindle:embed:0001?mime=image/jpg
kindle:embed:0004?mime=image/jpg
kindle:embed:0005?mime=image/jpg
kindle:embed:0009?mime=image/jpg
kindle:embed:000B?mime=image/jpg
kindle:embed:000D?mime=image/jpg
kindle:embed:000E?mime=image/jpg
kindle:embed:000G?mime=image/jpg
kindle:embed:000J?mime=image/jpg
kindle:embed:000K?mime=image/jpg
kindle:embed:000M?mime=image/jpg
kindle:embed:000O?mime=image/jpg
kindle:embed:000Q?mime=image/jpg
kindle:embed:000R?mime=image/jpg
kindle:embed:000S?mime=image/jpg
| [495]
0095 - 005f: CONTBOUNDARY [12]

But that still does not explain why the low res images stored in the old mobi were not properly copied to the azw3 file so that is a bug.

So can all azw3 capable readers support "2400x3840" sized HD images. Should the unpacking of azw3 default to using the new HD containers or would that break things?

KevinH

DiapDealer · 05-26-2014, 06:20 PM

Quote:

Originally Posted by AaronShep

This is a preview file downloaded direct from Amazon KDP. When unpacked with KindleUnpack 0.65 (AppleScript version), the folder mobi8 > OEBPS > Images is empty. The same folder is empty in the output from unpacking the AZW3 file.

That folder (mobi8 > OEBPS > Images) has 28 images in it when I unpack the sample you provided with KindleUpack v0.65 and the latest v0.66 (that doesn't seem to be linked in the first post of the thread).

The extraction process seems to be happening correctly and all images are present (linked and viewable) in the source-code produced from the azw3 portion of the original file by KindleUnpack.

I am, however, seeing the same problem others are mentioning with regard to images being missing from the azw3 file produced when "splitting" the Kindlegen/KDP-produced dual-format file (with both v0.65 and v0.66) into its component MOBI/AZW3 parts.

I'm guessing something in the a newer version of KindlePreviewer has broken something in KindleUnpack's Splitter function (EDIT: as verified by KevinH above).

I've no idea why the Images folder is empty when Aaron attempts to unpack.

DiapDealer · 05-26-2014, 06:36 PM

Quote:

Originally Posted by KevinH

So can all azw3 capable readers support "2400x3840" sized HD images. Should the unpacking of azw3 default to using the new HD containers or would that break things?

I've got a Kindle Keyboard (which should be the oldest device supporting azw3) I could test it on, but I doubt it will support that size of an image. I suspect we might be up against a situation where Amazon sends different AZW3s to different devices based on screen resolution (newer Fires get the HD images while older tablets & eInk devices get the lower resolution version). Have we determined if the HD container exists in standalone or retail AZW3s? Or is it only a part of the newest multipurpose KindlePreviewer/Kindlegen output?

AaronShep · 05-26-2014, 06:55 PM

I believe only the HD Fires use the larger images. I know that the iOS Kindles don't, and Scot says the older Fires don't, either. Whether the other KF8 Kindles can handle the images if sent there alone or are allowed to read them, I don't know and have no way to test.

Possibly I'm not getting any images in my mobi8 folders because of the script error messages interacting with the AppleScript container. It may be aborting something before any images are deposited. If so, that would clear up by itself if the other issue is resolved.

This is not a problem in the newest version of Previewer, because I got the same results in the older 2.9.

KevinH · 05-26-2014, 07:59 PM

Hi,

Yes the KDP generated file has first_content and last_content fields in the Old Mobi header as set to 0xffff (offsets are 0xc0 [Halfword in size] and 0xc2 [Halfword in size].

The fallback code to find the first and last resources when this happens is complete rubbish.

I am in the process of fixing it now so that splitting should now work properly again.

Damn Amazon can't even keep its own fields consistent across its own tools!
One tool does it one way and another tool does it differently.

Anyway, I should have a fix soon.

Kevin

Quote:

Originally Posted by DiapDealer

I am, however, seeing the same problem others are mentioning with regard to images being missing from the azw3 file produced when "splitting" the Kindlegen/KDP-produced dual-format file (with both v0.65 and v0.66) into its component MOBI/AZW3 parts.

I'm guessing something in the a newer version of KindlePreviewer has broken something in KindleUnpack's Splitter function (EDIT: as verified by KevinH above).

I've no idea why the Images folder is empty when Aaron attempts to unpack.

KevinH · 05-26-2014, 08:09 PM

Hi,

Okay, I think I have fixed the fallback code to work properly but I don't have a real Kindle Device to test with.

So please unzip and replace the mobi_split.py file in KindleUnpack/lib/ with the new version I have attached here and give it a test and let me know if it works now for you.

I have also attached the unified diffs in case anyone want to see what I changed.

If it works, I will add it to an upcoming KindleUnpack v67 release along with tkeo's other v66a fixes.

I will keep the experimental epub3 code until after I can give it a full review when I have more time.

Thanks,

Kevin

KevinH · 05-26-2014, 08:25 PM

Quote:

Originally Posted by R. Scot Johns

But your error with regard to the anomaly of missing images is in assuming that this is occurring in the software, and thus rightly a bug. But that is an unknown. The pipeline follows a long and tortuous route long before it get to KindleUnpack, or even Kindlegen, for that matter.

... which is why I said given proper input. ;-)

This was and is a bug. Hopefully a fully squashed one. It comes from some Kindle tools putting incorrect information inside headers whereas other tools do not. The concept of needing a fallback when the required information could not be found was present in the software but was never needed before and therefore never tested and turned out to be rubbish.

Quote:

Using my previous example, if a sunspot causes a disruption of data flow during wireless transmission of a file from Amazon to my Kindle device, and that file is then unpacked, this is not a bug, but an anomaly, caused by a completely random event that cannot be replicated.

No, it is just a bug in the tcip packet checksum and error detection code of the underlying operating system. ;-)

Take care,

KevinH

AaronShep · 05-26-2014, 08:29 PM

The AZW3 file now displays properly. Thanks!

Still getting empty mobi8 image folders when using the AppleScript.

05-26-2014, 05:44 PM	#741
R. Scot Johns Author/Illustrator Posts: 14 Karma: 2952 Join Date: Mar 2012 Location: Boise, ID Device: iPad 2 & 3, Kindle Paperwhite, Kindle Fire 1 & 2, HD7 & HD8.9, RazrMax	Okay, Aaron, I've got images in both folders when extracted, and the downloaded file views correctly throughout. But I am now seeing the issue with the AZW3 file showing only borders, beginning with "Tiger ran away" about 9 pages in. Moreover, double-tapping on the missing image area does nothing, so no file is being referenced at all. Also, the cover does not appear in the carousel with the azw3 version, but does for the complete file. Extracting the AZW3 produces images in both folders, but only 11 of them, ending with the last one that I saw in the opened book. I also got this error message from KindleUnpack: Error: Referenced image 12 in style url was not recognized in <img style="width: 100%; height: auto;" width="1440" height="432" id="x0000_i1074" src="kindle:embed:000C?mime=image/jpg" alt=""/> And then the same error for all the images after that. Last edited by R. Scot Johns; 05-26-2014 at 05:50 PM. Reason: Additional testing

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Can i rotate text and insert images in Mobi and EPUB?	JanGLi	Kindle Formats	5	02-02-2013 04:16 PM
PDF to Mobi with text and images	pocketsprocket	Kindle Formats	7	05-21-2012 07:06 AM
Mobi files - images	DWC	Introduce Yourself	5	07-06-2011 01:43 AM
pdf to mobi... creating images rather than text	Dumhed	Calibre	5	11-06-2010 12:08 PM
Transfer of images on text files	anirudh215	PDF	2	06-22-2009 09:28 AM

05-26-2014, 03:23 PM	#736
R. Scot Johns Author/Illustrator Posts: 14 Karma: 2952 Join Date: Mar 2012 Location: Boise, ID Device: iPad 2 & 3, Kindle Paperwhite, Kindle Fire 1 & 2, HD7 & HD8.9, RazrMax	An anomaly is an unknown deviation from the common expectation, and can be caused by any number of random factors along the chain, such as a signal fluctuation during transmission that causes a loss of data, or a user error. It does not inherently imply a bug exists, since a bug is something which generally has a consistent cause that can be repeated, traced and fixed. I use the term anomaly in the sense that there is an unknown variable, which is why I asked if anyone else has seen it occur. Apparently the answer to that question is no, which makes it even more of an anomaly than before. Of course, I'm inherently a philosopher, not a programmer, so I tend to use terms differently than others do quite often. As for "compressed" versus "uncompressed" the answer should be obvious. Kindlegen applies compression to input images (assuming you do not use the -C0 option in the command line). The resulting "compressed" images are sent to standard resolution devices, and I presume these are what end up in the mobi7/mobi8 images folders on extraction. The original source images remain "uncompressed" (i.e. not resized in either resolution or file size), and these I assume are contained in the .azw3 file as well as the kindlegensrc.zip, deriving now from those stored in the HD CONTAINER if I understand correctly. I've asked Aaron to post the file in question, but whether he will or not is up to him. I don't really need the DumpMobiHeader script, as you've answered my question sufficiently. For practical purposes of content creation all I needed to know was where the images exist, and in what form, in order to control the file size for distribution. Thanks!

05-26-2014, 04:18 PM	#738
EbokJunkie Addict Posts: 230 Karma: 13495 Join Date: Feb 2009 Location: SoCal Device: Kindle 3, Kindle PW, Pocketbook 301+, Pocketbook Touch, Sony 950, 350	Guys, I saw cover file (only cover) missing from mobi8 while present at mob7 folder when I tried to unpack azw3. That happened always with Python 2.7.2. Installed 2.7.6 and covers returned to mobi8.

05-26-2014, 04:20 PM	#739
R. Scot Johns Author/Illustrator Posts: 14 Karma: 2952 Join Date: Mar 2012 Location: Boise, ID Device: iPad 2 & 3, Kindle Paperwhite, Kindle Fire 1 & 2, HD7 & HD8.9, RazrMax	Okay, yeah, I follow you on the compression issue. From my pov there are simply two different versions of the same images being passed through Kindlegen, one set that is untouched, and the other that is modified. The use of the term compressed seemed obvious to me to describe what has happened to the smaller images, since Kindlegen used this term itself. But I see your point quite clearly. My bad. But your error with regard to the anomaly of missing images is in assuming that this is occurring in the software, and thus rightly a bug. But that is an unknown. The pipeline follows a long and tortuous route long before it get to KindleUnpack, or even Kindlegen, for that matter. Using my previous example, if a sunspot causes a disruption of data flow during wireless transmission of a file from Amazon to my Kindle device, and that file is then unpacked, this is not a bug, but an anomaly, caused by a completely random event that cannot be replicated. But as we do not know the cause of data loss, it may be a bug in the software, or maybe the images were never in there in the first place.

05-26-2014, 05:28 PM	#740
AaronShep Connoisseur Posts: 56 Karma: 3274 Join Date: Dec 2011 Device: iPad	I've posted the requested sample file at this location: http://www.newselfpublishing.com/private/MouseDeer.mobi This is a preview file downloaded direct from Amazon KDP. When unpacked with KindleUnpack 0.65 (AppleScript version), the folder mobi8 > OEBPS > Images is empty. The same folder is empty in the output from unpacking the AZW3 file. Also, when the AZW3 file is viewed, the images disappear about a third of the way into the book, replaced by empty boxes. I found these same results whether viewing in Previewer, Kindle for Mac, or Fire HD. By contrast, all images display as expected when viewing the preview file before unpacking. When a preview file was generated for the same book in Previewer 2.922 on the desktop and unpacked, the same image folders were empty. However, the AZW3 file showed the images as expected when viewing. I'm on Mac OS X 10.6.8 with Python 2.7.6.

05-26-2014, 05:58 PM	#742
AaronShep Connoisseur Posts: 56 Karma: 3274 Join Date: Dec 2011 Device: iPad	So, it looks like I need to run the script directly to see error messages. Who woulda thunk.

05-26-2014, 06:55 PM	#746
AaronShep Connoisseur Posts: 56 Karma: 3274 Join Date: Dec 2011 Device: iPad	I believe only the HD Fires use the larger images. I know that the iOS Kindles don't, and Scot says the older Fires don't, either. Whether the other KF8 Kindles can handle the images if sent there alone or are allowed to read them, I don't know and have no way to test. Possibly I'm not getting any images in my mobi8 folders because of the script error messages interacting with the AppleScript container. It may be aborting something before any images are deposited. If so, that would clear up by itself if the other issue is resolved. This is not a problem in the newest version of Previewer, because I got the same results in the older 2.9.

05-26-2014, 08:29 PM	#750
AaronShep Connoisseur Posts: 56 Karma: 3274 Join Date: Dec 2011 Device: iPad	The AZW3 file now displays properly. Thanks! Still getting empty mobi8 image folders when using the AppleScript.

Advert

Advert