[GUI Plugin] KindleUnpack - The Plugin - Page 29

DiapDealer · 02-01-2020, 08:37 AM

At first glance, it appears like an embedded image href of that nature should already be dealt with correctly (from a regex point of view). I'll have to take a closer look. But that's going to be in KindleUnpack itself rather than the plugin.

Quote:

Originally Posted by thiago.eec

Hi, @DiapDealer.

Would it be possible for KindleUnpack auto-correct the cover links (NCX and inline TOC) when converting a KF8 to EPUB?

Generally, it comes like this:

Code:

<a href="kindle:embed:0005?mime=image/jpg">Cover</a>

DiapDealer · 02-01-2020, 02:34 PM

The problem with links like this is that there's nothing to "correct." KindleBook covers are not "pages" as such. They're just typically images and nothing else. As such, if the hrefs were indeed "correct," they would still be illegal epub hrefs insofar as they point directly to images. Which is not supported in epub. There's no good way to handle this situation without inventing pages to display these images (which is not likely something that KU is going to do). So even if they were "correct" they'd still need to be fixed before they were usable in an epub. It's a bit of a damned-if-you-do-damned-if you-don't situation.

In all likelihood, kindlebooks that contain these kinds of links after running KindleUnpack on them were probably not actually made from epubs to begin with. We try to make it very clear that KindleUnpack's KF8 to Epub "conversion" is only an approximation. If it (the KindleBook) wasn't made from an epub, the resulting epub is not guaranteed to be valid.

The bottom line is the the inline html toc and the ncx both need hrefs to pages not images. And if there was no page in the kindlebook, we really can't go making one up.

thiago.eec · 02-01-2020, 04:27 PM

Quote:

Originally Posted by DiapDealer

The problem with links like this is that there's nothing to "correct." KindleBook covers are not "pages" as such. They're just typically images and nothing else. As such, if the hrefs were indeed "correct," they would still be illegal epub hrefs insofar as they point directly to images. Which is not supported in epub. There's no good way to handle this situation without inventing pages to display these images (which is not likely something that KU is going to do). So even if they were "correct" they'd still need to be fixed before they were usable in an epub. It's a bit of a damned-if-you-do-damned-if you-don't situation.

In all likelihood, kindlebooks that contain these kinds of links after running KindleUnpack on them were probably not actually made from epubs to begin with. We try to make it very clear that KindleUnpack's KF8 to Epub "conversion" is only an approximation. If it (the KindleBook) wasn't made from an epub, the resulting epub is not guaranteed to be valid.

The bottom line is the the inline html toc and the ncx both need hrefs to pages not images. And if there was no page in the kindlebook, we really can't go making one up.

Thanks for the reply.

In my files, they all have a cover page, although that might not be the case for every book. Also, KU would have to figure out which page is the cover.

Thats ok, it is very simple to correct this, anyway. Thanks!

DiapDealer · 02-01-2020, 07:40 PM

I'm not ruling out doing something about it. I'm just nor sure what, yet.

bookandtea · 04-29-2020, 01:03 AM

I'm using the latest version of Calibre, and I have the tools from Apprentice Alf's blog (here) downloaded and installed.

Tonight I downloaded a book in AZW4 format. I'd like to change that to PDF or epub so I can read it on my iBooks app. I downloaded and successfully installed KindleUnpack.

When I select the book, and click the KindleUnpack button along the toolbar, there's a red lock and the menu is greyed out. Now, in the previous response to someone else a few pages back, your instruction was to download the DRM removal tool - but I've already done that, I use those tools regularly. What else could be going wrong?

pdurrant · 04-29-2020, 04:22 AM

Quote:

Originally Posted by bookandtea

I'm using the latest version of Calibre, and I have the tools from Apprentice Alf's blog (here) downloaded and installed.

Tonight I downloaded a book in AZW4 format. I'd like to change that to PDF or epub so I can read it on my iBooks app. I downloaded and successfully installed KindleUnpack.

When I select the book, and click the KindleUnpack button along the toolbar, there's a red lock and the menu is greyed out. Now, in the previous response to someone else a few pages back, your instruction was to download the DRM removal tool - but I've already done that, I use those tools regularly. What else could be going wrong?

Clearly the DRM didn't get removed when you imported the book into calibre. For help on that, ask at Apprentice Alf's blog.

PoP · 07-06-2020, 08:18 AM

Intrigued by this post about Kindle Editions with Audio-Video ebooks, I have downloaded one. Expectedly, It gets delivered to K3, PW3, and K4PC in the AZW/MOBI format with no audio or video media files.

I Alf-Harpered it and was just curious to inspect the source html, but then got this error message from KindleUnpack plugin:

Click image for larger version

Name: KindleUnpack.png
Views: 243
Size: 59.3 KB
ID: 180435

Is this particular to that type of book or only a mundane error?

BTW the source seems to have been unpacked anyway and I can inspect it.

DiapDealer · 07-06-2020, 12:07 PM

That there is an error displaying an error! That shouldn't really happen regardless of what kind of book it is (though I've never downloaded one of the books you're talking about). I'll have to take a look at the area in question. Thanks for the info.

DiapDealer · 07-06-2020, 12:52 PM

Give this version a whirl and see if the error for that book is "properly" shown, now.

You still may get an error with that particular book, but it should be a different one. One that will hopefully give a better clue as to what the underlying unpacking engine doesn't like about it.

PoP · 07-06-2020, 01:51 PM

^

Exactly, a different error:

Code:

calibre, version 4.20.0
ERREUR : Exception non gérée: <b>AttributeError</b>:'unicode' object has no attribute 'write'

calibre 4.20 [64bit]  embedded-python: True is64bit: True
Windows-10-10.0.19041-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 2.7.16+
Windows: ('10', '10.0.19041', 'SP0', u'Multiprocessor Free')
Interface language: fr
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 33, 0) && DeDRM (6, 8, 0) && Package KFX (from KFX Input) (1, 33, 0) && Count Pages (1, 10, 0) && KFX metadata reader (from KFX Input) (1, 33, 0) && KFX Input (1, 33, 0) && KindleUnpack - The Plugin (0, 82, 2) && TTS to MP3 (0, 4, 0) && KindleXRay (0, 0, 3) && X-Ray Creator (3, 2, 1) && Obok DeDRM (6, 7, 0) && Hyphenate This! (0, 0, 9) && Set KFX metadata (from KFX Output) (1, 43, 0) && KFX Output (1, 43, 0) && APNX Generator (1, 1, 0) && EpubCheck (0, 2, 2)
Traceback (most recent call last):
  File "calibre_plugins.kindleunpack_plugin.action", line 269, in unpack_ebook
  File "calibre_plugins.kindleunpack_plugin.utilities", line 113, in showErrorDlg
  File "traceback.py", line 233, in print_exc
  File "traceback.py", line 124, in print_exception
  File "traceback.py", line 13, in _print
AttributeError: 'unicode' object has no attribute 'write'

BTW it happens on that $0 book.

DiapDealer · 07-06-2020, 06:18 PM

Ack! That's still not the error message I was hoping to get. I'm going to have to rethink my polyglot error dialog. It works for calibre-py3, but not calibre-py2. When I updated the plugin to accommodate both, I didn't check to see that error conditions worked for both versions of python. I'll get it figured out and post another test version.

Thanks for the free test case link!

DiapDealer · 07-06-2020, 09:57 PM

The image resources in that book are pretty messed up. I fixed the error dialog error that I inadvertently introduced with my changes to support both the Python 2 and the upcoming Python 3 versions of calibre. And I patched the kindleunpack core engine to test for cover thumbnail image indexes that are somehow out of range. I'll see about making that change permanent in the kindleunpack repository.

The bottom line is that even though you may have gotten the html unpacked, you might not have been getting all of the images from that book unpacked (and you still might not, given all the unrecognized image resources and sections not containing recognized resources that I see in the debug output).

Give this version a whirl on that book (same version number as the last test version ... sorry).

EDIT: Removed beta attachment since it's been incorporated in the official release (first page of the thread).

DNSB · 07-07-2020, 01:31 AM

Quote:

Originally Posted by DiapDealer

Give this version a whirl on that book (same version number as the last test version ... sorry).

This version unpacked the .mobi file without giving me a visible error. The same 40 images in the Images directory. Then I tried debug mode. Sheesh, it generated a mass of errors.

Spoiler:

Code:

stdout+stderr from file dialog helper: ['', '']
piped data from file dialog helper: ['O\xcc\x93Y\xfe\x9b\xf1l\xa4+\x02-\x98)q\xf4\x14P\x82\x9e2\x80\xe8b\x0e\x01\xbcA\xe4%\x96o', 'C:\\Users\\David\\Desktop\\Outies']
Palm DB type: BOOKMOBI, 91 sections.
Warning: Bad key, size, value combination detected in EXTH  406 16 0000000000000000
Unpacking a Mobipocket 7 book...
Processing Mobipocket 7 section of book...
Mobi Version: 7
Codec: utf-8
Title: His Holiness the Dalai Lama (Enhanced Edition)
EXTH Title: His Holiness The Dalai Lama (Enhanced Edition): A Message of Spiritual Wisdom
Palmdoc compression
Unpacking images, resources, fonts, etc
Failed to extract font: font00030 from section 30
Failed to extract font: font00031 from section 31
Failed to extract font: font00032 from section 32
Failed to extract font: font00033 from section 33
Failed to extract font: font00034 from section 34
Failed to extract font: font00035 from section 35
Failed to extract font: font00036 from section 36
Failed to extract font: font00037 from section 37
Extracting image: image00038.gif from section 38
Warning: Section 39 does not contain a recognised resource
Extracting image: image00040.jpeg from section 40
Extracting image: image00041.jpeg from section 41
Extracting image: image00042.jpeg from section 42
Warning: Section 43 does not contain a recognised resource
Extracting image: image00044.jpeg from section 44
Extracting image: image00045.gif from section 45
Extracting image: image00046.jpeg from section 46
Extracting image: image00047.jpeg from section 47
Extracting image: image00048.jpeg from section 48
Extracting image: image00049.jpeg from section 49
Extracting image: image00050.jpeg from section 50
Extracting image: image00051.jpeg from section 51
Extracting image: image00052.jpeg from section 52
Extracting image: image00053.jpeg from section 53
Extracting image: image00054.jpeg from section 54
Warning: Section 55 does not contain a recognised resource
Extracting image: image00056.jpeg from section 56
Extracting image: image00057.jpeg from section 57
Extracting image: image00058.jpeg from section 58
Extracting image: image00059.jpeg from section 59
Extracting image: image00060.jpeg from section 60
Extracting image: image00061.jpeg from section 61
Extracting image: image00062.jpeg from section 62
Extracting image: image00063.jpeg from section 63
Extracting image: image00064.gif from section 64
Warning: Section 65 does not contain a recognised resource
Extracting image: image00066.jpeg from section 66
Warning: Section 67 does not contain a recognised resource
Extracting image: image00068.jpeg from section 68
Warning: Section 69 does not contain a recognised resource
Extracting image: image00070.jpeg from section 70
Warning: Section 71 does not contain a recognised resource
Extracting image: image00072.jpeg from section 72
Extracting image: image00073.jpeg from section 73
Extracting image: image00074.jpeg from section 74
Warning: Section 75 does not contain a recognised resource
Extracting image: image00076.jpeg from section 76
Warning: Section 77 does not contain a recognised resource
Extracting image: image00078.jpeg from section 78
Extracting image: image00079.gif from section 79
Extracting image: cover00080.jpeg from section 80
Extracting image: image00081.jpeg from section 81
Extracting image: image00082.jpeg from section 82
Extracting image: image00083.jpeg from section 83
Extracting image: image00084.jpeg from section 84
Extracting image: image00085.jpeg from section 85
Extracting image: image00087.jpeg from section 87
Unpacking raw markup language
Write ncx
Find link anchors
Insert data into html
Insert hrefs into html
Remove empty anchors from html
Insert image references into html
Error: Referenced image 18 was not recognized as a valid image
Error: Referenced image 28 was not recognized as a valid image
Error: Referenced image 30 was not recognized as a valid image
Error: Referenced image 32 was not recognized as a valid image
Error: Referenced image 34 was not recognized as a valid image
Building an opf for mobi7/azw4.
Error: Cover image 50 was not recognized as a valid image
Number given for Cover Thumbnail is out of range: 57
Error: Cover Thumbnail image 57 was not recognized as a valid image
stdout+stderr from file dialog helper: ['', '']
piped data from file dialog helper: ['\xc7\xd5\x9fj\xaa\xbf\xae\xd3\x99{/,\xc6C\xa9\x90\xbb\xacG{3\xefoR@\xb1\x12N_\xd3n\xa9', 'C:\\Users\\David\\Desktop\\Outies']
Palm DB type: BOOKMOBI, 91 sections.
Warning: Bad key, size, value combination detected in EXTH  406 16 0000000000000000

Also I modified line 35 in action.py from:

Code:

    dont_add_to = frozenset(['menubar-device', 'toolbar-device', 'context-menu-device'])

to:

Code:

    dont_add_to = frozenset(['context-menu-device'])

and I can add it to the main toolbar with device connected. Thanks again for that bit of advice.

DiapDealer · 07-07-2020, 06:15 AM

Yeah, that's the same output I was seeing. That book's a bit of a mess. Clearly the process to strip out the multi-media resources and present this thing as a mobi is a bit less thorough than it could be.

Thanks for reminding me about the menu thing. I'll incorporate that before officially releasing v0.82.2 of the plugin, too.

PoP · 07-07-2020, 06:44 AM

Quote:

Originally Posted by DiapDealer

...
Give this version a whirl on that book (same version number as the last test version ... sorry).

Quote:

Originally Posted by DNSB

This version unpacked the .mobi file without giving me a visible error.
....

Concurred

Thanks, a greatly useful plugin.

07-06-2020, 08:18 AM	#427
PoP curly᷂͓̫̙᷊̥̮̾ͯͤͭͬͦͨ ʎʌɹnɔ Posts: 3,002 Karma: 50506927 Join Date: Dec 2010 Location: ♁ ᴺ₄₅°₃₀' ᵂ₇₃°₃₇' ±₆₀" Device: K3₃.₄.₃ PW3&4₅.₁₃.₃	unicode argument expected Intrigued by this post about Kindle Editions with Audio-Video ebooks, I have downloaded one. Expectedly, It gets delivered to K3, PW3, and K4PC in the AZW/MOBI format with no audio or video media files. I Alf-Harpered it and was just curious to inspect the source html, but then got this error message from KindleUnpack plugin: Is this particular to that type of book or only a mundane error? BTW the source seems to have been unpacked anyway and I can inspect it. Last edited by PoP; 07-09-2020 at 02:37 PM. Reason: fixed link

07-06-2020, 12:52 PM	#429
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Give this version a whirl and see if the error for that book is "properly" shown, now. You still may get an error with that particular book, but it should be a different one. One that will hopefully give a better clue as to what the underlying unpacking engine doesn't like about it. Last edited by DiapDealer; 07-06-2020 at 09:48 PM. Reason: Remove attachment

07-06-2020, 09:57 PM	#432
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	The image resources in that book are pretty messed up. I fixed the error dialog error that I inadvertently introduced with my changes to support both the Python 2 and the upcoming Python 3 versions of calibre. And I patched the kindleunpack core engine to test for cover thumbnail image indexes that are somehow out of range. I'll see about making that change permanent in the kindleunpack repository. The bottom line is that even though you may have gotten the html unpacked, you might not have been getting all of the images from that book unpacked (and you still might not, given all the unrecognized image resources and sections not containing recognized resources that I see in the debug output). Give this version a whirl on that book (same version number as the last test version ... sorry). EDIT: Removed beta attachment since it's been incorporated in the official release (first page of the thread). Last edited by DiapDealer; 09-25-2020 at 11:36 AM. Reason: Remove beta attachment

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] EpubSplit	JimmXinu	Plugins	426	04-26-2024 06:46 PM
[GUI Plugin] EpubMerge	JimmXinu	Plugins	522	04-01-2024 10:25 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 08:39 AM
[GUI Plugin] Ex Libris	AlPe	Plugins	87	10-10-2020 03:10 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

02-01-2020, 02:34 PM	#422
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	The problem with links like this is that there's nothing to "correct." KindleBook covers are not "pages" as such. They're just typically images and nothing else. As such, if the hrefs were indeed "correct," they would still be illegal epub hrefs insofar as they point directly to images. Which is not supported in epub. There's no good way to handle this situation without inventing pages to display these images (which is not likely something that KU is going to do). So even if they were "correct" they'd still need to be fixed before they were usable in an epub. It's a bit of a damned-if-you-do-damned-if you-don't situation. In all likelihood, kindlebooks that contain these kinds of links after running KindleUnpack on them were probably not actually made from epubs to begin with. We try to make it very clear that KindleUnpack's KF8 to Epub "conversion" is only an approximation. If it (the KindleBook) wasn't made from an epub, the resulting epub is not guaranteed to be valid. The bottom line is the the inline html toc and the ncx both need hrefs to pages not images. And if there was no page in the kindlebook, we really can't go making one up.

02-01-2020, 07:40 PM	#424
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	I'm not ruling out doing something about it. I'm just nor sure what, yet.

04-29-2020, 01:03 AM	#425
bookandtea Mint Magic Posts: 22 Karma: 10 Join Date: Oct 2018 Device: Kobo Forma, Kindle Oasis	I'm using the latest version of Calibre, and I have the tools from Apprentice Alf's blog (here) downloaded and installed. Tonight I downloaded a book in AZW4 format. I'd like to change that to PDF or epub so I can read it on my iBooks app. I downloaded and successfully installed KindleUnpack. When I select the book, and click the KindleUnpack button along the toolbar, there's a red lock and the menu is greyed out. Now, in the previous response to someone else a few pages back, your instruction was to download the DRM removal tool - but I've already done that, I use those tools regularly. What else could be going wrong?

07-06-2020, 12:07 PM	#428
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	That there is an error displaying an error! That shouldn't really happen regardless of what kind of book it is (though I've never downloaded one of the books you're talking about). I'll have to take a look at the area in question. Thanks for the info.

07-06-2020, 06:18 PM	#431
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Ack! That's still not the error message I was hoping to get. I'm going to have to rethink my polyglot error dialog. It works for calibre-py3, but not calibre-py2. When I updated the plugin to accommodate both, I didn't check to see that error conditions worked for both versions of python. I'll get it figured out and post another test version. Thanks for the free test case link!

07-07-2020, 06:15 AM	#434
DiapDealer Grand Sorcerer Posts: 27,572 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Yeah, that's the same output I was seeing. That book's a bit of a mess. Clearly the process to strip out the multi-media resources and present this thing as a mobi is a bit less thorough than it could be. Thanks for reminding me about the menu thing. I'll incorporate that before officially releasing v0.82.2 of the plugin, too.

Advert

Advert