Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 10-30-2014, 11:58 AM   #1096
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi DiapDealer and tkeo,

I pushed the fix to both master and python2and3 branches. It will be in the next release once I get some free time to incorporate tkeo's audio and video unpacking code into both the master and python2and3 branches.

tkeo if you want me to push a patch that will make using Furigana metadata under auto indicate a target of epub 3, please provide patches for both branches if possible.

DiapDealer, please tell the bug reporter thanks for reporting the bug!

Take care,

KevinH

Last edited by KevinH; 10-30-2014 at 12:25 PM. Reason: fix typo
KevinH is offline   Reply With Quote
Old 11-01-2014, 09:27 AM   #1097
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 93
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
Hi Kevin,

I have made patches that make using Furigana metadata under auto indicate a target of epub 3.

But it cannot be tested in python2and3 branch because errors occur in command line mode as follows.

python2
Code:
Traceback (most recent call last):
  File "kindleunpack.py", line 11, in <module>
    from .compatibility_utils import PY2, binary_type, utf8_str, unicode_str
ValueError: Attempted relative import in non-package
python3
Code:
Traceback (most recent call last):
  File "kindleunpack.py", line 11, in <module>
    from .compatibility_utils import PY2, binary_type, utf8_str, unicode_str
SystemError: Parent module '' not loaded, cannot perform relative import
Thanks,
Attached Files
File Type: txt patch.master.txt (574 Bytes, 11 views)
File Type: txt patch.python2and3.txt (574 Bytes, 13 views)
tkeo is offline   Reply With Quote
Old 11-01-2014, 09:58 AM   #1098
PoP
Antonín ♯♭♪♮♫ ᵖʸᶠᵍᶜʳˡ
PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.
 
PoP's Avatar
 
Posts: 518
Karma: 7391817
Join Date: Dec 2010
Location: ♁ ᴺ₄₅°₃₀' ᵂ₇₃°₃₇' ±₆₀"
Device: K3.₄, PRS-350, SGS3, Rπ, iPad Air
Catching up with the thread, I discovered that for some time KindleUnpack has experimentally been able to unpack dictionaries, and possibly can even unpack inflections!

Using the current release of KindleUnpack Calibre Plugin
I tried it successfully on a set of dictionaries Thank you developers.

But it would not work on my homebrew NADSAT dictionary and reported the following:
Code:
calibre, version 2.8.0
ERREUR : KindleUnpack - The Plugin v0.77.0: 2

Traceback (most recent call last):
  File "calibre_plugins.kindleunpack_plugin.extraction", line 186, in unpack_ebook
  File "calibre_plugins.kindleunpack_plugin.utilities", line 282, in unpackMOBI
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 870, in unpackBook
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 793, in process_all_mobi_headers
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 581, in processMobi7
  File "calibre_plugins.kindleunpack_plugin.core.mobi_dict", line 204, in getPositionMap
KeyError: 2
Thought I would just bring it to your attention.
PoP is offline   Reply With Quote
Old 11-01-2014, 10:48 AM   #1099
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi tkeo,

I recently merged a pull request that made those changes. I will checkout the python2and3 branch and fix it to work.

And thanks for your patches. I will incorporate them as well.

KevinH

Quote:
Originally Posted by tkeo View Post
Hi Kevin,

I have made patches that make using Furigana metadata under auto indicate a target of epub 3.

But it cannot be tested in python2and3 branch because errors occur in command line mode as follows.

python2
Code:
Traceback (most recent call last):
  File "kindleunpack.py", line 11, in <module>
    from .compatibility_utils import PY2, binary_type, utf8_str, unicode_str
ValueError: Attempted relative import in non-package
python3
Code:
Traceback (most recent call last):
  File "kindleunpack.py", line 11, in <module>
    from .compatibility_utils import PY2, binary_type, utf8_str, unicode_str
SystemError: Parent module '' not loaded, cannot perform relative import
Thanks,
KevinH is offline   Reply With Quote
Old 11-01-2014, 10:54 AM   #1100
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi Pop,

Thanks for that bug report. It says that a fileposition does not exist that should. I would really need a copy of the dictionary ( or a sample that exhibits the same issue ) so that I can figure out what might be funny.

So please post a url to the problem dictionary or if you would prefer you can privately send me a url via personal messaging on this site (pm KevinH) so that I can track down the error and get it fixed.

Thanks,

KevinH


Quote:
Originally Posted by PoP View Post
Catching up with the thread, I discovered that for some time KindleUnpack has experimentally been able to unpack dictionaries, and possibly can even unpack inflections!

Using the current release of KindleUnpack Calibre Plugin
I tried it successfully on a set of dictionaries Thank you developers.

But it would not work on my homebrew NADSAT dictionary and reported the following:
Code:
calibre, version 2.8.0
ERREUR : KindleUnpack - The Plugin v0.77.0: 2

Traceback (most recent call last):
  File "calibre_plugins.kindleunpack_plugin.extraction", line 186, in unpack_ebook
  File "calibre_plugins.kindleunpack_plugin.utilities", line 282, in unpackMOBI
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 870, in unpackBook
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 793, in process_all_mobi_headers
  File "calibre_plugins.kindleunpack_plugin.core.kindleunpack", line 581, in processMobi7
  File "calibre_plugins.kindleunpack_plugin.core.mobi_dict", line 204, in getPositionMap
KeyError: 2
Thought I would just bring it to your attention.
KevinH is offline   Reply With Quote
Old 11-01-2014, 11:18 AM   #1101
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,511
Karma: 43600000
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
Hi tkeo,

I recently merged a pull request that made those changes. I will checkout the python2and3 branch and fix it to work.
Oops. The relative imports thing is my fault. I never tested running from the command-line ... only from the gui "package." Ironic, since I was looking for a way to simplify importing the command-line scripts into other projects.

In light of this, it's probably best to just roll back that last merge entirely. Even separating the gui-helpers from the core unpacking scripts is probably not all that important at this point (and we can easily add that back in later if desired).

Last edited by DiapDealer; 11-01-2014 at 11:20 AM.
DiapDealer is offline   Reply With Quote
Old 11-01-2014, 12:01 PM   #1102
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi DiapDealer,

No need to rollback as I was able to get both to work by adding:

Code:
__path__ = ["lib", ".", "kindleunpack"]
at the top to force it to be a package and accept either lib.blah or .blah as import paths to allow both gui and command line

So no worries. python2and3 via the command line should now work again.

KevinH
KevinH is offline   Reply With Quote
Old 11-01-2014, 12:07 PM   #1103
PoP
Antonín ♯♭♪♮♫ ᵖʸᶠᵍᶜʳˡ
PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.
 
PoP's Avatar
 
Posts: 518
Karma: 7391817
Join Date: Dec 2010
Location: ♁ ᴺ₄₅°₃₀' ᵂ₇₃°₃₇' ±₆₀"
Device: K3.₄, PRS-350, SGS3, Rπ, iPad Air
Quote:
Originally Posted by KevinH View Post
Hi Pop,

...
I would really need a copy of the dictionary ( or a sample that exhibits the same issue ) so that I can figure out what might be funny.

So please post a url to the problem dictionary
...
Thanks KevinH.

The dictionary is in the last zip attached to this post :http://www.mobileread.com/forums/sho...d.php?t=205634
PoP is offline   Reply With Quote
Old 11-01-2014, 12:14 PM   #1104
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi tkeo,

I merged your fixes to target epub3 when Furigana metadata is present. I took the liberty of modifying your python2and3 patch to remove the .keys() since simply testing in metadata will work well with both python2 and python3.

Thanks!

KevinH
KevinH is offline   Reply With Quote
Old 11-01-2014, 12:30 PM   #1105
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,511
Karma: 43600000
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
Hi DiapDealer,

No need to rollback as I was able to get both to work by adding:

Code:
__path__ = ["lib", ".", "kindleunpack"]
at the top to force it to be a package and accept either lib.blah or .blah as import paths to allow both gui and command line

So no worries. python2and3 via the command line should now work again.

KevinH
Thanks for fixing that Kevin!
Was able to sync my fork with your master to pick up that fix, and will begin looking at converting the prefs store from ini to json.
DiapDealer is offline   Reply With Quote
Old 11-01-2014, 12:33 PM   #1106
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi PoP,

The error meant that one entry was missing an end position (it did not exist in the tagMap).

I think this is caused by your very very last entry in the dictionary not having a proper ending </idx:entry> tag.

Here is the snippet of code:
Code:
<idx:entry name="word" scriptable="yes">
        <h2>
          <idx:orth>Zvook</idx:orth><idx:key key="Zvook">
        </h2>
        <I>fr:</I> son <I>en:</I> sound <I>ru:</I> звук, zvuk (sound)

<div> <img hspace="0" vspace="0" align="middle" src="../Images/sep.gif"/> <table width="100%" bgcolor="#992211"><tr><th widht="100%" height="2px"></th></tr></table> </di\
v>

    </mbp:frameset>


  </body>
</html>
Notice there is no </idx:entry> to match the opening <idx:entry blah> tag for this very last word.

So please try adding the final </idx:entry> tag right before the </mbp:frameset> and recompiling.

It should then unpack successfully (I hope!). Please let me know if that does not do the trick.

Also, I just noticed that your use of <idx:key> does not seem to have an ending tag either (please check this all throughout your dictionary). If there is no <idx:key> content then it should be self-closed:
<idx:key name="blah" /> otherwise there should be a matching ending </idx:key> tag someplace as well for each beginning tag.

Take care,

KevinH

Quote:
Originally Posted by PoP View Post
Thanks KevinH.

The dictionary is in the last zip attached to this post :http://www.mobileread.com/forums/sho...d.php?t=205634

Last edited by KevinH; 11-01-2014 at 12:42 PM. Reason: updated on missing ending idx:key as well
KevinH is offline   Reply With Quote
Old 11-01-2014, 04:21 PM   #1107
PoP
Antonín ♯♭♪♮♫ ᵖʸᶠᵍᶜʳˡ
PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.PoP ought to be getting tired of karma fortunes by now.
 
PoP's Avatar
 
Posts: 518
Karma: 7391817
Join Date: Dec 2010
Location: ♁ ᴺ₄₅°₃₀' ᵂ₇₃°₃₇' ±₆₀"
Device: K3.₄, PRS-350, SGS3, Rπ, iPad Air
Quote:
Originally Posted by KevinH View Post
Hi PoP,

The error meant that one entry was missing an end position (it did not exist in the tagMap).

I think this is caused by your very very last entry in the dictionary not having a proper ending </idx:entry> tag. ... So please try adding the final </idx:entry> tag

It should then unpack successfully (I hope!). Please let me know if that does not do the trick.

Also, I just noticed that your use of <idx:key> does not seem to have an ending tag either (please check this all throughout your dictionary). If there is no <idx:key> content then it should be self-closed:
<idx:key name="blah" /> otherwise there should be a matching ending </idx:key> tag someplace as well for each beginning tag.

Take care,

KevinH
Thanks: adding the missing idx:entry closing tag for the last dictionary entry allowed KindleUnpack to work flawlessly. I am impressed, all the index entries, including the inflections, are properly extracted. Great job.

I also more properly self-closed all the idx:key tags.

As usual, find the updated dictionary here.
PoP is offline   Reply With Quote
Old 11-02-2014, 08:38 AM   #1108
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 93
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
Hi Kevin,

I have made two patches for the python2and3 branch.

1. to unpack AUDI and VIDE sections.

As same as the patch posted previously,
the AUDI and VIDE sections are unpacked into HDimages.
Tags in xhtmls are not linked to unpacked files.
It need to change unpacking directory and to make the tags linked.

2. to use OrderedDict in DEBUG mode.

Please commit these patches if acceptable.

Take care,
tkeo
Attached Files
File Type: txt patch.AUDIandVIDE.python2and3.txt (1.9 KB, 10 views)
File Type: txt patch.orderddict.python2and3.txt (1.7 KB, 15 views)
tkeo is offline   Reply With Quote
Old 11-02-2014, 11:00 AM   #1109
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 1,077
Karma: 444444
Join Date: Nov 2009
Device: many
Hi tkeo,

The audio video one is broken. According to Amazon publishing guidelines only Mobi 7 support Audio/Video and any audio/video files must be stored in a folder named "audiovideo".

So hold off on that. It was never meant to work in the KF8 part. At least not yet, which is why the link/urls were not updated or replaced by proper internal Kindle ones in the KF8 part.

BTW: I spent some time looking at audio/video and it seems that in the mobi 7 header, offset 0xe8 which we now call "unknown3" always points to the first record where extra indexes are stored. These extra INDX sections have yet to be decoded. They are related to the audio and video sections and links. Furthermore offset 0xc4 in the mobi 7 header (we refer to this as "unknown0") now seems to have a 1.

So we really need to figure out what these extra INDX sections are and how to decode them, what/why the "unknown3" points to them, etc.

So there is still too much reverse-engineering to be done before we can support audio/video in the older Mobi 7's. And when we do understand them, the unpacking part is easy and should go into a new folder called "audiovideo" inside the mobi7 folder when unpacked.

Take care,

KevinH

ps. I have committed your DEBUG use of OrderedDict patch. Thank you!

Last edited by KevinH; 11-02-2014 at 01:12 PM.
KevinH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Can i rotate text and insert images in Mobi and EPUB? JanGLi Kindle Formats 5 02-02-2013 05:16 PM
PDF to Mobi with text and images pocketsprocket Kindle Formats 7 05-21-2012 08:06 AM
Mobi files - images DWC Introduce Yourself 5 07-06-2011 02:43 AM
pdf to mobi... creating images rather than text Dumhed Calibre 5 11-06-2010 01:08 PM
Transfer of images on text files anirudh215 PDF 2 06-22-2009 10:28 AM


All times are GMT -4. The time now is 04:00 PM.


MobileRead.com is a privately owned, operated and funded community.