Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 02-11-2014, 10:24 AM   #661
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 73
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
Hi KevinH,

Quote:
Originally Posted by KevinH View Post
One other thing, I am not a big fan of minidom at all. It seems generally bloated and barfs if any true unicode is used (at least on 2.X). I see you wrote both a xml.dom.minidom version and a regular expression version of things. Every time I have used a xml elementTree or some other XML parser (either standard package or add-ons) in python 2.X I have run into problem cases that simply do not parse well or get confused with encodings, resulting in non-robust operation on some platforms (Mac, Win, or Linux).

So unless you feel strongly about it (and given the re vs dom code sizes are about the same), I would rather stick with regular expressions version as they are easier for people to modify and fix are are robust to most encoding issues.

I see you have also written a metadata parsing routine that supports epub 3 like "refines" on named items. This is quite nice but using it in epub 2 spec devices might cause problems.
Firstly, I have no reason to stick to using dom. So I will revert the code to use re. Parsing RESC section using the dom makes look code simpler and shorter than using re. (The re version needs a Metadata class I wrote, whereas the dom version does not.) I think the dom is suitable to represent an epub structure; however, currently, less familiar and less stable than the re.

On my environment (python 2.7.6 windows 32bit), the minidom is able to parse utf-8 and stored as an Unicode string; re-encoding utf-8 is necessary to use. It is quite confusing for me. (Utf-8 is one of the encodings of unicode, isn't it?) If the minidom stored elements as utf-8 strings, it would be very easy, I think.

Quote:
I really think we should incorporate your code and try and create an epub 3 generator version of KindleUnpack to stay in epub 3 space and not try to mix private extensions into what is primarily epub 2 code.

What do you think?
I have considered. It might be better to create the KindleUnpack of pure epub 3 version, separately to epub 2 version. Since the pure epub 3 ebooks will become popular but many epub 2 books will remain because of no necessity of the epub 3 features.

But, now, there are many epub 2 ebooks available but epub3 books are not so popular, and as you mentioned, books basically based on epub2 plus partly included epub3 definition are published from vendors. So, I think, it is not time for creating the pure epub3 version of the KindleUnpack.

Quote:
Thank you for the examples, I will play around with them. I can't believe that a Kindle device supports the spine/page spread properties by keeping and parsing the RESC section on the fly during reading. My guess is they must include or encode that information in some other way but I that is just a guess and I could be wrong.
As for the RESC section. I've guessed, the RESC section is prepared to store the information that K8 format does not define. It's just guess no evidence.

Thanks,
tkeo
tkeo is offline   Reply With Quote
Old 02-11-2014, 11:39 AM   #662
pdurrant
The Grand Mouse
pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.
 
pdurrant's Avatar
 
Posts: 30,833
Karma: 85467920
Join Date: Jul 2007
Location: Norfolk, England
Device: NOOK ST GlowLight
I don't have time to join in development, but I think that splitting KindleUnpack into ePub2/ePub3 versions would be a mistake.

Surely we can detect whether a Kindle book contains stuff that requires ePub 3, and only output an ePub 3 when that is present?

Even a command-line switch between ePub 2 and 3 output would be preferable to my mind.

I would also mention that KindleUnpack should be producing something that can be run back through KindleGen and produce the same file as the original (assuming the same KindleGen version is used). This is a great feature that should not be lost in trying to produce an ePub-like output as well.
pdurrant is offline   Reply With Quote
Old 02-11-2014, 01:00 PM   #663
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 8,767
Karma: 39536849
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
It would be pretty hard to determine if the source material was ePub3 or just ePub2 with elements of ePub3/CSS3. Or indeed whether an ePub was even used at all to create the Kindlebook. It's all getting pretty gray.

Surely, the OPF (and maybe NCX vs NAV document) is where the only real differences lie between outputting ePub3 as opposed to ePub2, right? We do nothing now (to my knowledge) to filter out any xhtml/css that's not ePub2 "compliant", so I wouldn't think it would out of the question to have two different OPF generation routines and just give the user the option to choose 3 vs 2 based on what they're seeing. Perhaps even the option to forego ePub generation altogether (just the OEBPS output)?

EDIT: it occurred to me that we could be trying to push html5 into an xhtml-shaped box at times, though. But that still seems only a matter of leaving off the DTD and xhtml namespace, no? Specific attributes and links are massaged, but the markup itself should come through relatively unscathed--regardless of whether it's xhtml or html5--shouldn't it?

Last edited by DiapDealer; 02-11-2014 at 01:16 PM.
DiapDealer is offline   Reply With Quote
Old 02-11-2014, 07:10 PM   #664
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi,
It sounds like everyone wants to go with epub2 plus extensions for things supported by Kindlegen. So I will strip out the minidom stuff leaving the re code, add in the 3 or 4 minor bug fixes not yet in his version, add tkeo as one of the authors and produce a KindleUnpack version 0.64 to prevent confusion. It will continue to work as before and produce an epub2 format with his extensions added in.

Hope to have something by next week or before.

How does that sound?

Kevin
KevinH is offline   Reply With Quote
Old 02-12-2014, 09:16 AM   #665
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 73
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
Quote:
Originally Posted by KevinH View Post
Hi,
It sounds like everyone wants to go with epub2 plus extensions for things supported by Kindlegen. So I will strip out the minidom stuff leaving the re code, add in the 3 or 4 minor bug fixes not yet in his version, add tkeo as one of the authors and produce a KindleUnpack version 0.64 to prevent confusion. It will continue to work as before and produce an epub2 format with his extensions added in.

Hope to have something by next week or before.

How does that sound?

Kevin
Hi KevinH,

Thanks for joining me to the autors.
I agree with you: to go with epub2 plus extensions for things supported by Kindlegen, is preferable.

I should have named my modifed version as v0.62mod or something like that at first place.

Though you have already been fixing the code, I have separeted the code I have written to an individual file 'mobi_k8resc.py' and changed to use re instead of dom. 'mobi_k8proc.py' is reverted to be identical to v0.62. I have made to possible to turn of/off of processing RESC. I will post it to this weekend probably.

Thanks,
tkeo
tkeo is offline   Reply With Quote
Old 02-12-2014, 10:34 AM   #666
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi tkeo,

Wonderful! Thank you. I will wait to get that, add in the missing minor bug fixes and author info and produce a release.

Take care,

KevinH
KevinH is offline   Reply With Quote
Old 02-16-2014, 08:57 AM   #667
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 73
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
KindleUnpack v0.63a

Hi,

I have updated my modifed version of KindleUnpack.

From v0.63,
  1. Separeted the code I have written to an individual file 'mobi_k8resc.py' and changed to use re instead of dom. 'mobi_k8proc.py' is reverted to be identical to v0.62.
  2. Added a cover creation and renaming cover image.

The RESC setcion processing and the cover creation are able to tun on/off by booleans PROC_K8RESC and CREATE_COVER_PAGE respectively.

Please remove any part of my modification that is not necssarily nor appropriate in later version.

As for the cover page, some spine in RESC has striped tace of it. I tried to create such a mobi, but I couldn't. In my experiment, the cover page is moved to last page insead of striped. I attach a mobi and a src.
Although the cover xhtml page is not required in EPUB regulation, I have made insertion of it also, because my primary epub viewer does not show cover image itself.

I hoe this version works on any environment.

Thanks,
Attached Files
File Type: zip KindleUnpack_v63a.zip (62.0 KB, 38 views)
File Type: mobi rtl_example3.mobi (208.1 KB, 20 views)
File Type: zip rtl_example3_src.zip (55.1 KB, 21 views)
tkeo is offline   Reply With Quote
Old 02-23-2014, 11:09 AM   #668
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
KindleUnpack V064

Hi All,

Okay, I have integrated a bug fix for full unicode command lines not being properly parsed, and integrated a contributed bug fix to better identify some JPEG files, into tkeo's major changes to support RTL books and to properly parser the RESC and extract more information from the spine. I have also added tkeo to the list of authors and updated the copyright to 2014.

Attached is the most up-to-date version of KindleUnpack with all outstanding changes integrated.

All testing and bug reports are welcomed especially with some of the new features just added.

Please report any issue or problems here.

KindleUnpack V064

Thanks,

KevinH
Attached Files
File Type: zip KindleUnpack_v64.zip (69.1 KB, 37 views)
KevinH is offline   Reply With Quote
Old 02-23-2014, 12:46 PM   #669
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 8,767
Karma: 39536849
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Nice work guys!

One quick, persnickety thing: the "title" attribute for the new (experimental) cover-page resource item in the guide section of the OPF is currently being set to the actual full title of the ebook. Probably not going to cause any problems that I know of (not sure the attribute is used by devices/apps, to tell the truth), but it doesn't really make sense to put it there either. It can be quite long at times, and who knows how long it will before something in there(a character perhaps) borks the processing. Something like "Cover" should suffice for that particular reference item's "title" in all cases.
DiapDealer is offline   Reply With Quote
Old 02-23-2014, 01:10 PM   #670
pdurrant
The Grand Mouse
pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.
 
pdurrant's Avatar
 
Posts: 30,833
Karma: 85467920
Join Date: Jul 2007
Location: Norfolk, England
Device: NOOK ST GlowLight
Quote:
Originally Posted by KevinH View Post
Please report any issue or problems here.

KindleUnpack V064
I've updated the first post with the new archive (sans the backup copy of one of the scripts), and an updated AppleScript with the new scripts in it.

Thanks to all!
pdurrant is offline   Reply With Quote
Old 02-23-2014, 04:27 PM   #671
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi DiapDealer,

Quote:
Originally Posted by DiapDealer View Post
Nice work guys!

One quick, persnickety thing: the "title" attribute for the new (experimental) cover-page resource item in the guide section of the OPF is currently being set to the actual full title of the ebook. Probably not going to cause any problems that I know of (not sure the attribute is used by devices/apps, to tell the truth), but it doesn't really make sense to put it there either. It can be quite long at times, and who knows how long it will before something in there(a character perhaps) borks the processing. Something like "Cover" should suffice for that particular reference item's "title" in all cases.

I will look into it. I can't quite tell from your post... Is this tkeo's generate an xhtml cover page file name or the general opf entry for the generated file or only the guide entry itself in the opf?

KevinH
KevinH is offline   Reply With Quote
Old 02-23-2014, 06:00 PM   #672
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 8,767
Karma: 39536849
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
Hi DiapDealer,

I will look into it. I can't quite tell from your post... Is this tkeo's generate an xhtml cover page file name or the general opf entry for the generated file or only the guide entry itself in the opf?

KevinH
Not the file name. It's only the title attribute for the (new) cover-page guide entry in the opf itself.

Like I said, not a big deal at all (at least I don't think so). It just seems like overkill for that attribute's value to be the entire title of the ebook.

EDIT: Found it in mobi_k8resc.py. Something to the tune of the patch/diff file attached (had to change the file's extension to .txt so the forum would accept it) is what I had in mind.
Attached Files
File Type: txt mobi_k8resc.txt (539 Bytes, 21 views)

Last edited by DiapDealer; 02-23-2014 at 07:35 PM.
DiapDealer is offline   Reply With Quote
Old 02-23-2014, 08:41 PM   #673
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi DiapDealer,

Wonderful! Thanks for the patch. I will add it for the next release.
Nice Work!

KevinH

Quote:
Originally Posted by DiapDealer View Post
Not the file name. It's only the title attribute for the (new) cover-page guide entry in the opf itself.

Like I said, not a big deal at all (at least I don't think so). It just seems like overkill for that attribute's value to be the entire title of the ebook.

EDIT: Found it in mobi_k8resc.py. Something to the tune of the patch/diff file attached (had to change the file's extension to .txt so the forum would accept it) is what I had in mind.
KevinH is offline   Reply With Quote
Old 02-24-2014, 06:33 AM   #674
tkeo
Connoisseur
tkeo began at the beginning.
 
Posts: 73
Karma: 10
Join Date: Feb 2014
Location: Japan
Device: Kindle PaperWhite, Kobo Aura HD
Quote:
Originally Posted by DiapDealer View Post
Not the file name. It's only the title attribute for the (new) cover-page guide entry in the opf itself.

Like I said, not a big deal at all (at least I don't think so). It just seems like overkill for that attribute's value to be the entire title of the ebook.

EDIT: Found it in mobi_k8resc.py. Something to the tune of the patch/diff file attached (had to change the file's extension to .txt so the forum would accept it) is what I had in mind.
Hi,

I have written the code base on the description of,
http://wiki.mobileread.com/wiki/Ebook_Covers.

It seems that the required attribute in the reference tag for cover page is type="cover" and the title is not required.
I'm not sure what is the most proper way for a cover page.

Thanks,
tkeo is offline   Reply With Quote
Old 02-24-2014, 08:35 AM   #675
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 8,767
Karma: 39536849
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by tkeo View Post
It seems that the required attribute in the reference tag for cover page is type="cover" and the title is not required.
I'm pretty sure that's correct. That's why it didn't make much sense to me to insert something as potentially large (and un-simple) as the ebook's EXTH title as the value for that un-required attribute. Just slap "Cover" or "Cover Page" in there and be done.

Just out of curiosity, are you manufacturing the "linear=No" attribute for the created cover page's spine entry? Or is that something that unpacks "as is"?

I prefer the somewhat "standard" svg method of creating a cover page, but that's just a personal preference. Feel free to take a look at my hacked-up mobi_k8resc.py file if you're interested at all. I've added a 'borrowed' method to get the necessary dimensions of jpeg, gif, or png images (and removed the 'linear' attribute for my own selfish reasons). Don't feel any pressure to adopt my changes or anything. I don't mind adjusting the "official" codebase to suit my needs. That's how I get my practice.

Thanks for your input on the project, by the way. Much appreciated!
Attached Files
File Type: zip mobi_k8resc.zip (5.2 KB, 22 views)

Last edited by DiapDealer; 02-24-2014 at 08:51 AM.
DiapDealer is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Can i rotate text and insert images in Mobi and EPUB? JanGLi Kindle Formats 5 02-02-2013 04:16 PM
PDF to Mobi with text and images pocketsprocket Kindle Formats 7 05-21-2012 07:06 AM
Mobi files - images DWC Introduce Yourself 5 07-06-2011 01:43 AM
pdf to mobi... creating images rather than text Dumhed Calibre 5 11-06-2010 12:08 PM
Transfer of images on text files anirudh215 PDF 2 06-22-2009 09:28 AM


All times are GMT -4. The time now is 04:56 PM.


MobileRead.com is a privately owned, operated and funded community.