Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 11-08-2013, 05:08 PM   #1
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
Need for a dual mobi metadata tweaker

Hi,
Okay in its own thread, the topic is to create a metadata tweaker that will work with dual mobis as created by kindlegen so that people can tweak the metadata but NOT split the mobi 7 and mobi 8 parts.

Right now it would need the following capability:

1) add EXTH 501 element with value EBOK to both headers so that cover display will work better for some devices


There also seems to be some interest in allowing a uuid style ASIN element to be changed to something else but I am not sure why this is needed for something that was a personal document since it would not be an exact match to any official ASIN from Amazon's catalog.

I did not think kindlegen even generated an ASIN field? Does it?
KevinH is offline   Reply With Quote
Old 11-08-2013, 05:22 PM   #2
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,273
Karma: 42298328
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
I did not think kindlegen even generated an ASIN field? Does it?
No it doesn't.

It's just another of those things that affects whether something shows up as a 'book' vs a 'document' on some devices. Usually, just the non-empty presence of the 113 EXTH record is enough for those devices that check for it. I'm not familiar with the 504 record myself, but maybe it's the same scenario?

Maybe it would be best to let the user choose what they want to put into the ASIN record(s)... if at all possible? I've never seen them contain a uuid-looking string like was previously mentioned. I've only ever seen them contain the usual 10 character Amazon ASIN string. *shrugs*

Last edited by DiapDealer; 11-08-2013 at 05:31 PM.
DiapDealer is offline   Reply With Quote
 
Enthusiast
Old 11-08-2013, 06:24 PM   #3
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
Hi,
Would it be better to present a list of all meta values in each header with some sort of checkbox to delete specific ones plus text input fields to prompt for specific 501, 113, and 504 values to add.

We could probably pull something like that together using basic gui widgets, but then we would be bumping up against Mobi Perl which I do not want to duplicate the functionality of unless no one is currently supporting it.

Perhaps a better approach would be to patch Mobi Perl to handle dual mobis instead? It has been literally years since I have written any Perl but I once knew how to do that. Do you think the mobi perl author would be interested in adding dual mobi support?

If not, perhaps just hacking up a 501 EBOK adder will be enough to start and go from there.

KevnH
KevinH is offline   Reply With Quote
Old 11-08-2013, 06:43 PM   #4
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,273
Karma: 42298328
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
tompe's still around from time to time. I wouldn't want to speak for him, so perhaps we should just ask in the mobiperl thread if he has any desire to see the functionality of mobiperl extended (or minds having another tool that overlaps a bit in functionality)?
DiapDealer is offline   Reply With Quote
Old 11-09-2013, 02:37 AM   #5
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 425
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
Fake ASIN allow self-made e-books to use HQ cover system.

Combo MOBI with no modifications -> Ugly Personal tag. Low quality build-in cover image.

EXTH 501 set to EBOK. -> No Personal tag. Low quality build-in cover image.

EXTH 501 set to EBOK. EXTH 113 and 504 - Fake ASIN. -> No Personal tag. No cover at all unless ASIN is real one. In that case HQ cover is downloaded from Amazon cloud.
But when we upload this file with Calibre a HQ cover will be sideloaded to Kindle cache.
HQ cover can be also uploaded manually: /system/thumbnails/thumbnail_{uuid}_{cdetype}_portrait.jpg

Format of fake ASIN is not important. @DiapDealer - previously mentioned format is used by Calibre. And it is output of python function: uuid.uuid4()

Last edited by AcidWeb; 11-09-2013 at 04:43 AM.
AcidWeb is offline   Reply With Quote
Old 11-09-2013, 06:09 AM   #6
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 9,273
Karma: 42298328
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Gotcha. Makes sense to use a uuid so you never have to worry about duplicates, I guess.

So if this tool would generate a uuid for those two asin records, you're then going to have to manually name and sideload the cover image you want to use to your device? Or is there a calibre feature/plugin that checks the asin record(s) and handles the naming/sideloading of the image automatically when using calibre to send books to your device?
DiapDealer is offline   Reply With Quote
Old 11-09-2013, 06:16 AM   #7
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 425
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
If cdetype and asin is set Calibre will automatically handle upload of HQ cover.
That is one of Calibre features.

Last edited by AcidWeb; 11-09-2013 at 06:19 AM.
AcidWeb is offline   Reply With Quote
Old 11-09-2013, 11:22 AM   #8
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
Hi,
When I get some time next week I will hack together a quick script from pieces of DumpMobiHeader.py and pieces of mobi_split.py to create a very simple command line tool that will take one parameter which is an ASIN string and the add (or change if it already exists) 501 to be EBOK and 113 and 504 to be set to the passed in ASIN string.

Once I get something hacked together, you can test it to see if it does what you want or not. If anything much more than that is needed, we ask the Mobi-Perl author (tompe) if they want to support dual mobis before doing anything else.

How does that sound?

KevinH
KevinH is offline   Reply With Quote
Old 11-09-2013, 11:28 AM   #9
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 425
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
IMHO perfect. That is enough for regular users. Power-users will have solid foundations to make own modifications - if they are will be needed.

Personally that is all I need. If that will work I will embed it to KCC ASAP.
AcidWeb is offline   Reply With Quote
Old 11-09-2013, 06:04 PM   #10
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
Here is a pre-pre-pre alpha code to test and play with.

Warnings!
- It has not really been properly tested
- It has been thrown together with bunches of little pieces
- It is not unicode safe (for paths and things)
- This is just really proof of concept
- I do almost no error checking
- it is just a quick and dirty mishmash

kbhend$ python dualmetafix.py thisisatest test.mobi out.mobi
DualMetaFixer v001
test.mobi .MOBI
adding enth so trimming 000000000000000000000000
adding enth so trimming 00000000000000000000000000000000000000
adding enth so trimming 00000000000000000000000000000000000000
adding enth so trimming 000000000000000000000000
adding enth so trimming 00000000000000000000000000000000000000
adding enth so trimming 00000000000000000000000000000000000000

When I examine the out.mobi using DumpMobiHeader_v012.py I see the following:

Mobi 7 Header:
EXTH MetaData

Key: "ASIN_(504)"
Value: "thisisatest"

Key: "ASIN_(113)"
Value: "thisisatest"

Key: "CDE_Type_(501)"
Value: "EBOK"

and in the Mobi 8 Header I see:

EXTH MetaData

Key: "ASIN_(504)"
Value: "thisisatest"

Key: "ASIN_(113)"
Value: "thisisatest"

Key: "CDE_Type_(501)"
Value: "EBOK"


So please test this to see if it actually shows the cover you want with one of your dual mobi ebooks. Please note, nothing is written to the input file but use this with file copies just to be super safe.

Please let me know what errors you get and if it helps at all. If so, I will spend the time to make it unicode safe and clean it up, etc. Otherwise, it may not be worth the effort.

See the attached zip.

KevinH
Attached Files
File Type: zip dualmetafix.zip (2.5 KB, 37 views)
KevinH is offline   Reply With Quote
Old 11-10-2013, 10:55 AM   #11
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
dualmetafix.py v003

I had some more spare time and adding in some basic error testing and attempted to make things unicode safe by grabbing all arguments as utf-8 encoded and converting paths to full unicode only when needed.

So I would say this is now beta quality. Still not pretty. Still a real mishmash.

The palm database access stuff would be better as its own class and and the header and exth should all probably be their own classes as well to encapsulate the access routines.

Oh well. Please give this a try and let me know what troubles you run into. I am sure there are ways to speed this up as well by memory mapping the file and replacing specific sections directly in memory as opposed to using python strings but let's see if this works and is fast enough for what is needed.

Attached is dualmetafix.py version v003
Attached Files
File Type: zip dualmetafix.zip (3.8 KB, 38 views)
KevinH is offline   Reply With Quote
Old 11-10-2013, 10:59 AM   #12
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 425
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
I'm waiting for tester with Paperwhite to check it out but headers look OK.
Processing speed is quite high too - I tested it on 500mb MOBI files and it need only few seconds to complete the job. Current version is working fast enough.

EDIT:
It might be silly but please add credits and licence information to this code. I will be embedding it soon.

Last edited by AcidWeb; 11-10-2013 at 11:11 AM.
AcidWeb is offline   Reply With Quote
Old 11-10-2013, 11:24 AM   #13
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
Hi,

Since it is extracted from recent KindleUnpack code (and really does not use any of the original mobiunpack code at all) it is under the same license and basic credits:

print " This program is free software: you can redistribute it and/or modify"
print " it under the terms of the GNU General Public License as published by"
print " the Free Software Foundation, version 3."
print " Copyright © 2013 P. Durrant, K. Hendricks, S. Siebert, fandrieu, DiapDealer, nickredding."

So please just add that to the usage() function.

Kevin
KevinH is offline   Reply With Quote
Old 11-10-2013, 11:27 AM   #14
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 425
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
I will add it to file header and KCC README/Credits.
This script will not output any text when it will be part of KCC.
AcidWeb is offline   Reply With Quote
Old 11-10-2013, 11:40 AM   #15
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 954
Karma: 410248
Join Date: Nov 2009
Device: many
BTW,

If this works, it should also work just as well for mobis that have only one header - either mobi 7's or mobi 8's as long as they have their own EXTH region already created. Some really old mobis will not have an EXTH and so the program will not work for those older ones.

KevinH
KevinH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Recommended settings to convert dual-column PDF to useable MOBI format Cephas Atheos Conversion 7 09-18-2012 07:32 AM
Insert metadata as page at start of book adds does not replace (mobi to mobi) linusnc Calibre 2 07-19-2012 03:54 PM
Update Mobi header/file metadata without doing a Mobi to Mobi conversion RecQuery Conversion 2 06-30-2012 11:43 AM
EPUB (CSS) tweaker app Loccy Conversion 9 01-23-2011 10:22 PM
Firefox Tweaker: Flexbeta FireTweaker XP Alexander Turcic Lounge 0 08-16-2004 04:51 AM


All times are GMT -4. The time now is 12:13 PM.


MobileRead.com is a privately owned, operated and funded community.