Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 11-10-2013, 06:59 PM   #31
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi,

So what happens if you pass the kindlegen output through this script specifying a valid uuid as the asin (just like calibre generates) and load the result into calibre? Won't calibre then properly upload it to the device and sideload everything?

Perhaps calibre looks for its own creator code or some other calibre generated metatag in the exth but that should be easy to mimic in dualmetafix.py.

Alternatively, submitting a patch to calibre should work, as the same fake asin should be added to both headers when it creates a dual mobi.

Sorry this didn't help.

Take care,

KevinH
KevinH is offline   Reply With Quote
Old 11-10-2013, 08:29 PM   #32
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,422
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If a mobi file has an EBOK and an asin, amazon's software will *always* try to download the cover from the amazon servers, using that asin. The stupid part is that if the download fails (as it always will, for a fake asin), the software does not fallback to using the cover present in the mobi file. Not just that, this change was deliberately introduced by Amazon at some point. Earlier versions of Amazon's apps did not behave in this brain dead fashion.

calibre gets around this by putting a copy of the cover into the cover download cache on the device. This causes the Kindle software to just use that cached cover if the download fails.

Naturally, this only works with eink Kindles since on android/idevices calibre does not know how to upload covers into the cache for the Kindle app. Hence, this FAQ entry: http://manual.calibre-ebook.com/faq....droid-ipad-etc
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 11-10-2013, 11:15 PM   #33
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi Kovid,
When calibre creates a joint/dual mobi (one with both Mobi 7 and Mobi 8 pieces inside), does calibre remember to set EBOK for metadata 501 and the same asin (uuid) in both the mobi 7 header and the later mobi 8 header in the joint mobi file?

I thought from the earlier discussion in this thread, that it only sets that in the first section (mobi 7 header) and not the mobi 8 header that comes after the BOUNDARY section?

I thought from what AcidWeb wrote that calibre did not make those changes in the second header, which is partially causing the problem from what he wrote.

Thanks,

Kevin
KevinH is offline   Reply With Quote
Old 11-11-2013, 12:48 AM   #34
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,422
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Yes it does. AcidWeb is talking about *updating* metadata in existing joint mobi files, for which calibre will only update the first header record, i.e. the mobi 6 header record. He is creating his joint files using kindlegen, for the comic support, and then trying to use some tool to change the metadata records in the joint file.

I have no interest in adding support to calibre for updating both metadata records in a joint file, but patches are welcome. However, given that MOBI is a "fragile" format and making this kind of change would require extensive testing on various kindle devices/apps to ensure it doesn't break anything, dont expect to get those patches merged easily.

A better approach, IMO, is to simply add support for the KF8 fixed layout format to calibre's azw3 output plugin, when converting comic files (cbr/cbz).

That will require a bit of reverse engineering again, but it should not be too hard. However, I am currently too busy writing Sigil's replacement to get involved.
kovidgoyal is offline   Reply With Quote
Old 11-11-2013, 02:41 AM   #35
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
I have admit that adding hybrid MOBI editing to Cablire could be QA nightmare.

@kovidgoyal - KF8 fixed layout format is only tip of iceberg. To achieve proper output quality entire KCC would need to be rewritten as Calibre plugin. And we not planning to do that for multiple reasons.

AFAIK Paul Durrant will try optimize KindleUnpack code for splitting MOB8 filled with graphic. We will see how that goes.

Output as AZW3 without ASIN is currently best option I see.
In light of recent discoveries editing both of headers does not give us anything. Yes. It is much faster but output is just not practical. I'm terrible sorry for wasting your time KevinH.

Last edited by AcidWeb; 11-11-2013 at 03:31 AM.
AcidWeb is offline   Reply With Quote
Old 11-11-2013, 10:43 AM   #36
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 806
Karma: 312304
Join Date: Nov 2009
Device: many
Hi AcidWeb,
No problems. I am sure Paul will be able to speed things up. The original code was not written for speed so I am sure merging the graphics sections into one block or something along those lines should help as it is the allocation and deallocation of long strings that really hurts python performance. Thus the trick of appending segments of strings to a list and then joining the list once at the end so python will only allocate the space needed for the entire block once and won't be allocating and deallocating blocks as every concatenation is done.

Take care,

KevinH




Good luck
KevinH is offline   Reply With Quote
Old 05-04-2014, 02:42 PM   #37
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
ARISE! :-)

Well. At last I have real PW2 so I could research this topic from scratch without tormenting my testers.

@KevinH - it seems that your script is fully usable. No idea what changed but it simply... work. I will incorporate your code in next KCC release (and drop KindleUnpack).

The process looks like this:
* Source EPUB file is converted with newest Kindlegen using -dont_append_source option.
* We receive hybrid MOBI (M6/KF8 headers) without SOURCE section.
* Ouput file is processed by DualMetaFix. We change fields 113, 501, 504 in both headers.

Now if we upload ready file manually to Kindle - it appear without Personal banner. Kindle is trying to download cover but it fails due fake ASIN and fallback to generic placeholder cover. It don't use low-res cover from file at all.

If we upload ready file with Calibre - fields 113, 501, 504 are untouched. And Calibre upload book cover to Kindle cache. And everything is displayed correctly - we have cover and no Personal banner. But we must remember that Calibre upload cover from own metadata database NOT from MOBI file itself.

If we send that file through Amazon Cloud we receive AZW3 file (KF8 only) - personal document with generic placeholder cover.

/offtopic
It would be nice feature if Calibre could import cover from MOBI. I have some code what can cut covers from MOBI files but it is hacky, hack and currently I don't have time to research Calibre code and implement it properly.

Last edited by AcidWeb; 05-04-2014 at 03:14 PM.
AcidWeb is offline   Reply With Quote
Old 05-05-2014, 01:46 AM   #38
quiris
Connoisseur
quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.
 
quiris's Avatar
 
Posts: 92
Karma: 1354
Join Date: Oct 2013
Location: Poland
Device: Kindle Paperwhite
Quote:
Originally Posted by AcidWeb View Post
If we send that file through Amazon Cloud we receive AZW3 file (KF8 only) - personal document with generic placeholder cover.
As workaround of this incredibly stupid issue* I use my own app ExtractCoverThumbs to automatically extract from existing books on Kindle and sideload it back via USB missing cover thumbnails for books downloaded from MYK cloud: http://www.mobileread.com/forums/sho...7&postcount=15

* Answer from Amazon: As per our technical team, "For Pdoc files sent for conversion via email, we strip the cover image. For Pdocs sideloaded via usb, we do not change the files, as the files are not sent to our server, but directly side loaded into device. For Books purchased in our website, we preserve the cover image. For certain mobi files, we do not reconvert it, but directly send the file received. Hence cover image is not stripped. This is the expected behavior of our system."
quiris is offline   Reply With Quote
Old 05-05-2014, 02:54 AM   #39
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
Yeah - I seen your code. It work correctly but I prefer approach that just get cover from file without unpacking entire MOBI.

I work on 300MB+ MOBI files and performance is important.

Last edited by AcidWeb; 05-05-2014 at 03:37 AM.
AcidWeb is offline   Reply With Quote
Old 05-12-2014, 08:54 AM   #40
quiris
Connoisseur
quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.
 
quiris's Avatar
 
Posts: 92
Karma: 1354
Join Date: Oct 2013
Location: Poland
Device: Kindle Paperwhite
Quote:
Originally Posted by AcidWeb View Post
Yeah - I seen your code. It work correctly but I prefer approach that just get cover from file without unpacking entire MOBI.
All I need is a simple function to extract cover image from Mobi/AZW3. KevinH, could you point me how to do it? Unfortunately, I don't know binary Mobi format so I used KindleUnpack for that.
quiris is offline   Reply With Quote
Old 05-12-2014, 08:56 AM   #41
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
I have experimental grade code for that:
https://github.com/AcidWeb/KindleBut...r/File.py#L118
AcidWeb is offline   Reply With Quote
Old 05-12-2014, 09:19 AM   #42
quiris
Connoisseur
quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.quiris is no ebook tyro.
 
quiris's Avatar
 
Posts: 92
Karma: 1354
Join Date: Oct 2013
Location: Poland
Device: Kindle Paperwhite
So is it absolutely necessary to process entire resource section to find all images? Are they 'mixed' with "FLIS", "FCIS", "FDST", "DATP" etc?
quiris is offline   Reply With Quote
Old 05-12-2014, 09:43 AM   #43
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
Considering that CoverOffset is not static and not always first image record is cover - yes.
Also this code not really process them. It is more like building index of resources to interpret CoverOffset number.
AcidWeb is offline   Reply With Quote
Old 05-13-2014, 03:07 AM   #44
EbokJunkie
Zealot
EbokJunkie began at the beginning.
 
Posts: 122
Karma: 16
Join Date: Feb 2009
Location: SoCal
Device: Kindle 3, Kindle PW, Pocketbook 301+, Pocketbook Touch, Sony 950, 350
KindleButler fails to load external/custom covers. Tried multiple cover formats, always get Failed to load custom cover.
EbokJunkie is offline   Reply With Quote
Old 05-13-2014, 03:09 AM   #45
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 399
Karma: 258422
Join Date: Mar 2013
Location: Poland
Device: Kindle Paperwhite, Kindle Keyboard
That entire project is experimental, untested and not finished.

It is not officially released yet.
AcidWeb is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Recommended settings to convert dual-column PDF to useable MOBI format Cephas Atheos Conversion 7 09-18-2012 07:32 AM
Insert metadata as page at start of book adds does not replace (mobi to mobi) linusnc Calibre 2 07-19-2012 03:54 PM
Update Mobi header/file metadata without doing a Mobi to Mobi conversion RecQuery Conversion 2 06-30-2012 11:43 AM
EPUB (CSS) tweaker app Loccy Conversion 9 01-23-2011 10:22 PM
Firefox Tweaker: Flexbeta FireTweaker XP Alexander Turcic Lounge 0 08-16-2004 04:51 AM


All times are GMT -4. The time now is 08:45 AM.


MobileRead.com is a privately owned, operated and funded community.