Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 08-24-2012, 07:21 AM   #46
dilo_sec
Member
dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.dilo_sec ought to be getting tired of karma fortunes by now.
 
Posts: 21
Karma: 244219
Join Date: Jul 2011
Device: K3
Being naturally curious, I also dumped the pagemap section of my 2nd ebook - the start looks the same (these could be fixed values), the value at 0x0038 looks to denote the number of bits, 0010 (16-bit) or 0020 (32-bit), for the page offset values.

A dump of the pagemap is below, including part of the page-map.xml source - I've not figured what the text following "pageMap": means - it would not be needed during mobi unpacking to generate a page-map.xml.

-- start
0x0000: 50414745 PAGE

0x0004: 00000008 ?
0x0008: 00010001 ?
0x000C: 0000002A ?

0x0010: 0000 block 0?
0x0012: 001E size of block, 0x0032-0x0014 = 1E

0x0014: 7B0A {
0x0016: 2020 20226669 ... "fileRevisionId" : "1"
0x0030: 7D0A }

0x0032: 0001 block 1?
0x0034: 0079 size of block, 0x00B3-0x003A = 79
0x0036: 0183 pages in pagemap
0x0038: 0020 bits is page offset values

0x003A: 7B0A {
0x003C: 2020 20226465 ... "description" : "PageMap from source by kindlegen",
0x0073: 2020 20227061 ... "pageMap": "(1,c, |i),(3,r,2),(7,r,7),(10,r,11),(12,a,1)"
0x00B1: 7D0A }

0x00B3: 00000028 cover
0x00B7: 000025C0 i
0x00BA: 000028D5 ii
0x00BE: 00002C0F iii
<snip>
-- end pagemap

-- start
<page-map xmlns="http://www.idpf.org/2007/opf">
<page name=" " href="cover.html"/>
<page name="i" href="frontm.html"/>
<page name="ii" href="frontm.html#page_ii"/>
<page name="iii" href="frontm.html#page_iii"/>
<page name="iv" href="frontm.html#page_iv"/>
<page name="v" href="frontm.html#page_v"/>
<page name="vii" href="frontm.html#page_vii"/>
<page name="viii" href="frontm.html#page_viii"/>
<page name="ix" href="frontm.html#page_ix"/>
<page name="x" href="frontm.html#page_x"/>
<page name="xi" href="frontm.html#page_xi"/>
<page name="xii" href="frontm.html#page_xii"/>
<page name="1" href="chapter01.html"/>
<page name="2" href="chapter01.html#page_2"/>
<page name="3" href="chapter01.html#page_3"/>
<page name="4" href="chapter01.html#page_4"/>
<page name="5" href="chapter01.html#page_5"/>
<snip>
-- end page-map.xml

Last edited by dilo_sec; 08-24-2012 at 08:07 AM. Reason: fixed a wrong offset!
dilo_sec is offline   Reply With Quote
Old 08-24-2012, 01:10 PM   #47
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Hi,

Wow! Nice work indeed!

You are taking almost all of the fun out of it by figuring things out before anyone else has even had a chance to stare at it yet.

Thanks for leaving a few unknowns for the rest of us! ;-)

KevinH


Quote:
Originally Posted by dilo_sec View Post
KevinH,

The pagemap section maps the page numbers to offsets into the raw (uncompressed) html in the mobi file - (only) the offsets are different for Mobi6 and Mobi8 parts.

I've decoded the pagemap section as follows: (the use of some values are unknown?)

-- start
0x0000: 50414745 PAGE

0x0004: 00000008 ?
0x0008: 00010001 ?
0x000C: 0000002A ?

0x0010: 0000 block 0?
0x0012: 001E size of block, 0x0032-0x0014 = 1E

0x0014: 7B0A {
0x0016: 2020 20226669 ... "fileRevisionId" : "1"
0x0030: 7D0A }

0x0032: 0001 block 1?
0x0034: 0054 size of block, 0x008E-0x003A = 5A
0x0036: 000A pages in pagemap
0x0038: 0010 ?

0x003A: 7B0A {
0x003C: 2020 20226465 ... "description" : "PageMap from source by kindlegen",
0x0073: 2020 20227061 ... "pageMap" : "(1,a,1)"
0x008C: 7D0A }

0x008E: 0281 page_1
0x0090: 0500 page_2
0x0092: 074A page_3
0x0094: 0959 page_4
0x0096: 0B49 page_5
0x0098: 0DCA page_6
0x009A: 1014 page_7
0x009C: 1176 page_8
0x009E: 1413 page_9
0x00A0: 164E page_10
--end pagemap

Apologies about the formatting - all the tabs and mutiple spaces have been scrunched up to 1 space!

Hope this is useful ...

DS.
KevinH is offline   Reply With Quote
Advert
Old 08-24-2012, 01:37 PM   #48
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Hi DS,

You wrote:

0x003A: 7B0A {
0x003C: 2020 20226465 ... "description" : "PageMap from source by kindlegen",
0x0073: 2020 20227061 ... "pageMap": "(1,c, |i),(3,r,2),(7,r,7),(10,r,11),(12,a,1)"
0x00B1: 7D0A }

0x00B3: 00000028 cover
0x00B7: 000025C0 i
0x00BA: 000028D5 ii
0x00BE: 00002C0F iii
<snip>
-- end pagemap

-- start
<page-map xmlns="http://www.idpf.org/2007/opf">
<page name=" " href="cover.html"/>
<page name="i" href="frontm.html"/>
<page name="ii" href="frontm.html#page_ii"/>
<page name="iii" href="frontm.html#page_iii"/>
<page name="iv" href="frontm.html#page_iv"/>
<page name="v" href="frontm.html#page_v"/>
<page name="vii" href="frontm.html#page_vii"/>
<page name="viii" href="frontm.html#page_viii"/>
<page name="ix" href="frontm.html#page_ix"/>
<page name="x" href="frontm.html#page_x"/>
<page name="xi" href="frontm.html#page_xi"/>
<page name="xii" href="frontm.html#page_xii"/>
<page name="1" href="chapter01.html"/>
<page name="2" href="chapter01.html#page_2"/>
<page name="3" href="chapter01.html#page_3"/>
<page name="4" href="chapter01.html#page_4"/>
<page name="5" href="chapter01.html#page_5"/>



So ...

"pageMap": "(1,c, |i),(3,r,2),(7,r,7),(10,r,11),(12,a,1)"

must in some way code up information in the "preface" pages up to the point where normal page numbering begins.

Do you have a sample that includes appendices or bibliography numbered in a different scheme (maybe A-1 or A2), etc that might show us more?

If not, we can always cook up a few samples that we might be able to use to figure out the exact meaning and construction of:

"pageMap": "(1,c, |i),(3,r,2),(7,r,7),(10,r,11),(12,a,1)"

I am thinking here more along the lines of getting calibre to generate the same information when it creates joint or azw3 mobis.

Also perhaps we need to rename what we call the SRCS offset in the header to be more generic. Perhaps something along the lines of an offset to additional pieces that need not be part of the mobi itself (PAGE, SRCS, and ...?), so perhaps ADDONS offset and count might be a more appropriate way of thinking about this header field in both the Mobi6 and Mobi8 headers.

Thanks again for finding and figuring this out. Patches and changes for Mobi_Unpack are of course welcome! ;-)

Take care,

KevinH
KevinH is offline   Reply With Quote
Old 10-15-2012, 01:34 PM   #49
SusanM
Bemused by possibilities
SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.
 
SusanM's Avatar
 
Posts: 58
Karma: 480244
Join Date: Jul 2012
Device: iPad3, Kobo
Great Script

Quote:
Originally Posted by pdurrant View Post
Kindlegen 1.1 and later now always adds the source files used in compiling the kindle ebook as the penultimate record in the kindle ebook.

So I wrote a python script that strips out the sources record from Kindle format ebooks. And for those on Macs I wrote a nice Applescript wrapper and also put the python script in the AppleScript bundle to make things easy.

If you're going to upload to the Amazon store, this script is probably unnecessary, as Amazon will strip the sources before delivery anyway.

But if you're going to upload the ebook somewhere else, (e.g. the MobileRead Library), you might well want to make the file as small as possible.

KevinH has enhanced the original python script to handle newer Kindle format ebooks correctly.

Here they are. A python script, and an AppleScript wrapper around it. If you're on a Mac you only need the Applescript, as it includes the Python script in it. The Applescript is a simple drag&drop operation — drag your KindleGen generated file onto it, and it creates one named [oldname]_stripped.mobi.

As always, please comment with any bug reports or problems.
Great script and thanks!
SusanM is offline   Reply With Quote
Old 10-15-2012, 01:55 PM   #50
SusanM
Bemused by possibilities
SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.
 
SusanM's Avatar
 
Posts: 58
Karma: 480244
Join Date: Jul 2012
Device: iPad3, Kobo
I have tried both scripts for Kindlegen conversion and the KindleStripper posted by pdurrant. Nice addition to eBook tools.

Last edited by SusanM; 10-15-2012 at 01:58 PM.
SusanM is offline   Reply With Quote
Advert
Old 10-27-2012, 12:49 AM   #51
Skeezix
Enthusiast
Skeezix began at the beginning.
 
Posts: 28
Karma: 10
Join Date: Jun 2011
Device: Kindle
Looks like something changed with kindlegen 2.7. Using kindlestrip on a MOBI generated with 2.7 corrupts the file, so that my Kindle Keyboard cannot read it. The stripped MOBI I generated with 2.5 (using the same opf) works fine.
Skeezix is offline   Reply With Quote
Old 10-27-2012, 01:30 AM   #52
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,464
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
No clue there was even a 2.7 kindlegen out there.
DiapDealer is online now   Reply With Quote
Old 10-27-2012, 03:33 AM   #53
pdurrant
The Grand Mouse 高貴的老鼠
pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.
 
pdurrant's Avatar
 
Posts: 71,406
Karma: 305065800
Join Date: Jul 2007
Location: Norfolk, England
Device: Kindle Voyage
Quote:
Originally Posted by Skeezix View Post
Looks like something changed with kindlegen 2.7. Using kindlestrip on a MOBI generated with 2.7 corrupts the file, so that my Kindle Keyboard cannot read it. The stripped MOBI I generated with 2.5 (using the same opf) works fine.
Thanks for the note. Time to take a look at KindleGen 2.7!
pdurrant is offline   Reply With Quote
Old 10-27-2012, 06:42 AM   #54
pdurrant
The Grand Mouse 高貴的老鼠
pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.
 
pdurrant's Avatar
 
Posts: 71,406
Karma: 305065800
Join Date: Jul 2007
Location: Norfolk, England
Device: Kindle Voyage
Now fixed. First post updated.

KindleGen 2.7 adds an additional record that stores the output from KindleGen. KindleStrip almost took account of this, but failed on one line. KindleStrip 1.35 now handles this correctly, and this extra record also gets stripped out properly.
pdurrant is offline   Reply With Quote
Old 10-27-2012, 01:36 PM   #55
Skeezix
Enthusiast
Skeezix began at the beginning.
 
Posts: 28
Karma: 10
Join Date: Jun 2011
Device: Kindle
Thanks for fixing that so quickly!
Skeezix is offline   Reply With Quote
Old 11-01-2012, 02:35 AM   #56
steppe
Enthusiast
steppe began at the beginning.
 
Posts: 32
Karma: 10
Join Date: Jun 2012
Device: Kindle 4 NT (broke), Kindle Paperwhite, Kindle 2
Thumbs up

Quote:
Originally Posted by pdurrant View Post
Now fixed. First post updated.
Thank you from the bottom of my heart for this useful tool. I compiled a fixed layout ebook with about 200 pages of photos and the file size with the source attached was over 80MB. KDP doesn't allow one to upload files bigger than 50MB, but the thing is my file is only 33 MB without the source files attached by KindleGen 2.7. So if your file plus the source is bigger than 50MB there is no way you can use KDP to remove the source attachment. Thanks to KindleStrip I removed the attachment and uploaded the 33MB file without any problems. The resulting fixed layout ebook works great both in the "Online Previewer" and in Kindle Previewer.
steppe is offline   Reply With Quote
Old 11-02-2012, 01:24 PM   #57
SusanM
Bemused by possibilities
SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.SusanM ought to be getting tired of karma fortunes by now.
 
SusanM's Avatar
 
Posts: 58
Karma: 480244
Join Date: Jul 2012
Device: iPad3, Kobo
Kindlestrip

Thank you for sharing!
SusanM is offline   Reply With Quote
Old 11-02-2012, 08:53 PM   #58
jayz
Member
jayz began at the beginning.
 
Posts: 18
Karma: 10
Join Date: Jun 2012
Device: kindle fire,android
How do you use it on a mac?
jayz is offline   Reply With Quote
Old 11-03-2012, 04:14 AM   #59
pdurrant
The Grand Mouse 高貴的老鼠
pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.pdurrant ought to be getting tired of karma fortunes by now.
 
pdurrant's Avatar
 
Posts: 71,406
Karma: 305065800
Join Date: Jul 2007
Location: Norfolk, England
Device: Kindle Voyage
Quote:
Originally Posted by jayz View Post
How do you use it on a mac?
Download the AppleScript application.. (KindleStrip 1.35.app.zip)

Unzip it by double-clicking on the zip file, if your browser hasn't already unzipped it for you.

Drag an ebook created by KindleGen onto the AppleScript application.


In general, only use it for books you're distributing yourself, not for books uploaded to KDP. Steppe's situation was unusual.
pdurrant is offline   Reply With Quote
Old 11-03-2012, 04:28 AM   #60
jayz
Member
jayz began at the beginning.
 
Posts: 18
Karma: 10
Join Date: Jun 2012
Device: kindle fire,android
"not for books uploaded to KDP"
Why? Can you explain.
jayz is offline   Reply With Quote
Reply

Tags
k5 tools, mobi2mobi

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Applescript Wrapper Application for Kindlegen pdurrant Kindle Formats 50 02-18-2020 01:16 AM
how to use python script with windows xp tuufbiz1 Other formats 12 01-08-2011 08:22 AM
How do I get a shortcut for a Python script onto the taskbar in W7? Sydney's Mom Workshop 6 03-28-2010 08:11 PM
Nedd a little help with a python script gandor62 Calibre 1 08-07-2008 09:59 PM
Python script to create collections gwynevans Sony Reader Dev Corner 2 03-13-2008 12:29 PM


All times are GMT -4. The time now is 06:16 PM.


MobileRead.com is a privately owned, operated and funded community.