Test oeb2mobi output for me?

[tompe]

Originally Posted by nrapallo

*Thank you* for finally confirming my suspision that the byte count to the filepos/link is "off" in mobi2html (and consequently in Mobi2IMP). I've had to sometimes add upto 200 extra bytes to find the "anchor" tag the filepos was referring to in my conversions from .prc to .imp. I had no idea why I had to do this and never would have thought the UTF-8 decoding could have precipitated this, but it does make awful good sense to me now that you mentioned this!

My Mobi2IMP solution (which was a brute force naive approach) was to scan forward in the uncompressed text (html) from the stated filepos position and look for the first '<' to plop the anchor (for that filepos)! 99% of the times it worked, but it was not elegant nor foolproof!

If you give ma a pointer to or a file with this problem I can see if it is easy to fix.

[nrapallo]

Originally Posted by tompe

If you give ma a pointer to or a file with this problem I can see if it is easy to fix.

Sure can!

Most Feedbooks.com Mobipocket/Kindle offerings have this problem (as they usually are UTF-8 encoded).

For example, see The Ant King and Other Stories ebook (.mobi).

The extracted files/results of Mobi2IMP are included in the .zip below. The below .txt file shows the dos window output of Mobi2IMP. In particular, look at the section following:

Adding name attributes
FIXED 3: 0000026250 (6) - Wasn't an anchor: reak/><a

Note the number in parentheses i.e. 6 shows how many characters over that I had to go to find the first "<". If I could find one within the first 200-300 bytes, then I would print FIXED, otherwise I would just issue a WARNING.

I've always seen this behaviour with Feedbooks.com .prc/.mobi ebooks.

[llasram]

Originally Posted by tompe

If you give ma a pointer to or a file with this problem I can see if it is easy to fix.

It looks like Mobiperl also isn't handling the "standard" variable-width integer encoded trailing data indicated by the other bits of the extra data flags field.

[pdurrant]

I think these must be the bytes that were messing up the first version of the Mobipocket decoder, and that the second version tried to fix without really understanding them. (The fifth version seems to handle them correctly, although I still haven't followed exactly what's going on here.)

It would be very nice to get the controlling bits and format of the trailing bytes set out clearing in the wiki...

Originally Posted by llasram

It looks like Mobiperl also isn't handling the "standard" variable-width integer encoded trailing data indicated by the other bits of the extra data flags field.

[llasram]

Originally Posted by pdurrant

I think these must be the bytes that were messing up the first version of the Mobipocket decoder, and that the second version tried to fix without really understanding them.

Indeed -- I'm not sure the community as whole really understands them. Calibre's rules for parsing them look to be the same as mobidedrm 0.5's, except that Calibre will ignore the extra data flags field if the MOBI header is shorter than 0xe4 bytes or *longer* than 0xe8 bytes. I vaguely recall being responsible for the test case which led to that one, but I'm now suspecting it may have been an interaction with an earlier version of mobidedrm. And neither handles bit 1 of the extra data flags, although Calibre will when Kovid gets around to pulling from lp:~llasram/calibre/staging .

Originally Posted by pdurrant

It would be very nice to get the controlling bits and format of the trailing bytes set out clearing in the wiki...

I've got a whole ream of stuff I figured out writing oeb2mobi that I need to add to the wiki... The effect of bit 1 of extra data flags, the format of "uncrossable" boundary records, the format of the FCIS record, and the format of the index records (although I'm still working on that...).

[kovidgoyal]

Originally Posted by llasram

And neither handles bit 1 of the extra data flags, although Calibre will when Kovid gets around to pulling from lp:~llasram/calibre/staging .

Done. I've also refactored mobi2oeb to use lxml instead of BeautifulSoup for a significant speedup

[tompe]

Originally Posted by llasram

It looks like Mobiperl also isn't handling the "standard" variable-width integer encoded trailing data indicated by the other bits of the extra data flags field.

I will wait for the wiki description of this...

[llasram]

Originally Posted by tompe

I will wait for the wiki description of this...

Done .

[Elsi]

You may already know how this displays in the Kindle, but here are some scans of the screen. (Kindle's screenshot function wasn't working; not sure why.) I scanned @ 300dpi, then reduced the image, exported to JPG with 15% optimization, so the fuzziness is due to the scanning, not the screen itself.

On two of the images, I circled a portion of the author field that doesn't display properly. Also, I really like the way the chapters begin 1/2 way down the page & hope to get your CSS so I can apply it to the books I'm making.

[llasram]

Originally Posted by Elsi

You may already know how this displays in the Kindle, but here are some scans of the screen.

Awesome! Thank you . Looks pretty good... I may leave the cover generation at 600x800. Hmm...

Originally Posted by Elsi

On two of the images, I circled a portion of the author field that doesn't display properly.

Yarh... Already fixed. Although it looks like it's been re-arranged -- does the Kindle treat the ',' or '&' character specially, like trying to rearrange "Last, First" to "First Last"?

Originally Posted by Elsi

Also, I really like the way the chapters begin 1/2 way down the page & hope to get your CSS so I can apply it to the books I'm making.

Heh. I think that's actually a bug. It's supposed to be only 5 or so lines down, and shows up that way in Mobipocket Desktop. I'm actually hoping I've fixed when I post another build of the file . But if that's what you want, you should be able to achieve the effect with a 'margin-top: 50%' property on your chapter headers.

[Jellby]

Originally Posted by Elsi

On two of the images, I circled a portion of the author field that doesn't display properly.

Funny, it seems it takes the semicolon as a separator for different authors, and the comma as the separator between first and last name, so:

Alexandre Dumas, pè;re

(considering that the è entity is not decoded, it has a semiocolon then) is parsed as:

First author: pè (first name) Alexandre Dumas (last name)
Second author: re (last name)

Also, I really like the way the chapters begin 1/2 way down the page & hope to get your CSS so I can apply it to the books I'm making.

I usually try to get something a bit more space-efficient...

[wallcraft]

Originally Posted by llasram

I may leave the cover generation at 600x800.

I suggest making this tunable. A default of 600x800 would be ok, although 525x640 works well on a wider range of devices.

[Elsi]

Originally Posted by Elsi

Also, I really like the way the chapters begin 1/2 way down the page & hope to get your CSS so I can apply it to the books I'm making.

Originally Posted by llasram

Heh. I think that's actually a bug. It's supposed to be only 5 or so lines down, and shows up that way in Mobipocket Desktop. I'm actually hoping I've fixed when I post another build of the file . But if that's what you want, you should be able to achieve the effect with a 'margin-top: 50%' property on your chapter headers.

Originally Posted by Jellby

I usually try to get something a bit more space-efficient...

I'll agree that 1/2 the page is too far down, but I've not been happy with the default placement as shown in this first image (from The Moving Picture Girls). I also would like to try something like the pseudo-watermark used in the commercial book Fortune and Fate by Sharon Shinn as shown in the second image.

[pdurrant]

Oh - thank you. Do you have a sample file with bit one set? It looks to me like the mobipocket decoder will need adjustment to cope with such files.

Originally Posted by llasram

Done .

[tompe]

Originally Posted by nrapallo

Sure can!

Most Feedbooks.com Mobipocket/Kindle offerings have this problem (as they usually are UTF-8 encoded).

For example, see The Ant King and Other Stories ebook (.mobi).

I looked at the output from --rawhtml but could not find any UTF-8 characters... But there is null characters in the file. But that is the data directly from the Perl module unpacking the compressed data so this is probably releated to something else. UTF-8 ought not to produce null characters.

I thought that the unpacking of the record was totally independent on the character set used. Right or wrong?