html2mobi (a mobigen replacement written in Perl)

[igorsk]

dwSize is not a pointer. It's the size of MOBI header (EXTH immediately follows it so it looks like a pointer). For version 3 set it to 0x74. EXTH contains only metadata (and DRM ids) so it shouldn't matter if it's missing.

[tompe]

Originally Posted by igorsk

dwSize is not a pointer. It's the size of MOBI header (EXTH immediately follows it so it looks like a pointer). For version 3 set it to 0x74. EXTH contains only metadata (and DRM ids) so it shouldn't matter if it's missing.

But it seems to matter or something else is wrong. On Gen3 I get a corrupted title and this corruption changes depending on how long the MOBI header is. So I assume it tries to read unitialized memory. Or how do I specify that the metadata does not exists?

What is the difference between version 3 and 4 and why is it preferred to use version 3?

[igorsk]

Version 4 adds support for DRMv2 (PID-based) and extents MOBI header to 0xD0 bytes. It's not that v3 is "preferred" but it should be enough for your purposes. Maybe Cybook's parser assumes header v4 and doesn't check the actual size. Since you say you get corrupted title, my guess is that it expects the extended title record to be present (in addition to the title at the beginning of the prc file). This extra info is specified by these fields:

44 Dword  dwTitleOffset  //from the start of record 0, codepage is dwCodepage
48 Dword  dwTitleLength

Here's the format of EXTH header which immediately follows the MOBI header (i.e. at 0x10+MOBI.dwSize in rec0). I beleive you also need to set flag 0x40 in dword at MOBI+0x70 if EXTH is present.

0 dd dwSignature //'EXTH'
4 dd dwSize  //including everything
8 dd dwCount //count of extra data items
extra item:
  0 dd id
  4 dd size
  8 <size-8> data

ids:
  1: drm_server_id
  2: drm_commerce_id
  3: drm_ebookbase_book_id
100: Author
101: Publisher
102: Imprint
104: ISBN
105: Subject
106: PublishingDate
107: Review
108: Contributor
109: Rights
110: SubjectCode
111: Type
112: Source
113: ASIN
114: VersionNumber
115: Sample
116: StartReading
203: hasFakeCover

[mrkai]

Originally Posted by igorsk

I beleive you also need to set flag 0x40 in dword at MOBI+0x70 if EXTH is present.

...and in the Version 4 mobi files generated by the latest Creator, this seems to be set to 0x50.

-K

[igorsk]

Well, 0x50 does include 0x40

[tompe]

Thanks a lot. I will add the extended header tonight and see if it works.

In the Alice mobi file dwTitleOffset is 0xFFFFFFFF so I assume that means that this title is not there. Where is this title usually placed? Is it in the MOBI header or after it?

But using EXTH information seems more flexible so I might just go with that.

Actually I had set the MOBI+0x70 flag to 0x50 whan I tested so that might explain why the Gen3 reader misbehaved when no extended header was available.

[igorsk]

Extended title is somewhere after the EXTH header (it's not present in EXTH records btw). If not present, the one from the start of PRC is used (I think it's assumed to be in 1252 codepage).

[tompe]

I am trying to parse the EXTH using the mobi file in the Alice in Wonderland example from Mobipockets web pages. It does not seem to follow your description exactly. The title string is in the block but what I get is:

 EXTH doctype: EXTH
EXTH  length: 152
EXTH n_items: 7
ITEM: 64 16 - 100 22 - Carroll, Lewis
ITEM: 6e 11 - 110 17 - FIC004000
ITEM: 69 10 - 105 16 - Classics
ITEM: 12c 2e - 300 46 - ì¾ôíù
ITEM: c9 c - 201 12 - 
ITEM: cb c - 203 12 - 
ITEM: ca c - 202 12 -

Ideas?

[igorsk]

IDs 3xx and 2xx are binary fields. I don't know their purpose (yet?). As I said, you shouldn't worry about EXTH now, it shouldn't be necessary for your purpose.

[tompe]

Originally Posted by igorsk

IDs 3xx and 2xx are binary fields. I don't know their purpose (yet?). As I said, you shouldn't worry about EXTH now, it shouldn't be necessary for your purpose.

But I cannot get it to work on the Gen3 at all without using this. I tested now to set the EXTH flag to 0 and to not have any data at all after the MOBI header. My Gen3 said "Multipart books not supported" or something similar. The only way I have got it to load is by having an EXTH.

Maybe I should test just adding an author in EXTH. That might work...

[mrkai]

Igorsk, do you have a mobi v3 sample file? It looks like something is a bit different in v4 with regards to the EXTH...

-K

[tompe]

I missread a hextable. The title I saw was the extended title and when I added extended title I get correct display of title in the library. But when I open the file the Gen3 hangs or says that Multobook is not supported. So there probably is some remaining problem with the data in the header.

Well, some progress at least. Back to more testing...

[tompe]

It works! It seems that the Gen3 did not like images of size 600x800.

So here is the first release. The state of the code is very alpha. Please let me know if it does not work or if you find bugs or if you have suggestions for enhancements. I have tested with the Alice in Wonderland exemple from Mobipockets web site and I load it to my Gen3 and to my Palm T5 and read it. Here are the scripts:

http://www.ida.liu.se/~tompe/mobiperl/mobiperl-0.01.tar

perldoc -t html2mobi

NAME
    html2mobi - A script to convert html files or an opf file to mobi

SYNOPSIS
    html2mobi file.html

    html2mobi file1.html file2.html ...

    html2mobi file.opf

DESCRIPTION
    A script to convert html files or an opf file to a mobi format file.

OPTIONS
    --title TITLE
        Specify the title for the book. This overrides the value given in
        the opf file.

    --mobifile MOBIFILE
        Name of the output file. This overrides the default value.

    --htmlfile HTMLFILE
        Saves the html that is packed into mobi format. This html code
        contains Mobipocket specific things that are added automatically.
        This is mostly useful for debugging.

    --coverimage IMAGE
        The image to be used as cover in a library listing like the one in
        Cybook Gen3. The image will be rescaled to a suitable format
        (180x240). If no image is specified the first image in the source
        files is used.

    --gentoc
        For a collection of html files generate the table of contents
        automatically.

    --pda
        Scale images to work for pda's (must be used for Alice to work on my
        Palm T5).

    --scale f
        Scale all images that are smaller then a certain size with scale
        factor f.

EXAMPLES
       html2mobi Alice_In_Wonderland.opf

       html2mobi Alice_In_Wonderland.html

TODO
       - Specify margins with flags

       - Follow local links when given a root html file

BUGS
       - Image sizes is not handled correctly for Palm T5. Large images
         works on Gen3 but are problematic on T5.

       - Guide specified toc not generated

       - Not correct author in EXTH

       - Plus a lot more.... this is an alpha version

AUTHOR
    Tommy Persson (tpe@ida.liu.se)

[JSWolf]

What image sizes are you allowing up to? Are you resizing if you have a LRGE image?

[tompe]

Originally Posted by JSWolf

What image sizes are you allowing up to? Are you resizing if you have a LRGE image?

Well I will allow whtever the user want to specify and try to find out what the limts are on different devices. If the limits are different. Since the Mobigen format is not documented anywhere it is hard to know so the best you can do is test it for specific devices.

The cover image should be 600x800. I resized it to 80% of that for now. I also have s scale argument so you can scale all your images if you want. I will add whatever feature is requested in connection with images.