Mobiperl Mobiperl - Perl tools for handling MobiPocket files

[tompe]

Originally Posted by JeffElkins

Nope, didn't work.

If you do just mobi2mobi on the input file what is the output? When you say it did not work does it mean that you got an error message or just that you did not see the image? If you do "mobi2html file.mobi unpackdir" is the coverimage then in the unpackdir?

If you send me the file I will test to add a cover image and fix eventual bugs. Send it in a private message here or visa email to tpe at ida.liu.se

[JeffElkins]

Originally Posted by tompe

If you do just mobi2mobi on the input file what is the output? When you say it did not work does it mean that you got an error message or just that you did not see the image? If you do "mobi2html file.mobi unpackdir" is the coverimage then in the unpackdir?

If you send me the file I will test to add a cover image and fix eventual bugs. Send it in a private message here or visa email to tpe at ida.liu.se

jpg --outfile newmymobi.mobi
Database Name: 01_Master_and_Commander.html
      Version: 0
         Type: BOOK
      Creator: MOBI
         Seed: 293
        Resdb:
 AppInfoDirty:
        ctime: 1200688185 - Fri Jan 18 15:29:45 2008
        mtime: 1200688201 - Fri Jan 18 15:30:01 2008
      baktime: -2082844800 - Thu Dec 31 19:00:00 1903
---------------------------------------------------
FIRST IMG Record 0
---------------------------------------------------
Image record index: 265 (477 x 800)
IMAGE INDEX: 265
PDHEADER  Version: 2
PDHEADER   Length: 1078340
PDHEADER NRecords: 264
PDHEADER  Recsize: 4096
PDHEADER  Unknown: 0
MOBIHEADER    ciflg: 65535
MOBIHEADER    ciptr: 65535
MOBIHEADER  doctype: MOBI
MOBIHEADER   length: 228
MOBIHEADER booktype: 2 - BOOK
MOBIHEADER    codep: 1252
MOBIHEADER   uniqid: 2968844172
MOBIHEADER      ver: 4
MOBIHEADER  exthflg: 80
MOBIHEADER language: 9 - 9 - 0 - ENGLISH -
COVEROFFSET:
EXTH doctype: EXTH
EXTH  length: 52
EXTH n_items: 2
EXTH    item: 100 - Author - 18 - Unspecified Author
EXTH    item: 201 - CoverOffset - 4 - 0x0000
LONGTITLE: 01_Master_and_Commander.html
-----------------
Setting record 265 to 01_cover.jpg
Reading data from file: 01_cover.jpg
Setting extended header data: coveroffset - 0
EXTH setting data: coveroffset - 201 - 0 - 0x30
EXTH replacing data: 201 - 0 - 0x30
GETSTRING: Author - Unspecified Author
CoverOffset - not printable

PM on the way...

[tompe]

I think I need the file to fix this (haven't got it yet...).

[JeffElkins]

Originally Posted by tompe

I think I need the file to fix this (haven't got it yet...).

http://www.elkins.org/attach.zip

You can download it from the link above. The attachment wouldn't take via a PM.

[tompe]

Originally Posted by JeffElkins

You can download it from the link above. The attachment wouldn't take via a PM.

Well, there is a bug in my code. Will fix it and fix some other bugs and make a new release during the weekend.

[JeffElkins]

Originally Posted by tompe

Well, there is a bug in my code. Will fix it and fix some other bugs and make a new release during the weekend.

Thanks for the hard work you're putting into this.

[tompe]

Originally Posted by JeffElkins

Thanks for the hard work you're putting into this.

It is still fun I found two bugs with your test example so a good test case. It is working now adding cover image. I will fix the author/image bug for files without an EXTH header tomorrow and then make a new release.

[JeffElkins]

Originally Posted by tompe

It is still fun I found two bugs with your test example so a good test case. It is working now adding cover image. I will fix the author/image bug for files without an EXTH header tomorrow and then make a new release.

Two more bugs I've encountered:

1. lit2mobi crashes when saving .lit file: http://www.elkins.org/example1.lit

2. lit2mobi completes conversion, but book has no carriage returns or paragraph breaks: http://www.elkins.org/example2.lit

Both the original lit files seem fine using Microsoft Reader.

[tompe]

Originally Posted by JeffElkins

Two more bugs I've encountered:

1. lit2mobi crashes when saving .lit file: http://www.elkins.org/example1.lit

2. lit2mobi completes conversion, but book has no carriage returns or paragraph breaks: http://www.elkins.org/example2.lit

Both the original lit files seem fine using Microsoft Reader.

1. It does not crash for me. When I release the next version you can test it again and tell me if it crashes or not.

2. The problem is that Mobipocket does not support the <pre> tag. But I added my fix for that to lit2mobi so it look readable now.

[JeffElkins]

Originally Posted by tompe

1. It does not crash for me. When I release the next version you can test it again and tell me if it crashes or not.

2. The problem is that Mobipocket does not support the <pre> tag. But I added my fix for that to lit2mobi so it look readable now.

Thanks Tommy!

[tompe]

Version 0.0.26 with Windows binaries is available from:

http://www.ida.liu.se/~tompe/mobiperl/


Changes in 0.0.26:

• --prefixtitle now works for mobi2mobi and when input is a PalmDOC file.
  
• Fixed bug with mobi2mobi and --coverimage. It did not work if the file did not contain any images.
  
• Fixed rescaling bug. Now also images whose height is greater than 640 are rescaled. This is done because of a bug in the Gen3.
  
• Fixed bug in mobi2mobi --fiximagesizes. In Linux the wrong tmp file was read.
  
• Introduced a flag --gen3imagefix to mobi2mobi that can be used if you have a book that hang the Gen3. These kind of hangings are often caused by a large image in the book. This is a firmware bug in the Gen3.
  
• Added call to fix_pre_tags in lit2mobi
  
• Now it is possible to add author information to a Mobipocket file that does not have an EXTH.

[JeffElkins]

Originally Posted by tompe

Version 0.0.26 with Windows binaries is available from:

http://www.ida.liu.se/~tompe/mobiperl/


Changes in 0.0.26:

• --prefixtitle now works for mobi2mobi and when input is a PalmDOC file.
  
• Fixed bug with mobi2mobi and --coverimage. It did not work if the file did not contain any images.
  
• Fixed rescaling bug. Now also images whose height is greater than 640 are rescaled. This is done because of a bug in the Gen3.
  
• Fixed bug in mobi2mobi --fiximagesizes. In Linux the wrong tmp file was read.
  
• Introduced a flag --gen3imagefix to mobi2mobi that can be used if you have a book that hang the Gen3. These kind of hangings are often caused by a large image in the book. This is a firmware bug in the Gen3.
  
• Added call to fix_pre_tags in lit2mobi
  
• Now it is possible to add author information to a Mobipocket file that does not have an EXTH.

lit2mobi example1.lit
Unpack file example1.lit in dir ctmp
+---[ ConvertLIT (Version 1.8) ]---------------[ Copyright (c) 2002,2003 ]---
ConvertLIT comes with ABSOLUTELY NO WARRANTY; for details
see the COPYING file or visit "http://www.gnu.org/license/gpl.html".
This is free software, and you are welcome to redistribute it under
certain conditions.  See the GPL license for details.
LIT INFORMATION.........
DRM         =  1
Timestamp   =  a6b86513
Creator     =  00000058
Language    =  00001009
Writing out "Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld" as "Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm" ...
Successfully written to "ctmp/Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm".

Writing out "RW_~Cover01" as "~Cover01.jpg" ...
Successfully written to "ctmp/~Cover01.jpg".

Writing out "RW_~Cover02" as "~Cover02.jpg" ...
Successfully written to "ctmp/~Cover02.jpg".

Writing out "RW_~Cover03" as "~Cover03.jpg" ...
Successfully written to "ctmp/~Cover03.jpg".

Writing out "RW_~Cover04" as "~Cover04.jpg" ...
Successfully written to "ctmp/~Cover04.jpg".

Writing out "RW_~Cover05" as "~Cover05.jpg" ...
Successfully written to "ctmp/~Cover05.jpg".

Exploded "example1.lit" into "ctmp/".
Read in HTML tree from opf
Opf: Initialize from file: example1.opf
CONTENT: <?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE package
  PUBLIC "+//ISBN 0-9673008-1-9//DTD OEB 1.0.1 Package//EN"
  "http://openebook.org/dtds/oeb-1.0.1/oebpkg101.dtd">
<package unique-identifier="OverDriveGUID">
 <metadata>
  <dc-metadata xmlns:dc="http://purl.org/dc/elements/1.0/" xmlns:oebpackage="http://openebook.org/namespaces/oeb-package/1.0/">
   <dc:Title>Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld</dc:Title>
   <dc:Creator role="aut">Philip Jose Farmer</dc:Creator>
   <dc:Identifier id="OverDriveGUID" scheme="GUID">{54B56D18-7D64-42EB-AE2A-3D55AC2117DF}</dc:Identifier>
  </dc-metadata>
  <x-metadata>
   <meta name="rwver-ReaderWorks-SDK-Control" content="1.0.1.817 (08/17/2001)" />
   <meta name="rwver-HTML-Input-Filter" content="1.0.1.0817 (08/17/2001)" />
   <meta name="rwver-Image-Input-Filter" content="1.0.1.817 (08/17/2001)" />
   <meta name="rwver-Text-Input-Filter" content="1.0.1.817 (08/17/2001)" />
   <meta name="rwver-LIT-file-generator" content="1.5.1.0280 (10/05/2000)" />
   <meta name="rw-License-Key" content="RWPTL" />
  </x-metadata>
 </metadata>
 <manifest>
  <item id="Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld" href="Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm" media-type="text/html" />
  <item id="RW_~Cover01" href="~Cover01.jpg" media-type="image/jpeg" />
  <item id="RW_~Cover02" href="~Cover02.jpg" media-type="image/jpeg" />
  <item id="RW_~Cover03" href="~Cover03.jpg" media-type="image/jpeg" />
  <item id="RW_~Cover04" href="~Cover04.jpg" media-type="image/jpeg" />
  <item id="RW_~Cover05" href="~Cover05.jpg" media-type="image/jpeg" />
 </manifest>
 <spine>
  <itemref idref="Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld" />
 </spine>
 <guide>
  <reference type="other.ms-thumbimage-standard" href="~Cover01.jpg" />
  <reference type="other.ms-coverimage-standard" href="~Cover02.jpg" />
  <reference type="other.ms-titleimage-standard" href="~Cover03.jpg" />
  <reference type="other.ms-thumbimage" href="~Cover04.jpg" />
  <reference type="other.ms-coverimage" href="~Cover05.jpg" />
 </guide>
</package>
OPF: TITLE: Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld
OPF: CREATOR: Philip Jose Farmer
Init from manifest
Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld - Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm - text/html
RW_~Cover01 - ~Cover01.jpg - image/jpeg
Could not read image file: ~Cover01.jpg
RW_~Cover02 - ~Cover02.jpg - image/jpeg
Could not read image file: ~Cover02.jpg
RW_~Cover03 - ~Cover03.jpg - image/jpeg
Could not read image file: ~Cover03.jpg
RW_~Cover04 - ~Cover04.jpg - image/jpeg
Could not read image file: ~Cover04.jpg
RW_~Cover05 - ~Cover05.jpg - image/jpeg
Could not read image file: ~Cover05.jpg
Warning, RW_~Cover01 missing from spine, adding
Warning, RW_~Cover02 missing from spine, adding
Warning, RW_~Cover03 missing from spine, adding
Warning, RW_~Cover04 missing from spine, adding
Warning, RW_~Cover05 missing from spine, adding
Init from guide
OPFTITLE: Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld
OPFAUTHOR: Philip Jose Farmer
Coverimage: ~Cover02.jpg
SPINE: adding Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld - Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm - text/html
Adding: Farmer, Philip Jose - Riverworld 06 - ( Shorts) Tales of Riverworld.htm - Farmer,_Philip_Jose_-_Riverworld_06_-_(_Shorts)_Tales_of_Riverworld
+++.+SPINE: adding RW_~Cover01 - ~Cover01.jpg - image/jpeg
SPINE: adding RW_~Cover02 - ~Cover02.jpg - image/jpeg
SPINE: adding RW_~Cover03 - ~Cover03.jpg - image/jpeg
SPINE: adding RW_~Cover04 - ~Cover04.jpg - image/jpeg
SPINE: adding RW_~Cover05 - ~Cover05.jpg - image/jpeg
All spine elements have been added
Have Read in HTML tree from opf
FIX PRE TAGS
Saving mobi file (version 4): example1.mobi
COVEROFFSET: 0
THUMBOFFSET: 1
EXTH setting data: author - 100 - Philip Jose Farmer - 0x5068696c6970204a6f7365204661726d6572
EXTH add: author - 100 - Philip Jose Farmer
EXTH setting data: coveroffset - 201 - 0 - 0x30
EXTH add: coveroffset - 201 - 0 - 0x30
EXTH setting data: thumboffset - 202 - 1 - 0x31
EXTH add: thumboffset - 202 - 1 - 0x31
MOBIHDR: imgrecpointer: 207
EXTH setting data: author - 100 - Philip Jose Farmer - 0x5068696c6970204a6f7365204661726d6572
EXTH add: author - 100 - Philip Jose Farmer
EXTH setting data: coveroffset - 201 - 0 - 0x30
EXTH add: coveroffset - 201 - 0 - 0x30
EXTH setting data: thumboffset - 202 - 1 - 0x31
EXTH add: thumboffset - 202 - 1 - 0x31
New record for image 207: ~Cover02.jpg
Reading data from file: ~Cover02.jpg - 510 x 680
[Image::BMP] ERROR: Not a bitmap: [~Cover02.jpg] at /usr/local/bin/MobiPerl/Util.pm line 488

Thanks for this update!

For me it still crashes on:

http://www.elkins.org/example1.lit
and
http://www.elkins.org/example3.lit

Both fail with a hard crash at /usr/local/bin/MobiPerl/Util.pm line 488

edit: http://www.elkins.org/example4.zip

Add coverimage fails:

mobi2mobi oldmobi.mobi --coverimage cover.jpg --outfile newmobi.mobi

When exploded, the coverimage .jpg has been included in the newmobi.mobi file


However, the globbed text seems to be corrected.

BTW, Think about setting up a paypal tip jar on your site. At the very least I'm sure you'll gain beer money...I know I owe you a case of your favorite

[tompe]

Aha, this is a bug I have not managed to track down since I did not have a test file. It is also a bug that only occur in Windows. So next time I boot Windows I will try to fix it using your test files.

There is one thing you could test. Could you test to rename the file to a name without ~ and change the name in the opf file and then run opf2mobi and see if it works?

Otherwise you could just remove the image files and the reference to them and run opf2mobi since the image files are not related to the book.

[JeffElkins]

Originally Posted by tompe

Aha, this is a bug I have not managed to track down since I did not have a test file. It is also a bug that only occur in Windows. So next time I boot Windows I will try to fix it using your test files.

There is one thing you could test. Could you test to rename the file to a name without ~ and change the name in the opf file and then run opf2mobi and see if it works?

Otherwise you could just remove the image files and the reference to them and run opf2mobi since the image files are not related to the book.

Sure, I'll test this today. Also, I don't run Windows. All my testing is done with Kubuntu 7.10

EDIT: Just tested. Removing the ~ has no effect. lit2mobi still crashes. However, editing the .opf file and removing image file references allows opf2mobi to complete.

[tompe]

Originally Posted by JeffElkins

Sure, I'll test this today. Also, I don't run Windows. All my testing is done with Kubuntu 7.10

Very good to know. It works for me using Debian unstable. Could it maybe be different versions of clit we are using? I use version 1.8.

The problem I think is that the reading of the file fails on line 486 in Util.pm. When that fails it tries to read it as a BMP file. But the file is a gif file with a jpg filename. So it should be read by "my $p = new GD::Image ("$filename");". What you could do is print out $filename at that point and confirm that it is the same as in the ctmp directory.

You could also write a simple perl script that just tries to read the problematic file and see if it works.

Since I do not manage to trigger this bug it is a bit hard for me to debug it...