Mobiperl Mobiperl - Perl tools for handling MobiPocket files

[Gudy]

Originally Posted by wallcraft

I don't see how it could be considered an effective DRM technique. Compression is also highly unlikely to be covered by patents[...]

Eh? Says who? There was a big noise a couple of years ago when some firm which had acquired the patent for the LZW(?) compression used in GIF image files started successfully enforcing said patent. If nothing else, it gave the hitherto lingering PNG format a nice boost.

Legally speaking, the Mobipocket compression scheme is probably not part of their DRM, so you likely don't have to fear the DMCA, but it may very well be covered by a patent. This patent in all likelihood would also cover a decompression routine for the compressed data.

[HarryT]

Originally Posted by jswinden

I do wish these guys would start creating executables. Having to install Python, Perl, NameYourStupidScriptingLanguage, etc., then having to attempt to figure out their poorly documented command line sytax is too complicated and time consuming for most people. And in the end, it always seems that the caveats make the whole process a huge waste of time!

Don't you think that you're perhaps being a little unreasonable in your complaints? The people who write these tools are putting in a huge amount of time and effort, and releasing them free of charge. We should be thanking them for their hard work, not complaining. That's my view, anyway!

[Jaapjan]

There is little you can do about these kind of complaints Harry. You'll always have people which are unwilling to put some of their own efforts into these tools to get it adapted or working for their situation. No matter if it is reading up to see if the tool does what they want, install a few support libraries or anything between those.

Sad fact of any developer, no matter on what program they're working.

[DMcCunney]

Originally Posted by Gudy

Eh? Says who? There was a big noise a couple of years ago when some firm which had acquired the patent for the LZW(?) compression used in GIF image files started successfully enforcing said patent. If nothing else, it gave the hitherto lingering PNG format a nice boost.

That was Unisys.

Lev and Zemple defined the original algorithm. Terry Welch described a simplified version that was easier to implement, becoming the W in LZW. At the time he wrote the paper describing the LZW algorithm, Terry worked for Sperry, now a unit of Unisys, and his contract gave his employer rights to his creations.

Unisys belatedly woke up, realized they had intellectual property rights to LZW compression, and started asking for money from sites with GIF images that used LZW compression.

As mentioned, it resulted in the PNG format, and it's a moot point now -- as far as I know, Unisys's rights have expired.

I wonder how much money they actually got from trying to enforce rights on LZW? I suspect not as much as they spent in legal fees doing it.

Legally speaking, the Mobipocket compression scheme is probably not part of their DRM, so you likely don't have to fear the DMCA, but it may very well be covered by a patent. This patent in all likelihood would also cover a decompression routine for the compressed data.

If it is covered by patent, that will be discoverable. I doubt it, though. I suspect this is "trade secret" -- they just don't tell anyone what they did and how they did it.
______
Dennis

[HarryT]

If it's huffman encoding, it's certainly not patented - that algorithm is in the public domain.

[Gudy]

Originally Posted by DMcCunney

As mentioned, it resulted in the PNG format, and it's a moot point now -- as far as I know, Unisys's rights have expired.

Yup, since 2003 in the US and since 2004 in most of the rest of the world.

Originally Posted by DMcCunney

If it is covered by patent, that will be discoverable. I doubt it, though. I suspect this is "trade secret" -- they just don't tell anyone what they did and how they did it.

The latter would be ideal for someone trying to reverse-engineer the algorithm, say, through chosen plain-text attacks or something similar. And yes, if there is a patent, it should theoretically be discoverable.

Also, Harry, it may be Huffman, but their claims lead me to believe that they made changes to the compression algorithm so they can start decompressing anywhere they need so they can start presenting page 500 in a book without having to decode all previous pages. Those changes may very well be patented.

[DMcCunney]

Originally Posted by Gudy

Also, Harry, it may be Huffman, but their claims lead me to believe that they made changes to the compression algorithm so they can start decompressing anywhere they need so they can start presenting page 500 in a book without having to decode all previous pages. Those changes may very well be patented.

Correct. They've stated that they couldn't simply use something like the Zip "Deflate" algorithm because they wanted to be able to start decompressing at any specified point in the file (like the dictionary definition for a particular word), and Zip always started at the beginning.

I think it would be a smart move for them to make the Creator and Reader programs open source efforts, but that proprietary compression algorithm might be an issue in doing so. It seems like something they wouldn't want to release.
______
Dennis

[JSWolf]

Ok, I installed it all and get all kinds of errors running MAKE. What do I do now?

[nrapallo]

tompe:

I finished adding the ability to directly convert to .IMP formats from the .html converted by your 'mobi2html' perl script. I used your 'mobi2html' as a base (version 0.0.28) and created a new perl script named 'mobi2imp.pl'.

Therein, I indicate what changes were made so that you (or I ) can update this module in future releases.

The 'mobi2imp.pl' is attached below and I also provide two sample conversions in the below .zip file for anyone who wants to test it out.

Feel free to include and/or modify this within your Mobiperl package!

You directly may not benefit (or even be able to convert to .IMP if not using Windows and the eBook Publisher software), but I believe this will allow those with many mobipocket .prc files to migrate to their ebookwise 1150 easily.

Is this OK with you?

EDIT 12 Feb 2008: version 2 - now 'Category Author Title' are optional and don't need to be provided. See mobi2IMP.bat and .zip file for details.

Regards,
-Nick

[Gudy]

Originally Posted by JSWolf

Ok, I installed it all and get all kinds of errors running MAKE. What do I do now?

I assume you have installed ActiveState's Perl and all the modules listed on the Mobiperl site. (You can install PAR-Packer-588 from the Perl Package Manager as well.), and also nmake from the MS KnowledgeBase article. (Copy NMAKE.ERR and NMAKE.EXE to C:\Perl\bin)

I also assume that you have extracted mobipocket-0.0.28.tar somewhere.

Go there and open the Makefile in your editor of choice. Use an editor that doesn't do anything to the file you don't tell it to do (like, say, converting tabs to spaces or "fixing" line lengths).

Delete all lines that start with "copy" and end with "c:\Perlb820\bin\"

Open a command prompt to that directory and type "nmake all"

Rejoice.

[igorsk]

Originally Posted by DMcCunney

Correct. They've stated that they couldn't simply use something like the Zip "Deflate" algorithm because they wanted to be able to start decompressing at any specified point in the file (like the dictionary definition for a particular word), and Zip always started at the beginning.

What they did was just compress source HTML in chunks so that each chunk after compression fits inside a PDB record. They also added an index for mapping an uncompressed text position to record number so that only necessary records need to be decompressed. None of this actually requires Huffman, it can be done with deflate or lzw or lzma or whatnot.

[JSWolf]

Originally Posted by Gudy

I assume you have installed ActiveState's Perl and all the modules listed on the Mobiperl site. (You can install PAR-Packer-588 from the Perl Package Manager as well.), and also nmake from the MS KnowledgeBase article. (Copy NMAKE.ERR and NMAKE.EXE to C:\Perl\bin)

I also assume that you have extracted mobipocket-0.0.28.tar somewhere.

Go there and open the Makefile in your editor of choice. Use an editor that doesn't do anything to the file you don't tell it to do (like, say, converting tabs to spaces or "fixing" line lengths).

Delete all lines that start with "copy" and end with "c:\Perlb820\bin\"

Open a command prompt to that directory and type "nmake all"

Rejoice.

What I get now is an unknown command "pp".

I did install everything according to the directions. But pp is not found.
What is pp and where do I get it?

[Gudy]

pp is part of the Par-Packer-588 package, so it looks like this one didn't install right.

Start the Perl Package Manager.
Check that http://theoryx5.uwinnipeg.ca/ppms/package.lst has been added under Edit -> Preferences -> Repositories
There should be two entries under that tab. That one and the default entry from ActiveState.

Now, check that View -> All Packages is active, then type "PAR-Packer" without the quotes in the search box.

You should see two entries. PAR-Packer (0.976) and PAR-Packer-588 (0.973). The latter, and only the latter, should be installed in the "site" area. If not, right click that entry and install it.

(ETA: One thing that may not be readily apparent: Going right click -> Install on an entry does not actually install the package, but merely queues it for installation. You need to click the green right arrow button (center button on the right side of the search field) to actually execute all queued actions.)

Then try again.

[JSWolf]

Now I get the following error...

Can't locate IO/Compress/Gzip.pm in @INC (@INC contains: C:/perl/site/lib C:/perl/lib .) at C:/perl/site/lib/Compress/Zlib.pm line 13.

Where do I get this?

[Gudy]

Buh? I believe that something about your Perl install is well and truly b0rked.

I certainly don't have Zlib.pm at that location...

Since I have no clue what's happening here, I declare myself at the end of my wits. Until someone comes along who knows more, try the attached mobiperl build. *crosses fingers*