LIT output

[llasram]

Hot off the digital presses, Kovid has just released calibre version 0.4.117, which integrates my LIT-generation support. For now, LIT-generation is available only on the command-line. For your command-lining pleasure, calibre provides two new tools: 'any2lit', which works basically like 'any2epub' and will convert just about anything to a LIT e-book; and 'oeb2lit', which more-or-less directly encapsulates OEBPS 1.x or 2.0 content into an OEBPS 1.x LIT e-book.

The tools have most of the features you would expect, and have the following shortcomings you might not:

• We still don't know the LIT anchor-hashing algorithm. Some links in books with more than six anchors per "page" (if using any2lit) or per HTML file (if using oeb2lit) will not work.
• CSS and markup is not yet transformed to the subset MSReader understands. The subset MSReader supports is somewhat idiosyncratic, and a future release will transform content written to deform gracefully into content which deforms gracefully.

Let us know of any improvements you'd like to see, and happy converting!

(P.S. to a moderator -- perhaps this should be made sticky?)

[=X=]

llsram, Great news! I'm glad LIT support made it into calibre.

Anyhow I've already got to do some testing. I've created two LIT files one for MOBI and one from HTML. The MOBI causes MSReader to crash and the HTML version creates a blank LIT file.

I've attached the LIT files, MOBI file, the HTML file, and the command line used to create the LIT from HTML file.

=X=

[llasram]

Originally Posted by =X=

Anyhow I've already got to do some testing. I've created two LIT files one for MOBI and one from HTML. The MOBI causes MSReader to crash and the HTML version creates a blank LIT file.

Nooooooooooo... After all the fighting to get the LZX compression code building on Windows, it appears that "building" is not the same as "working."

Ok, so another short-coming -- does not yet work on Windows.

To keep you interested while I track down the problem, attached is what the "any2lit" converted Mobipocket book is supposed to look like.

[llasram]

Originally Posted by llasram

Ok, so another short-coming -- does not yet work on Windows.

As it turns out, Windows is not a big-endian platform. Who knew? I've pushed a fix to Kovid, so it should hopefully end up in the next release. Apologies for the wait...

[=X=]

Originally Posted by llasram

As it turns out, Windows is not a big-endian platform. Who knew? I've pushed a fix to Kovid, so it should hopefully end up in the next release. Apologies for the wait...

Big-endian/little-endian is processor specific not OS. Chances are your going to run into little/big endian issues on Linux as well.

BTW I took a look at the LIT file you posted. It looks like the links do not work on the LIT file. I checked the MOBI and they do work. Also there are issues with the CSS. I suspect this is because you are converting to ePUB first then to LIT. ePUB uses CSS2 I don't know what version LIT supports, but CSS2 is stricter than CSS1. (Note the CSS problem is caused because CSS2 does not allow styles to start with a number, where this was valid in CSS1)

=X=

[llasram]

Originally Posted by =X=

Big-endian/little-endian is processor specific not OS. Chances are your going to run into little/big endian issues on Linux as well.

Er, yes. That was what passes for a joke in my tribe . There was a preprocessor check for big-endianess in the GPLed LZX compression code I appropriated. Unfortunately neither of the two preprocessor macros it compared are defined by the included set of headers under VC 2008, which meant the condition evaluated to "true".

BTW I took a look at the LIT file you posted. It looks like the links do not work on the LIT file. I checked the MOBI and they do work.

That would be one of the two outstanding issues already mentioned. Until someone figures out the hashing algorithm MSReader uses, links into files with more than six anchors won't work. Any2lit splits the input at page boundaries, which improves matters somewhat, but the book in question still specifies more anchors than will work. I could go a step further in working around the problem by only indexing anchors which are actually the targets hyperlinks, but that's just another layer of complexity / slowness which won't be necessary once we figure out the algorithm.

Also there are issues with the CSS. I suspect this is because you are converting to ePUB first then to LIT. ePUB uses CSS2 I don't know what version LIT supports, but CSS2 is stricter than CSS1. (Note the CSS problem is caused because CSS2 does not allow styles to start with a number, where this was valid in CSS1)

And that would be the other outstanding issue mentioned. Right now neither any2lit or oeb2lit do much of any style/markup pre-processing to produce something more palatable to MSReader. I've got some CSS "flattening" code which should get things most of the way there, but there are still some kinks I need to work out.

[igorsk]

You can detect endianness at runtime. Grabbed from some random page:

Uint32 Value32;
Uint8 *VPtr = (Uint8 *)&Value32;

VPtr[0] = VPtr[1] = VPtr[2] = 0; VPtr[3] = 1;

if(Value32 == 1)
   printf("I'm big endian\n");
else
   printf("I'm little endian\n");

[=X=]

Originally Posted by llasram

Er, yes. That was what passes for a joke in my tribe.

Ah well yes, it was late I see the sarcasm now .

Originally Posted by llasram

That would be one of the two outstanding issues .. .

My aplogoizes I did not mean to rehash known issues.

Originally Posted by igorsk

You can detect endianness at runtime. Grabbed from some random page:

Uint32 Value32;
Uint8 *VPtr = (Uint8 *)&Value32;

VPtr[0] = VPtr[1] = VPtr[2] = 0; VPtr[3] = 1;

if(Value32 == 1)
   printf("I'm big endian\n");
else
   printf("I'm little endian\n");

I use to be in favor of dong these checks in my code, but as systems are transitioning from 32 to 64 bit. It will complicate ones code w/o need. Its best to allow the OS to handle the edian/memory. Just make sure the macros work .

=X=

[macthekitten]

yikes i'm scared to even ask a question

Someone emailed me a LIT file, can I use Calibre to convert it to something the Sony PRS 700 can read?

[pilotbob]

Originally Posted by macthekitten

Someone emailed me a LIT file, can I use Calibre to convert it to something the Sony PRS 700 can read?

Yes. Assuming it is not DRM protected.

BOb

[=X=]

Last night I integrated any2lit on BookCreator. I've noticed that the final LIT files are not utilizing the style sheet. And as a result do not format as expected.

I'm not sure if it's a result of LIT or the opf that is built, here is why.
As a test I also decompiled the LIT file and created a mobi file with mobigen using the opf file. And the mobi file also did not use the style sheets.

At this point I'll admit I have no clue what I'm talking about, I just know when I do the above step with properly formatted LIT or MOBI files using the opf file seems to generate the expected results.

Also again on the TOC topic. I know you mentioned you have no intention of adding the level of support that exist for any2lrf or any2epub, but can you add a switch option where the user can pass in their own HREF to a toc element in the opf file. That is how WordRMR handles it and it works for the most part and is a simple solution.

=X=

[llasram]

Originally Posted by =X=

Also again on the TOC topic. I know you mentioned you have no intention of adding the level of support that exist for any2lrf or any2epub, but can you add a switch option where the user can pass in their own HREF to a toc element in the opf file. That is how WordRMR handles it and it works for the most part and is a simple solution.

Woah woah woah! I have every intention of adding the level of support which exists for any2lrf/any2epub. I'm just doing it kind of backwards... I pushed the current state to Kovid because I thought it would be useful to people to have *some* LIT-generation ability, but I see now that was a mistake. I'm almost done with Mobipocket generation, which I'm doing much more completely and I've implemented in a way which lets LIT and Mobipocket (and any other format) share everything which is common between. After that gets merged, I'm going to go back to LIT support and implement LIT-specific markup and CSS mangling, bringing it up to the level of the other supported formats.

As for your two specific issues -- MSReader should be picking up on an included stylesheet, with the caveat that MSReader doesn't support any contextual selectors. If you post one of the created LIT files I can hopefully figure out what's going on.

The TOC stuff I'm not 100% sure how to handle... If you generate your own OPF, then you can have a //guide/reference[@type="toc"] which MSReader will just pick up on. If you're letting any2* generate your OPF, I'm less sure... Kovid has been handling that layer, but I believe it just means adding an option to any2epub. I'll take a look at that this evening, as it would definitely be useful for Mobipocket support as well.

[=X=]

@llasram; Sorry for that last post as I did not mean to say that you did not have any intention to produce quality LIT. I was in particularly referencing the creation of the TOC. In a previous exchange between you and I you mentioned you did not now how to handle the TOC and where not worried about it. Now I'm realizing you just meant to put it off until a solution came to mind.

I'd also like to reassure you it was not a mistake to release the LIT support prematurely. You mentioned it was in beta and I understand that. I just thought I was being helpful buy providing you some feedback. My apologized is I came off the wrong way. How does that saying go "The way to hell is paved with good intentions"

I've attached both the LIT file and the HTML file that was used to create the LIT

=X=

[llasram]

Originally Posted by =X=

@llasram; Sorry for that last post as I did not mean to say that you did not have any intention to produce quality LIT.

And I'm sorry I was so touchy about it... I'm glad that you have found LIT generation useful though even as it is. So apologies all around, and now on with life .

The CSS issue... well, there are just so many ways something can go wrong. The CSS isn't being picked up in MSReader because I had a bug where I wasn't properly serializing relative URIs in LIT's compressed HTML format. If you extract the HTML then try to open it in a Web browser, the CSS wouldn't be picked up because I was switching the 'type' attribute of stylesheet-referencing <link/> tags to the OEBPS 1.x specific 'text/x-oeb1-css'. Then mobigen doesn't pick up the CSS because it can't handle (correctly!) URI-encoded filenames.

I've pushed fixes for the first two up to Kovid, but the mobigen issue is actually a bug in mobigen... You can report it to them, or just wait a few weeks for oeb2mobi/any2mobi .

[=X=]

Originally Posted by llasram

I've pushed fixes for the first two up to Kovid, but the mobigen issue is actually a bug in mobigen...

Great, !

Originally Posted by llasram

You can report it to them, or just wait a few weeks for oeb2mobi/any2mobi .

You kidding, of course I'll wait, I'm confident by the time your done with any2mobi it will be a much better product than mobigen.

Thank you
=X=