Call for imp support in Calibre

[scruffy]

Way back (6 months ago) I wrote:

Originally Posted by scruffy

Kovid, so long as we're on the subject... if you added in support for the GEB/Ebookwise, you'd make hundreds of new fans... adding IMP support would be a huge boost, and the existing code is already out there... In fact, I don't even own a Sony reader, but I've used libprs500, because your converters do a good job (for example, mobi2oeb). If you make libprs500 ebook agnostic, supporting imp/mobi/epub/lrf/oeb /etc, you'd increase the number of eyeballs and supporters...

And the answer:

Originally Posted by kovidgoyal

In principle, sure, practically, I don't have the time, I typically write new converters only when I need them But I'm happy to accept patches.

With all of the new imp based development happening... maybe someone can take the time to add this finally?

If there was a single ebook library that worked with my ebookwise, my n800 running fbreader, etc, I'm sure the total eyeballs (and potential developers scratching their itches) on calibre would jump dramatically...

[ashkulz]

There are multiple aspects to this:

1. adding support for reading metadata from .IMP and .RB books
2. adding support for storing .IMP and .RB books in the library
3. adding support for creating IMP and .RB books
4. serving .IMP and .RB books from calibre

#1 can be done very easily, it's a very small change.
#2 -- no idea, but should not be too much of an issue.
#3 -- from what I've seen, calibre prefers to do everything as a python library (possibly using ctypes) with no calls to external processes. If this policy can change, then impmake can be called to go from html -> imp (impmake can be renamed as html2imp) with the same concept applied to rbmake. This needs to be an external process as it may be running impmake under wine, else it would be supported under Windows only (which is not an incentive to me as I use Linux only). Also, duplicating rbmake's functionality can be quite an effort.
#4 -- this is a bit complex, as there are different methodologies for all 3 ebook versions.
for the 1100, you need something replacing rebcomm for USB communication.
for the 1150, you need something replacing eb1150 for replacing USB communication and in addition impserve running as a proxy (plus you need to configure your ebook).
for the 1200, you need to have impserve running as a proxy (plus you need to configure your ebook).

I'm pretty sure #1 and #2 should be acceptable to Kovid for patches. I don't think that #3 should be an issue for integrating into calibre. For #4, lots of thought will have to be given on re-doing (or re-using the existing tools) and deciding on the apporach (i.e. whether to integrate the changes or have it as an unofficial add-on).

I hope Kovid can chime in with his views also

[kovidgoyal]

1) and 2) are trivial.

As far as 3) is concerned, calibre needs a way to convert HTML+OPF files (preferably recursively) into .imp. It has no problems calling an external binary to do this (it calls pdftohtml for example and used to call clit). The problem is that whatever method is used needs to be cross-platform and distributable with calibre (so using wine for example is not really an option).

4) Is not so crucial as you can use the save to disk functionality to put them into a folder from where the server programs can automatically sync them.

[ashkulz]

I've added a patch for #1 as ticket #993. I'd appreciate some pointers on what needs to be done for #2 (I think it is taken care of by the patch, but I'm not sure).

Also, for #3: I think it'd be possible to check that impmake exists on the path (e.g. using impmake -v) and if so, doing the html2imp conversion [or else bailing out]. For bundling, impmake depends on eBook Publisher so that'd be required to be bundled. I feel that leaving it out of the bundle (and checking its presence at runtime) would make more sense, as unfortunately there is no native HTML -> IMP conversion (you'll need to build an almost complete HTML renderer which matches their ROM and fonts).

What do you think?

[kovidgoyal]

Your patch takes care of both 1) ad 2)

[ashkulz]

What do you think about what I posted about #3 above (not sure if you saw it or not)?

[kovidgoyal]

But impmake exists only on windows and under wine?

[mollybo]

Just saying that I would be delighted with .imp support. While I use a Sony 505, my husband switches between an old Rocket and an eBookwise. I'd love to share my converted books with him since I've built up quite the database in Calibre.

[scruffy]

Originally Posted by ashkulz

Also, for #3: I think it'd be possible to check that impmake exists on the path (e.g. using impmake -v) and if so, doing the html2imp conversion [or else bailing out]. For bundling, impmake depends on eBook Publisher so that'd be required to be bundled. What do you think?

I think there are a few options...

1) impmake

1a) or maybe using lit2sb (ie the perl script that Nick extracted/tweaked see https://www.mobileread.com/forums/showthread.php?t=28138 )

1b) or any other tool that needs the SBPublisher.Builder to make an imp.

Since Calibre can generate a opf and a cover image, a reduced version of lit2sb (ie ignoring the lit extract code parts) should suffice. Requires windows for SBPublisher, so it's not cross platform. [That hasn't stopped other tools like PDFRead from this answer, just don't continue on to creating an IMP on non-windows platforms... stop with the html/OPF/Cover, which is near exactly where Calibre stands today.]

2) Convert code (written in VB?) into python from
http://krausyaoj.tripod.com/reb1200.htm
License is open enough to allow using hunks of code wholesale likely, and at the end, hopefully we have a python native imp builder. (subst Perl or other language for Python if you wish.)

3?) rbmake: http://rbmake.sourceforge.net/rbmake.html
Completely opensource, and has the advantage of supporting older Rocket readers. But converting from rb to imp isn't trivial, though... so this isn't really a good answer, since it would require using something else to convert to imp still... but might be worth considering, if someone wanted the older readers to work with Calibre in the future.

[kovidgoyal]

I think I good solution is to add the infrastructure to allow users to add their own custom OPF -> output format tool via an option to "Add external tool"

Once the infrastructure is in place it will be easy to add profiles for imp and mobi based on impmake and mobigen.

Unfortunately, all my current calibre hacking time is allocated to other things, so the only way this will get done is if someone contributes a patch.

the way I envison it working is via "recipes" for tools. A recipe should define the command line options the tool accepts (at the very least a way to define the output file). Calibre can have a nice GUI to aid in the creation of recipes. Once a recipe is created, calibre will create a generic config dialog (like the LRF config dialog) based on the defined options and a user specified path to the actual binary and then it will be able to run the tool to convert any of the supported iinput formats into the desired output format (calibre already has code to generate an OPF file given an HTML file + metadata). calibre can ship with built-inrecipes for a couple of well known tools like mobigen and impmake.

[scruffy]

Originally Posted by kovidgoyal

I think I good solution is to add the infrastructure to allow users to add their own custom OPF -> output format tool via an option to "Add external tool"
...
calibre can ship with built-in recipes for a couple of well known tools like mobigen and impmake.

Nice idea - Makes sense and easy to grow. Fbreader support, OEB support, epub support, etc.

[JSWolf]

I create IMP eBooks by using Mobi2IMP. Needs eBook Publisher installed as well as mobi2html. Though I might like to see Mobi2IMP use Mobi2oeb.

[scruffy]

Originally Posted by kovidgoyal

Your patch takes care of both 1) ad 2)

Testing 4.84b6:

dragging an imp works fine...

all of the rbs I've tried had the title/author reversed (could be all of my rbs are screwy... ?) Author - title.rb was the filenames...

Add books - not working, not listed in types in the dialog box of filetypes supported. Needs some code support for the new types of imp and rb to look for?

[kovidgoyal]

b6 doesn't have the patch. You can adjust how calibre reads metadata from filenames by using the config dialog (avanced section).

[scruffy]

Originally Posted by kovidgoyal

b6 doesn't have the patch.

Oops, looks like the timing was good enough to have it in there already.

MOVING part of reply to other thread, as it's not IMP related...