Calibre2Web

[itimpi]

UPDATE

The Calibre2Web software has now been superceded by the mcuh more advanced and function rich Calibre2Opds software. The home site for the Calibre2Opds software is: https://wiki.mobileread.com/wiki/Calibre2opds

I have been developing the attached for my own use - and thought it might be of use to others? It generates the catalogs needed to allow Stanza for the iPhone/iTouch to download books via a personal web site. This bypasses the need for Stanza Desktop, and also allows for remote access to ones eBook collection when away from home

At the moment Calibre2Web will only run on a PC under Windows. This is because it is developed in VBScript (and hence the PC restriction) while I learnt about the Catalog formats used by Stanza.

If you are using another environment you may find it worth downloading just to look at the REAME file? I would be interested in any feedback on my ideas for how to develop the functionality in the future.

I am not sure if I am duplicating functionality freely available elsewhere. If so feel free to mention it as there is no point in putting effort into something that someone else has already done better.

EDIT: The Calibre2opds program is a replacement for Calibre2Web that is almost functionally identical, and has the great advantage of being written in Java making it cross-platform compatible capable of running on all the platforms that Calibre runs on. It now has some functionality that goes beyond what calibre2Web provides. I do not therefore intend to develop Calibre2Web any further and will instead support the Calibre2opds activity.

[kovidgoyal]

Cool, I look forward to the python version. I'm getting more and more tempted to get an iPhone

[itimpi]

Cool, I look forward to the python version.

I hope so

I finally have a Windows based Calibre development system up and running so I can work on that. I put the VBScript based version out early to try and get initial user feedback on what it does to see if others have the same need or whether they are after something different.

At the moment I am torn between trying to get the functionality up and running in a free-standing program, or whether to try and integrate it from the outset into the Calibre web server. I am leaning towards it being a free-standing program as I would find that easier to develop initially, although I would use many of the DB access routines already used by Calibre which should help protect from future database changes (if any). Do you have any strong views?

Another advantage as I see it from getting it running as a free-standing program would be that it could be run as a CGI program directly on the web server (as long as it supported python). It should not need any UI components.

[pilotbob]

Originally Posted by itimpi

I am not sure if I am duplicating functionality freely available elsewhere. If so feel free to mention it as there is no point in putting effort into something that someone else has already done better.

I'm confused.. how is this different than the content server provided by calibre?

BOb

[itimpi]

There are quite a few differences !

The first is that it can be run on a web server that is not the PC running Calibre (which applies to me)

More important is that that the Calibre server simply gives a flat structure consisting of a list of books. This program gives you a structure with a top level choice of Titles, Authors, Series. When you drill down a level it allows you to select between "All" or starting with a specific letter. You can now drill down another level. If going down the Authors path you can see which book belong to Series and which do not.

Basically it is more like the online catalog provided with Stanza. It makes handling large book collections much easier.

[kovidgoyal]

Originally Posted by itimpi

I hope so

I finally have a Windows based Calibre development system up and running so I can work on that. I put the VBScript based version out early to try and get initial user feedback on what it does to see if others have the same need or whether they are after something different.

At the moment I am torn between trying to get the functionality up and running in a free-standing program, or whether to try and integrate it from the outset into the Calibre web server. I am leaning towards it being a free-standing program as I would find that easier to develop initially, although I would use many of the DB access routines already used by Calibre which should help protect from future database changes (if any). Do you have any strong views?

Another advantage as I see it from getting it running as a free-standing program would be that it could be run as a CGI program directly on the web server (as long as it supported python). It should not need any UI components.

Feel free to develop it as a free standing program if you're more comfortable with that, though, I would like to eventually integrate it into calibre's server, as this makes it possible for people who don't run a web server to use it easily.

Note that you can run the calibre server directly in apache using mod_python, with only minor modifications to the code, though it is easier (and more robust) to run it as a standalone server and reverse proxy to it in apache.

[pilotbob]

Originally Posted by itimpi

There are quite a few differences !

The first is that it can be run on a web server that is not the PC running Calibre (which applies to me)

I see. Thanks for the info. So, what do you do.. you still have to put the library on the web server, right? So, why not run calibre-server on that PC?

Also, I would bet that Kovid would accept patches that improved the navigation of the web server program.

I was just asking because you requested to know if this was a "duplicate" effort and it does seem to be similar to calibre-server work.

BOb

[itimpi]

The machine I keep my Calibre library on and run my web server on is a NAS box that is not capable of running Calibre. Leaving a PC switched on just to run Calibre Server is not appropriate.

I have already agree with Kovid that the long term aim is to make the facility available as part of the Calibre suite. The intention is to be able to run the facility in both free-standing mode, and also as part of the Calibre server. However in the short term the free-standing facility is likely to come first as I can work on that independently of Kovid's work elsehere in Calibre and leave the integration into Calibre server as a second stage.

[kovidgoyal]

Use the interface in LibraryDatabase2 for database operations, this is more stable than the format of metadata.db itself.

[itimpi]

Thanks - that tells me where to look.

Although any individual fragment of code seems clear enough in isolation, trying to find ones way around the entirety of the Calibre source code is a bit daunting at the moment.

[pilotbob]

Originally Posted by itimpi

The machine I keep my Calibre library on and run my web server on is a NAS box that is not capable of running Calibre.

I see. I didn't know you could run a web server on a NAS box. And, if it can run web server software, it can't run other software? So, how it is able to run your software (calibre2web) that you mention?

Thanks,
BOb

[kovidgoyal]

Originally Posted by itimpi

Thanks - that tells me where to look.

Although any individual fragment of code seems clear enough in isolation, trying to find ones way around the entirety of the Calibre source code is a bit daunting at the moment.

Feel free to ask for navigation pointers. The calibre library code is in the module calibre.library. In particular the database interface is the class LibraryDatabase2 in database2.py and the content server is in server.py

In addition the file cli.py defines the calibredb command line interface to the database

[itimpi]

Conversion to Python is going slower than I hoped. The problem is not the Python language but getting aquainted with all the add-on libraries that are used with Calibre.

On that basis here is an improved version of my original calibre2web VBScript based solution. Any feedback on whether it is useful or not would be welcomed.

[astrodad]

Originally Posted by pilotbob

I see. I didn't know you could run a web server on a NAS box. And, if it can run web server software, it can't run other software? So, how it is able to run your software (calibre2web) that you mention?

Thanks,
BOb

Many of the new NAS boxes, such as the ReadyNAS from Netgear and Drobo from Data Robotics support running programs such as FTP servers, HTTP Servers, torrent clients, iTunes music servers, etc.

I for one would love something like this to serve up my content from a NAS, which can be accessed over the Internet.

[itimpi]

That is how I am using my current calibre2web VBScript based program. I run it on a PC to generate a set of static catalogs which are then hosted (together with the eBooks) under a web server on my NAS box. I can then access those catalogs using Stanza on my iPhone either over my home LAN, or remotely via a 3G network. Currently I have about 2000 books in the library so having good remote navigation around them and on-demand download is extremely convenient.

The python variant I am now working on will be capable of acting as CGI style code, dynamically generating catalog pages on demand, thus avoiding the need to pre-generate the static catalogs which should be much more effecient. It will also provide search capabilities which cannot be done with static catalogs. I still intend to leave in the ability to also generate the static catalogs for those who cannot run python based CGI programs on their NAS box/ Web server.

I was intending the Python variant to provide basically the same catalog structure as the VBScript generated static catalogs provide which was why I am interested in feedback on whether others think the current catalog structure is useful/appropriate