![]() |
[The Older version] Calibre2opds V2
Welcome from the calibre2opds team![/B]
The calibre2opds tool can generate OPDS and HTML catalogs from the metadata of the Calibre ebook library manager. These catalogs can then be used both to browse your ebook library online and also to download books on demand. Current Release: 2.4 Current beta: 2.5 beta 1 We hope you’ll have as much fun using this tool as we had writing it ! The calibre2opds team : David Pierron – lead programmer Dave Walker – programmer; documentation & features manager; tester Farid Soussi – html and css Useful Links The project website The project website (if the above does not work, until the DNS servers are updated, in 24h max.) Support Forum |
I uploaded a new release, which adds support for generating authors, series and recent additions catalogs.
|
You still need to generate the stanza.xml file, correct?
Edited to add: you do not need to generate the stanza file. |
Quote:
cheers |
da_jane: you found out eventually, but here is the confirmation : there is no need to generate the stanza.xml file, because it is in fact generated by the tool itself, in _catalogs/catalog.xml
Leep: this tool (and the Calibre2Web script it is based on) generates multiple catalogs, broken into smaller lists of books than the default Stanza catalog generated by Calibre. It is useful if you want to access your library, using Stanza, by author, serie, category or if you simply have a lot of books. |
The all books that is generated does not include all my books (and yes, they are in epub format).
|
I uploaded a new release (0.0.3), which includes an extensive test mode. Could you please download it, install it wherever you install it usually, and run it with the "--test" parameter, in the calibre library directory ?
Example : "c:\Program Files\calibredbxtract\bin\calibredbxtract.bat" --test >> test.log Assuming of course that the tool is installed in c:\Program Files\calibredbxtract ... This command will generate the test.log file, which is a text file that I'd kindly request you send me... (have a look inside if you're concerned about privacy, but basically it's a list of your books and the complete paths to their folders, along with the list of files that reside in these folders). If you generate and send me this file, I may be able to track the problem and correct it ! |
Quote:
Sending you the test.log shortly. |
Yet another release : this time, I corrected some (important) bugs that will help with finding the missing ePub books, and introduced a test mode to help people who have problems with the tool...
|
Jane is Da Bomb, and David is Da Man!
Question, Am I still going to have to find a way to deal with the spaces in my titles or is that already built in? Thanks to you both! Beau |
I have it working now, after a few problems (David did wonders fixing) The spaces in the names have no effect. I have many spaces and and I now have all my books visible.
|
To make it even easier for users, you can create a "batch file".
Go to Start -> Run Type "notepad" in the Run box. Notepad will open. Paste two lines of code: First is the pathname to the calibredbxtract bin folder. The easiest way to do this is to copy and paste the path from the address line in a windows explorer window. If you've placed it in the c:\Program Files\Calibre2\calibredbxtract then the first line of code will be c:\Program Files\Calibre2\calibredbxtract\bin The next line of code is where your Calibre library is calibredbxtract.bat + c:\Documents and Settings\Username\Dropbox\Public\secretname\ Again, there should be two lines of code in your file that look something like this: cd C:\Program Files\Calibre2\calibredbxtract\bin calibredbxtract.bat C:\Documents and Settings\User\My Documents\My Dropbox\Public\secretfoldername Save as "catalogmaker.bat" or something. Has to end with bat. This creates an executable file and all you have to do is double click on the file to run the command to create your catalogs. |
I released a new version (0.0.5) which corrects the problem reported by Bruce : the tool was missing a lot of books.
Please download it from the project page and report if you find any more bugs. Also, feel free to register an account at Sourceforge and submit bug reports, and support or feature requests, I'll be happy to implement them ! PS : I've got some nice features in mind, I hope you'll like them ;) PPS : Jane, I included your helpful explanations in the tool's readme file ; I hope you don't mind ! |
Quote:
To answer your question (not that I need to, Ian already did) : no, you won't have problems with spaces nor with parentheses ; this tool is smart ;) |
Thanks David! Version 0.0.5 really works beautifully.
|
I released a new version (0.0.6) which adds a lot of stuff :
Please download it from the project page ; I hope you'll enjoy it, remember to report if you find any bugs. Also, feel free to register an account at Sourceforge and submit bug reports, and support or feature requests, I'll be happy to implement them ! PS : I'd like to warmly thank Jane and Ian for their patience, their support, and their help ! |
I tried the latest version against my library which has about 8000 books in it and got the following
Code:
11:06:53,521 INFO [Catalog] Generating tags catalogNote that as well as being 8000 books, there will normally be about 4 formats for each books plus the cover so there are lot of files and folders in the library (total size on disk is ~12GB). The system I was running on was using Window 7 Pro and has 2GB RAM. |
The problem is that Java, by default, only reserves 64Mb of RAM for its processes.
In your case (a very, very large library), this is not enough. I made a correction, which will be included in the next release. In the meantime, you should be able to run the tool if you edit the calibredbxtract.bat file, inside the bin subfolder of the directory where you installed the tool : replace the line showing set EXTRA_JVM_ARGUMENTS= with set EXTRA_JVM_ARGUMENTS=-Xmx256m |
Quote:
Quote:
Quote:
Maybe you should add soething about the RAM requirements v Library size to the README file? You do not want to make the tool grab a lot of RAM for those who have much smaller libraries so simply telling people what to do if the error requires might be the way forward? |
Quote:
I think that 256 Mb of RAM is a very small chunk of most modern computers, especially with virtual memory ; also, the tool is meant to run a few times every hour or day, and will be running for 5 to 20 seconds on most computers. So, IMHO, leaving this as the default value is perfectly acceptable, and the advanced users who know they don't need as much and want to minimize the tool's impact on their computer can edit the script... Anyway, thanks for pointing out the problem, I have such a small library (176 books) that it would never have happened to me ;) Regards, David. PS: I took for granted that you used Windows, but the edit I mentionned works also in the Unix shell script that resides in the same directory, of course... |
With the RAM set to 256MB the tool completed without an issue (it looked a though peak RAm was about 110MB). However it took a LONG time, and generated 15314 files totalling 260MB in size!
Some comment on the output:
I have (finally) started getting seriously into the Python version that will be able to handle large libraries as the pages will be generated dynamically on demand directly from the metadata database. If you do not mind I will "borrow" some of the ideas from your implementation. One big advantage that dynamic pages give is the ability to support search functions as well as avoiding the time overhead of generating the static catalogs. I am trying to write the Python is such a way that it can either be integrated into Calibre, or alternatively run as a free-standing piece of code under a web server such as Apache (so that calibre itself does not need to be running). Whether I can actually achieve that I am not yet sure. It may as a by-product also be able to output the static catalog files - do you have any objection to it possibly superseding your tool? |
Of course, one advantage of a thoroughly free-standing web-database such as this is that we Apple users can dump the output into our "Sites" folder on our iDisk and access it anywhere without running a server at all!
Calibre doesn't take much resource when it's just running the online library. What's the advantage of your Python code? Better customization? Or are you just trying to avoid having to run 2 servers? Since it's in Python, I wonder if there are some features that would be useful if rolled into Calibre. - Jim |
You know what ? When I read and answered your first post about memory requirements, I didn't realize that it was you ;)
If I had, I'd have started a conversation about the project ! Quote:
I found out that Stanza did not support this feature of the OPDS standard, and then I rewrote everything to include the full book entry in the books lists. This, of courses, makes for a much bigger output than the first solution... And then, the thumbnail generated are, of course, a huge part in this big space... Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
That's great (having news about the Python version at last), I can't hide that I began this project because I thought that your Python script development was cancelled ! And of course I don't mind you borrowing, I even would like to help if I can... Maybe I can explain the pitfalls I managed to avoid, and if I can upgrade my Python-fu to second dan, I may even be able to help you code ? To be honest, I coded this in Java to be quick, because I know this language very well. But I know a bit of Python, and I'd love to learn more, if you're patient enough ;) Quote:
Feel free to contact me (dpierron at gmail dot com) about this, whatever your opinion about merging is ; in any case, I'll be more than happy to help and to share ! Cheers, David. |
Quote:
Quote:
Quote:
|
Quote:
Quote:
The thought was that the Python version would follow quite quickly, but the Calibre2Web script worked well enough that there was not enough personal pressure to get on with the Python version. There was also the overhead of learning one's way around the exisitng Calibre code. Now that my library size is growing I find that the static catalog approach is becoming a bit unwieldy in terms of the time required to generate the catalogs. Quote:
Quote:
Cheers, Dave |
Thank for useful tool, I have question, how can I edit or set text which is generated to html / xml pages, for example this text: "Alphabetical index of the single author - very useful indeed ;)"
|
Hi cistehovado, you're welcome !
There are two parts in my answer... First, the tool is not internationalized yet (although it's on my todo list, and if I've got enough time before XMas, I'll make it in the next release). When it will be internationalized, the texts will still not be user-editable, but if you want to translate the tool in your language, you'll be more than welcome ! At this point, all that will be needed will be to translate an english text file into whatever language you like, and send it back to me... Piece of cake, when you speak both english and that other language ;) Secondly, this cheesy sentence is there because there is only one author in your authors list ; this should not happen often, but maybe I could enhance the tool by skipping the authors list altogether when there is only one (and maybe, also extend this concept to other items ?)... I'll see what I can do. Regards, David. |
I can translate it of course, but I need all text used in program.
I really do have only 1 author ;-) I thought that that message there will be always, but its kinda programmer joke I see ;-) |
Quote:
Quote:
|
new release 0.0.7
1 Attachment(s)
I released a new version (0.0.7) which adds and corrects a lot of stuff :
Please download it from the project page ; I hope you'll enjoy it, remember to report if you find any bugs. Also, feel free to register an account at Sourceforge and submit bug reports, and support or feature requests, I'll be happy to implement them ! Beginning in this release, it is possible to translate the tool in other languages than english. I already contributed a french translation, and I count on you guys to help me get really international ;) The translation files are attached to this post ; they should be self-explanatory, if not, feel free to contact me. When a translation is finished, send the files to me and I'll include them in the next release. In these files, lines starting with a dash (#) are comments, and you have to escape (by doubling them) single quotes (i.e. ' becomes '') |
1 Attachment(s)
I translated almost all,
only the part "information messages" (and some few lines) are not translated yet, I need to see output from translated program, so I can choose sentences and words better ;-) |
Thanks cistehovado !
If you want to test, and know how to extract/compress a zip file, here is what you can do : Find the file named StanzaOutput-0.0.7-SNAPSHOT.jar, somewhere in the tool's installation directory (should be in repo\fr\free\davidsoft\calibre\StanzaOutput\0.0.7-SNAPSHOT). Extract this file (it's a simple zip file, use Winzip or, even better, Total Commander) and add your Enumerations_sk.properties and Localization_sk.properties files along with the _en and the _fr ones. Recompress the zip file, and start the tool with the "--lang sk " parameter and voilà ! |
Ok I will try it at home :D where do I have my book collection, btw dont you have an idea how to solve my problem https://www.mobileread.com/forums/sho...244#post706244
|
No, sorry, I'm not fluent in Python ; I'm certain Kovid will react and help you quite fast, he's usually reactive.
|
1 Attachment(s)
I made a new release (0.0.8) ; you'll find it at the usual place...
Changelog :
I'm preparing a demo catalog, which will be referenced in the tool's website ; I'll let you know when it'll be ready. For the people who would like to help in translating the tool, here are the translation files (attached) ; I'm ready to help anyone willing to help me ;) Last but not least, I'd like to thank everyone who supported me and specially Ian, who was kind and patient and whose excellent ideas were the base of almost all the options of the tool ;) Thanks Ian ! Cheers, David. |
:thanks: for this excellent tool, I was just about to write a library to access the calibre DB myself, so you saved me quite a bit of work :2thumbsup
One question, though, would you like to deploy the artifacts from your library directly to central? It would make using your code as a library for other application easier and would help with making other developers aware of your code also. I'm using the services of the good people at sonatype.org to mirror some of my artifacts (for now osmosis and postgis) to central, too, so I could lend a hand if necessary. Regards, Hakan |
Thanks for the compliments !
My tool is made of three libraries : one for accessing the Calibre db and modelizing the data, one which is a generic tool library, and one that is the actual catalog exporter. The problem is, these libraries evolve very fast at the moment, while I'm at the start of the project (and in a beta phase). Later, I'll happily oblige and publish the libraries in the M2Central repository, but for the moment I think that it would be quite unpractical. Don't you agree ? Anyhow, you're more than welcome to either make use of these open-source components, and even more welcome to come join me in this project, doing whatever you wanted to build at first... Regards, David. |
Change of name (the tool is now called calibre2opds), hosting (on Launchpad) and a new release (1.0.0) ...
I've written a welcome page to explain everything, and there's even a demo HTML catalog available - and also, a demo OPDS catalog, if you want to try it in Stanza (although no downloads of eBook files are permitted in the demo) |
Would it make me a irrecoverable megalomaniac if I asked a moderator to pin this thread ? Ooops too late, I did ask ;o)
|
A new version has been released :
|
| All times are GMT -4. The time now is 07:32 PM. |
Powered by: vBulletin
Copyright ©2000 - 3.8.5, Jelsoft Enterprises Ltd.
MobileRead.com is a privately owned, operated and funded community.