Alternatives to Calibre?

[chaley]

Several times in the past, when I have joined discussions like this one the result has been personal insult. I have been called arrogant, moron, a^%ole, and worse. We will see what happens this time.

I can understand why people want their own database structure (yes, the file structure is part of a "database"). I can understand that without it, calibre does not meet their requirements. Nothing wrong with that -- people really do have different requirements. What I have difficulty understanding is why we developers are somehow bad people for not instantly stopping whatever we would rather be doing to build something like this.

People who work on calibre do so for a number of reasons. Here are mine. The most common is that I wanted calibre to do something that *I* wanted. This is why I got involved with calibre 3 years ago and why I have put more than 2 person/years of real time into the project. I wanted custom columns. I wanted more powerful searching. I wanted my devices to behave how I wanted them. I wanted restrictions (now virtual libraries). I wanted a more powerful template language. Etc.

Another reason to do something is that it grabs my attention and is interesting for some reason. That is why I worked on the set arithmetic part of find duplicates (a small part of what Kiwidude built). That is why I worked on column coloring and column icons. And so on. I had fun, which is more than adequate compensation.

A third reason to do something is that I like the requester. You might consider it a gift made simply for the pleasure of giving it, or as thanks for some help or service. I have done that several times.

I have two main reasons not to do something. The first is that it meets none of the above criteria: it is of no personal use, is not personally interesting, and is not a gift. The second is a reaction to how we are asked. I react badly to "entitlement", where someone seems to feel that I owe them my time simply because they exist. I react equally badly to being insulted because I don't agree with someone.

To come back to the topic of this post -- changing the file structure -- there aren't many calibre developers with experience inside this part of calibre's guts. I happen to be one of them, but changing the file structure is of zero interest to me. I would be just as happy it all the books in my library were named with a unique ID, totally opaque. In fact this *is* how I name books on my android devices. Why should I sign up for months of work, zillions of problem reports and some amount of abuse to do something I don't care about? No reason I can think of. Complaining about it over and over isn't going to change what I think.

Those of you who want a different file structure are where I was 3 years ago. You want something and no one else seems interested in building it. So build it yourself! At the risk of seeming to speak for Kovid, my experience is that he will integrate things into calibre if a) they work reliably for the millions of calibre users, b) don't increase his support load, c) do something useful, and d) he has confidence that the implementer will hang around long enough to help with the inevitable problems.

If you don't have the skills required to do it yourself then you must wait until someone comes along who does have the skills, acquire the skills, go somewhere else, or build your own solution. Your choice.

[kacir]

Originally Posted by chaley

People who work on calibre do so for a number of reasons. Here are mine. ...

Thank you for a valuable insight.

[EowynCarter]

Originally Posted by chaley

Several times in the past, when I have joined discussions like this one the result has been personal insult. I have been called arrogant, moron, a^%ole, and worse. We will see what happens this time.

I can understand why people want their own database structure (yes, the file structure is part of a "database"). I can understand that without it, calibre does not meet their requirements. Nothing wrong with that -- people really do have different requirements. What I have difficulty understanding is why we developers are somehow bad people for not instantly stopping whatever we would rather be doing to build something like this.

People who work on calibre do so for a number of reasons. Here are mine. The most common is that I wanted calibre to do something that *I* wanted. This is why I got involved with calibre 3 years ago and why I have put more than 2 person/years of real time into the project. I wanted custom columns. I wanted more powerful searching. I wanted my devices to behave how I wanted them. I wanted restrictions (now virtual libraries). I wanted a more powerful template language. Etc.

Another reason to do something is that it grabs my attention and is interesting for some reason. That is why I worked on the set arithmetic part of find duplicates (a small part of what Kiwidude built). That is why I worked on column coloring and column icons. And so on. I had fun, which is more than adequate compensation.

A third reason to do something is that I like the requester. You might consider it a gift made simply for the pleasure of giving it, or as thanks for some help or service. I have done that several times.

I have two main reasons not to do something. The first is that it meets none of the above criteria: it is of no personal use, is not personally interesting, and is not a gift. The second is a reaction to how we are asked. I react badly to "entitlement", where someone seems to feel that I owe them my time simply because they exist. I react equally badly to being insulted because I don't agree with someone.

To come back to the topic of this post -- changing the file structure -- there aren't many calibre developers with experience inside this part of calibre's guts. I happen to be one of them, but changing the file structure is of zero interest to me. I would be just as happy it all the books in my library were named with a unique ID, totally opaque. In fact this *is* how I name books on my android devices. Why should I sign up for months of work, zillions of problem reports and some amount of abuse to do something I don't care about? No reason I can think of. Complaining about it over and over isn't going to change what I think.

Those of you who want a different file structure are where I was 3 years ago. You want something and no one else seems interested in building it. So build it yourself! At the risk of seeming to speak for Kovid, my experience is that he will integrate things into calibre if a) they work reliably for the millions of calibre users, b) don't increase his support load, c) do something useful, and d) he has confidence that the implementer will hang around long enough to help with the inevitable problems.

If you don't have the skills required to do it yourself then you must wait until someone comes along who does have the skills, acquire the skills, go somewhere else, or build your own solution. Your choice.

I don't have time / I don't want to code that, that's something I do understand.

But the message is more like "we're doing it right, don't even think about changing that".
That's the reason i finally went to make my own program rather than changing calibre. I didn't feel like these changes would be welcome, even if i bothered coding it.

[kacir]

Originally Posted by EowynCarter

But the message is more like "we're doing it right, don't even think about changing that".
That's the reason i finally went to make my own program rather than changing calibre. I didn't feel like these changes would be welcome, even if i bothered coding it.

Well, the developers *are* doing it right - from their point of view. The program works exactly as desired for the developers and for large numbers of users. So they have no incentive to sink in thousands man/hours to the project.
If I got chance to vote, as an user, I would vote for those thousands of man/hours to be spent adding other features.

I did have to change my way of handling books when I started to use Calibre for more than just a conversion machine and the way of how to access my PRS-500 from Linux. But the benefits that "the Calibre way" of handling things brought vastly outweigh the disadvantages.

I even maintained my own private fork of Calibre when I changed a small number of [mostly cosmetic] little things. Then I found a better way of handling those use-cases using standard built-in features, some plugins and an external program (autokey on Linux - similar to autohotkey on Windows)

[twowheels]

Originally Posted by kacir

(autokey on Linux - similar to autohotkey on Windows)

If only it were half of autohotkey... :-(

I have a 3056 line autohotkey script that makes it so easy to move my customizations from computer to computer -- something I've never been able to match the power or ease of on Linux. Even though I've been a huge fan of Linux and have used it full time on my own personal computers since 1994, running Slackware with twm on a 386sx.

Sure, I can do everything in Linux that autohotkey can do, but to be able to do it all in one place, in one config file, and compile it to an exe that I can portably carry from computer to computer... amazing... in Linux I have to apply patches to numerous config files, the format and location of which changes from version to version... I just gave up on trying to hyper customize it. Luckily it's better out of the box.

[jgaiser]

Originally Posted by chaley

Several times in the past, when I have joined discussions like this one the result has been personal insult. I have been called arrogant, moron, a^%ole, and worse. We will see what happens this time.

All well said. As has been pointed out a number of times -- The code is open source. If you don't like something and there is nobody to fix it, grab the code and fix it yourself. Not a code monkey? Then I guess you're going to have to live with what's available or go find something else.

For me, calibre does everything *I* expect it to.

[Kirtai]

Originally Posted by kacir

I did have to change my way of handling books when I started to use Calibre for more than just a conversion machine and the way of how to access my PRS-500 from Linux. But the benefits that "the Calibre way" of handling things brought vastly outweigh the disadvantages.

I found "the Calibre way" sufficiently better that I've been seeking out other applications to handle my other media files in the same way. Why should I go to all the trouble of managing files by hand when my computer can do it for me faster, better and more accurately than I can? This sort of data shuffling is what they're for.

[BetterRed]

Originally Posted by chaley

...I would be just as happy it all the books in my library were named with a unique ID, totally opaque...

I've often wondered why that wasn't done - but I'm pleased that it wasn't - because if I decided to stop using Calibre, say because I wanted to use some obscure OS like RTM or Elixi, then when I migrated my library I could make head and tail of it when it got there

I just scribbled the following in another thread, I think its worth repeating here:-

Originally Posted by BetterRed

Calibre can perform the task of managing & organising your e-book library's metadata, and if you choose it can also manage and organise the storage of your e-books on your computer's disks.

Whereas the music collection organisers I know of only do the metadata part, and you have to figure out and maintain a storage solution.

I suspect the strongly stated objections one sees on this forum to delegating the task of how and where to store e-books to Calibre, stem from a perceived loss of power. No disrespect, but organising a bunch of folders and files on a computer is nothing but boring administrivia, not much different to stacking packets of cereal on supermarket shelves.

My experience is that many (most ?) people end up with multiple copies of their e-books, audio files, photographs etc, scattered all over their computer, and they ain't backups either.

If you can find software to do the administrivia efficiently, then why wouldn't you use it?

However, the genius of Calibre is that you get to choose which bits you want and how you want to use them.

BR

[theducks]

Originally Posted by BetterRed

I've often wondered why that wasn't done - but I'm pleased that it wasn't - because if I decided to stop using Calibre, say because I wanted to use some obscure OS like RTM or Elixi, then when I migrated my library I could make head and tail of it when it got there

I just scribbled the following in another thread, I think its worth repeating here:-



BR

Would it really matter that much as long as Metadata.opf was in the folder with all the book info?

[DoctorOhh]

Originally Posted by BetterRed

I've often wondered why that wasn't done - but I'm pleased that it wasn't

Rumor has it that at some point before Calibre 0.4, when it was still known as libprs500 (pre 2008) the database of books was completely obfuscated and was changed to the open directory format we know today based on user input.

I say rumor because I have no first hand knowledge. I didn't start using calibre until sometime in 2008 around the 0.4.xx time frame.

[BetterRed]

Originally Posted by theducks

Would it really matter that much as long as Metadata.opf was in the folder with all the book info?

That assumes I could read the OPF on the obscure OS I was now using - yes I know its XML, but I'm not sure I could read that on RTM or something like RSX11S

If what DoctorOhh says is true, it would be interesting to know the rationale for changing it - if a book's folder were named by the book number there would be no need to rename folders and files when book titles and authors change - but there are bound to be other considerations.

BR

[kovidgoyal]

Originally Posted by DoctorOhh

Rumor has it that at some point before Calibre 0.4, when it was still known as libprs500 (pre 2008) the database of books was completely obfuscated and was changed to the open directory format we know today based on user input.

Originally, there was no directory structure, both the metadata and the files themselves were stored in the db. This was changed for the following reasons:

1) Performance of incremental backups with large libraries
2) The ability to have read-only access to your ebook files via the file system, if needed.
3) Robustness - database corruption now only leads to loss of metadata and even that can be recovered via the OPF files.
4) Large file limitations. Depending on file system and OS a single file can sometimes be restricted to a few GB in size.

[kacir]

Originally Posted by BetterRed

... - because if I decided to stop using Calibre, say because I wanted to use some obscure OS like RTM or Elixi, then when I migrated my library I could make head and tail of it when it got there

You would simply use "save to disk" function. Which is what I recommend to anybody who wants to have books from Calibre in some special directory structure and/or file naming convention. You can really go wild with the template language, you have great number of functions available there, together with a nice Template Editor GUI that lets you build any file structure and file naming convention you might wish.

This is the way I access the files in my Calibre database. I have set up file saving template just the way I like it and when I need to copy a bunch of books somewhere, I just do not use operating system copy with files, I ask Calibre to save the books for me to a directory. Why should I go hunting inside directories with books and copy files manually when I can use Calibre User Interface to perform various selects based on various metadata and then save books to a selected directory with a keyboard shortcut. It has several interesting (and beneficial) side effects, like refresh of the metadata inside e-book formats, such as mobi or epub.

IMHO, the file structure itself, such as it is - readable by humans, with opf files and other cool features, is only really important for disaster recovery.


To return to the original subject of this thread.
I am the type of person who is never completely satisfied with the software I have got at the moment and I am always looking for something better. I even keep trying out other text editors, despite the fact that I am already using Gvim, that I consider an almost ultimate editor for me. I keep trying beta and even alpha versions of software the I am already using. I keep trying out other Linux distributions and I sometimes compile [a little bit] customized versions of software.
So far I haven't seen any possible Calibre replacement / alternative that would have more than a tiny fraction of features and power that I have come to expect from my book management system.

[EowynCarter]

Originally Posted by kovidgoyal

Originally, there was no directory structure, both the metadata and the files themselves were stored in the db. This was changed for the following reasons:

1) Performance of incremental backups with large libraries
2) The ability to have read-only access to your ebook files via the file system, if needed.
3) Robustness - database corruption now only leads to loss of metadata and even that can be recovered via the OPF files.
4) Large file limitations. Depending on file system and OS a single file can sometimes be restricted to a few GB in size.

Books where stored in database ? Yup, not very convenient.
Tanks for these explanations.

Tanks chaley too. That the kind of things that should go in this topic :
https://www.mobileread.com/forums/sho...d.php?t=119175
Rather than "shut the hell up", or "folders are such an old way to organize your stuff".

[Adoby]

Hypothetically, if calibre didn't have the current internal folder structure, how should it work? I can see a lot of problems connecting metadata to the actual books, and allow changes to the library structure. Or handling deleted, duplicated and added books.