Going beyond ebooks - looking for inspirations

[Joanna]

It's been almost four years since I added the first book to Calibre. Quickly, I got addicted to this great application.

When you get used to this kind of managing possibilities, sooner or later you will miss them while managing other kind of data.

I know people use Calibre to manage other kinds of data - if you're one of them, I'd be glad if you could share some experience and briefly describe the structure of your library.

I realize it's a very general question but I don't have any specific problems (yet). I do know some popular solutions (e.g. zipping files or adding parts of an audiobook as separate files in a series) and have experimented with some ideas over the years - but I don't know which solutions are actually convenient and optimal in the long run.

Therefore, I am looking for inspirations before I'll try to reinvent the wheel

I'm particularly interested in managing:
- press (pdfs mostly),
- audiobooks,
- language materials (a mix of ebooks, audio materials, courses, software)
- music,
- films and TV series,
- software,
- Android apps (as .apk).

Alternatively, I'm also interested in the recommendations of managing software for such data (on Windows) - which software would be optimal for a Calibre addict?

[phossler]

I had always thought about using Calibre to organize things better also, but it appears to me that it's oriented towards a 'one file per format' structure.

I had tried to add a 2 MP3 audio book and it created two separate 'books'. Adding the first MP3 and then trying to add the second into the same 'book' would replace the first MP3. I would have preferred to have all MP3s in one 'book' without zipping them.

But I'm happy since it does a great job for my ebooks in epub and azw3 (plus a few in other formats), so I don't worry about it.

[theducks]

I think Calibre did it right with 1 file per format.

A Calibre Record is A BOOK, the same BOOK, regardless of file format.

If you have different EDITIONS or Languages, then they are DIFFERENT BOOKS and they need their own RECORD ( However the SERIES would have the SAME value, thus establishing a relationship to other editions)

2 Audio Books, assumes they were by different 'talent'. The metadata should match the content. (I guess you could make a case for a really reduced bit rate version of the same performance)

[Joanna]

Yeah, one file per format is in my opinion definitely the biggest obstacle if one is thinking about using Calibre to organize other things.

.zip is an option and I use it here and there if needed but a whole library with zipped files does not seem convenient.

Adding files as separate records and grouping them as a series (or using another column for that) is another option - but it requires a complete change of habits and makes it more difficult to simply browse the library (sooo many records). On the other hand, one could use e.g. some custom search or a virtual library to display only one file per item while simply browsing the collection.

Another option could be to store the data outside of the Calibre library and link to them in book details - but this would be quite difficult to maintain as one would have to manually change the link if any changes in the data folder were introduced. It also makes it more difficult to copy a library, and from what I've seen, Calibre uses full file path for links, not a relative one (or is there any option for a relative one). It would also be nice to have simply links to folders, not files (that's not an option either, or is it?).

That's why I've created this thread - hopefully, there are people who've come up with solutions they're really satisfied with, and they could share their insights . Or there are users who've found other software solutions for other kinds of data - maybe not as flexible and customizable as Calibre, but good enough.

Waiting for inspirations

---

EDIT: Just to clarify, I agree with you, theducks, that 1 file per format is good for e-books. It's just not optimal for storing other kinds of data (which is perfectly fine, it's not what Calibre has been created for, after all). This is why I'm looking for powerful workarounds or alternatives - I'm basically spoiled and that's, by all means, Calibre's fault ;P.

[kovidgoyal]

If I were you I'd just use zip files and use the open with function in calibre to open them with windows explorer, which will open the zip file as a folder, from where you can easily dispose of it as you would like. This of course assumes you are on windows.

[phossler]

Originally Posted by theducks

I think Calibre did it right with 1 file per format.

Audio Books, assumes they were by different 'talent'. The metadata should match the content. (I guess you could make a case for a really reduced bit rate version of the same performance)

1. Yes. Me also

2. ABs are typically in multiple files per single book. So zipping them like Kovid says will work, but IMHO is more trouble than it's worth. There are other tools oriented towards AB (even iTunes )

Since Calibre works so well for 'books' I just use it the way it's intended and don't try to force to go to where it's not designed to.

[Joanna]

Originally Posted by phossler

There are other tools oriented towards AB (even iTunes )

Any recommendations? (iTunes is a no-go for me).

Since Calibre works so well for 'books' I just use it the way it's intended and don't try to force to go to where it's not designed to.

Not exactly the topic of this thread

Kovid, thanks. It does look like a viable option. I'm already zipping files for Calibre occasionally. The zip solution has quite a lot of advantages (e.g. the ability to handle the audiobooks just as the e-books in the database) but the disadvantages are significant, too (more difficult to send to the device, and in the case of my language learning materials some software would have to be stored separately). Still, definitely something to consider.

And yes, I'm on Windows. Gotta add this information to the first post, asking for software recommendations without giving any OS specifics is sooo amateur and Windows-centric .

[chaley]

@Joanna: you might find this thread by BetterRed interesting. He describes how he uses the tool Evernote to link extra files and auxiliary book information to calibre records.

[fidvo]

Here are some of the ways I use (abuse) Calibre, and some other ways I've thought of but decided against.

Image Catalog:

I find that most images on my hard drive are already part of a series. For instance, I might have a series of pictures I took on vacation, so the entire vacation set would count as one series.

It's easy, then, to turn a series of images into a single .cbz file (just zip the images and rename the extension), which I then import into Calibre. Pretty straightforward. You could use the Published column to store the date the pictures were taken (or the date the first photo was taken if you have picture sets that span more than one day), or create your own custom date column for this.

I still haven't found a satisfactory way to use Calibre to store individual images that aren't easily categorized. You could technically go with 1 record per picture, but then you double the hard drive space because you'll use the same picture for the "book" record and the cover. You could just not store the cover, but since the picture itself is usually the single most important piece of data about it, you want that visible as you're browsing them.

Movies:

For movies, the Author usually isn't all that important, so I repurpose the field with a variation of the title so that they're easy to look up if for some reason you have to peek into the library folder. (I know. Don't do this)

For TV shows, I generally put the season and episode number in the title field and use a custom field for the episode title. I number the series with the formula: 1000*Season + Episode. So, for instance, Episode 5 of Season 3 would have a series index of 3005. This also allows me to insert special episodes that aren't a part of the regular season for the rare show that has them. I start counting these at (Season * 1000) + 100, so Special Episode 1 after Season 3 would have series index 3101. It sounds a lot more complicated than it really is.

I also have a plain text Notes record for each show for information about it.

Music:

Technically you could use Calibre as an iTunes replacement. Depending on the type of music, you might want to put the performer (modern music) or composer (classical music) in the Author field. Series would be Album, but you might want to create multiple fields of this type if you have "Best of" collections.

I decided against storing my music in Calibre because of the difficulty of transferring the files to my mp3 player, although I'm exploring some new possibilities of using Calibre's save-to-file template to save the files directly to a micro-SD card that my mp3 player will recognize.

Audio Lectures:

I have the same issue with using Calibre for Audio Lectures as I do with Music, so I don't use Calibre for this. I started to do it once, but changed my mind.

Lectures usually come in series, so I just used the lecturer as the author, and the lecture series as the series. I don't remember whether I put the lecture title in the title field or used a numbering system like for TV shows and used a custom column for the lecture title. You could make an argument for either system.

For the lectures I listen to, the only supplemental material is the occasional PDF file with lecture notes. I haven't come across any lectures that have more than one supplemental file per lecture, so I can't help you with cases where you might have several PDF's or software to run with it.

Notes:

Yes, Calibre makes a decent note taking system. I use a custom "Path" column for categorizing my notes into a hierarchical system, but I also use the built-in tags column to look up notes on different subjects. To add a new note, you could just use the Add New feature and have it generate a blank file (I mostly take notes in plain text files, so this works fine), but I work from a template file, which I can duplicate from that same Add New screen to create the note.

Recipes:

Calibre makes a surprisingly good recipe catalog. One record per recipe. Tag them however you want. I use the same type of system as I do with notes, using a template for creating new recipe records. To compile them all into my own custom recipe book, I use save-to-file then run a custom script I wrote to merge the files and convert it to HTML, which I can then import back into Calibre as a single book file. I plan to eventually turn that script into a Calibre plugin so that I can skip the whole save-to-file step, although it's so specialized to my workflow that I doubt I'll upload it here once I write it unless I can figure out how to generalize it so it doesn't make so many assuptions.

Software:

I don't use Calibre for this, but it seems to me like it would make a decent program launcher if you have a lot of programs to run. There are two ways to handle the programs themselves. You could, in theory, import the .exe files into Calibre then transfer the supporting files into the library manually. Everything about this screams "Don't do it!" because you're really not supposed to go poking around in the library folder. Plus, Calibre will rename the .exe file when you import it, which in some cases might cause the software not to run.

Another way would be to create shortcuts to the programs and store those in Calibre instead. Aside from portability issues, this is a better method in all regards.

Final Notes:

I use Calibre Portable so I can keep everything on my external hard drive. For extreme portability, I created a "Util" folder under my main Calibre Portable folder, which stores portable versions of programs such as VLC, Notepad++, and other viewers/editors. Then I use the Open With plugin with relative links to these programs. (I don't think the built-in Open With feature can handle relative links, which is why I still use the plugin) That way, if I'm ever on a computer without these file types registered or registered to a program I don't like, I can always use Open With to send them to the "embedded" version of the software.

[dickloraine]

One possibility for audio books is to convert them into a single file. There even is a relatively wide spread format from apple, especially for eBooks (supporting for example chapters). Foobar for example is able to convert a bunch of mp3s into it. Many (most) audio book players support it (audio book player, mort on android, apple natively, foobar mediamonkey etc. under Windows).

[Joanna]

@chaley: thanks for the link, it does look interesting! And I am an Evernote user, so that's definitely something to explore.

@fidvo: thanks a lot for your informative and inspiring post!

Movies

I find the idea of a movies library quite appealing; it's good to know it works well.

In my case, TV shows are less of a problem, as I can manage them quite well on my hard drive, using just normal folder structure (folders for shows, subfolders for seasons). I usually remember the shows quite well, so the benefits from short descriptions available in Calibre would not be that huge. But my movie folders are a huge mess. They cry for some meaningful structure, genre tags etc., and movie descriptions would come in handy, too.

One problem I would have (I've just realized ) is subtitles. I use separate subtitles quite frequently (for ripped DVDs etc.), and not rarely do I have more than one subtitle file per movie (as I may have different language versions for watching movies with different people). Those files have frequently the same format. This would be problematic as the movie players I use won't show subtitles from inside of a zipped file. Gotta think about it.

Music

Yeah, the difficulty of transferring the files to the mp3 player is the biggest problem I've identified. Could be nice for listening to the music on my computer but the truth is I listen to the music on my phone most of the times, so it seems that using Calibre for that would indeed be counterproductive.

Any ideas for good music management software? I've experimented with a couple of applications several years ago but haven't stuck with any. At the moment, I just have one music folder with separate subfolders for albums. As I don't have a lot of music, it's not that terribly chaotic as it sounds . Plus, I use Mp3Tag for the right tags and generating file names. Still, it doesn't seem to be optimal.

Audio lectures

It sounds like a really nice idea. If I had this kind of data, I would definitely give it a try. I'm thinking about an audiobook Calibre library (and I'm far from being the only one, considering there is even a plugin for that ) but it doesn't seem as intuitive as in the case of lectures, which usually are not split into multiple files.

On the other hand, this would work great for podcasts (one file = one complete 'book' in a series). It's a pity I don't store podcasts on my computer (with few exceptions), just download them directly to my phone. (By the way, if anyone is looking for a good Android app for podcasts, I can very much recommend Podcast Addict).

Notes

Wow, that's a truly creative way of using Calibre! It would not work for me (although I also take notes in plain text files mostly) but definitely interesting.
Same goes for Image Catalog.

Recipes

Interesting, I wouldn't never have thought about it, and as I still don't have a good recipe managament system I might give it a try.
I like the idea of merging the files and converting. For me, it would be even more useful if the resulting file could be epub or mobi - this would be a way to create a custom-made cook(e-)book and quickly send it to the e-book reader (and take it to the kitchen ). If you ever manage to make a plugin and generalize it, I'd be happy to try it.

Software

I don't have that many programs to run, and using Calibre as a program launcher would probably be more than crazy . For storing software, I was thinking more in line of installation files, and this should be doable. But I'm not sure whether it makes any sense (gotta figure it out). Maybe more for the Android app files (which I have plenty of - I usually save apks of interesting apps I've tried, as you never know whether they're gonna be available to download from Google Play when you're gonna need them).

Language learning materials

That's what gives me a serious headache. I'm dreaming about my Calibre language library (just realized it today ) but I'm not sure how to organize it.

The thing is: what I consider one item might be very diverse. Sometimes it's just a pdf with a book, sometimes it's a pdf and audio files. Or just audiofiles - because it's an audio-only course, or because I have a physical book (which I would like to note in Calibre). Sometimes I have multiple pdfs, or a pdf + audio + CD-ROM (as a CD image). I may also have just a CD image... or, in the worst-case scenario , I have a portable application, which would be dangerous for Calibre to handle, or a downloaded website.

All in all, a great chaos and a huge challenge...

By the way, I love the idea of a 'Util' folder in the Calibre Portable folder. I also use Calibre Portable and keep everything on my external hard drive

[Joanna]

@dickloraine

Thanks for the information, I haven't heard of it. That's could be "the" solution. I just have to find out whether my Android audiobook player (Ambling BookPlayer) supports this feature. Probably not (it's an old app which seems to be discontinued), but on the other hand, I could probably find some good replacement.

[BetterRed]

Originally Posted by chaley

@Joanna: you might find this thread by BetterRed interesting. He describes how he uses the tool Evernote to link extra files and auxiliary book information to calibre records.

I'm thinking of replacing Evernote with OneNote. In May 2015 Evernote broke their implementation of Tables, which I use a lot in this context, as of a day or two ago they still hadn't fixed it in the latest beta!!

I also do some things fidvo does

Originally Posted by fidvo

Image Catalog:

'cept I call it the Photo Album library

I don't bother with the extension rename, if its a solid archive of images (ie only images and no subfolders) calibre will 'convert' the .zip to cbz. It will also use the first image as a cover.

I put the Photographer name in Author and make up a Title (e.g. Fred & Freda's Wedding) - I have a a number of custom columns - Photo Count, Photo Height & Width, Location, People etc.

I create an album a month for the ad hoc photos

Movies:

Similar for movies - I put Director in Author, I also have a Cast column for significant roles, and some other custom columns like length.

I don't have any regular TV shows. But I do have lots of talking heads current affairs shows that I store in a separate library (Media) along with transcripts. In the related Evernote link I put urls to related material from the MSM and Soc Med commentariat

Music:

No - thought about it but I'm happy with Quodlibet.

Audio Lectures:

Similar - This is the sort of thing where I use an Evernote link which in turn holds, or references via file links, the audio/video lectures and notes.

Notes:

No - Evernote and Onenote are better for this. IMO calibre has too much of a 'fixed structure'. I use EN for ad-hoc notes and clips etc, and Onenote for structured (e.g. lecture, storyboard) notes. EN notes will often find there way into a Onenote Notebook

Recipes:

No - I usually make it up from what I have in fridge/cupboard, or use existing paper cookbooks,

Software:

No - I might drop windows shortcut files (lnk) into an Ever/One note

One other thing I do is to put links to Chrome shortcuts that access calibre servers in Evernote, so an Author Note will have one or more of something like this

%UserProfile%\Local\Google\Chrome\Application\chrome.exe http://This-PC:8082/browse/matches/authors/187

Where 8082 is the port that I've associated with a library, and 187 is the Author rowid in that library. Thus I can get to my 'books' from Evernote.

Thinking of dedicating Edge for the above, as I sometimes use Chrome for big browser-only downloads. I'll probably do it when Kovid releases the new server program.

BR

[BetterRed]

And some media players (such as FLV - I think) can play a zip of mpN's.

Good for audio and video published in multiple sections - avoids the tedium of joining them with something like Audacity or Handbrake.

Maybe there are Android apps that can do likewise.

BR

[fidvo]

Originally Posted by Joanna

One problem I would have (I've just realized ) is subtitles. I use separate subtitles quite frequently (for ripped DVDs etc.), and not rarely do I have more than one subtitle file per movie (as I may have different language versions for watching movies with different people). Those files have frequently the same format. This would be problematic as the movie players I use won't show subtitles from inside of a zipped file. Gotta think about it.

Depending on the file format of your movies, there are programs out there that can embed subtitles into the movie file. If you use soft subs, you can even do it without re-encoding the video stream, so you lose no picture quality. It also means you can choose between different subtitles for the same movie, or no subtitles at all.

I can't really say more because I've never done it myself, but if you're willing to do a little research and work, this solution might work for you.

Originally Posted by Joanna

I like the idea of merging the files and converting. For me, it would be even more useful if the resulting file could be epub or mobi - this would be a way to create a custom-made cook(e-)book and quickly send it to the e-book reader (and take it to the kitchen ). If you ever manage to make a plugin and generalize it, I'd be happy to try it.

Once it's HTML, Calibre should be able to convert it to epub or mobi easily. If I wrote a plugin, I might even be able to have the plugin convert it automatically.

My biggest issue with creating a plugin (aside from the fact that I've never written one before and it's been years since I've done any programming in Python) is that my script makes assuptions based upon the file names, directory structure, and contents of the file, and trying to generalize those might be a pain.

The directory structure wouldn't be too difficult to duplicate in Calibre; I could just require using the Title field for the recipe name and the Series field for the major section in the recipe book. (I break my recipes down by categories such as Main Dish, Dessert, Salad, etc) The problem is the layout of the text files containing the recipes.

Admittedly, it's pretty straightforward and if you converted most recipes to plain text they would end up in the proper format already, because my script simply identifies keywords such as "Ingredients:" and "Directions:" to insert formatting and make the HTML prettier. But my script only works on plain text, it can't handle auxiliary files like pictures, and it assumes the text files are laid out nice and neatly like that. Trying to make a plugin work on anything beyond that very basic system could be a nightmare.

That's why if I ever do write the plugin, it's not likely I'll upload it here. Inevitably people would (rightly) complain because it doesn't handle the way they manage their recipes.