Multiple Libraries v Virtual Libraries

[Inukami]

I have a few questions on multiple libraries v virtual libraries that I would appreciate any input on. I know there are many different opinions and preferences but I’m interested to know why you choose to manage your library a certain way etc…

I use both multiple libraries and virtual libraries, although thinking of really only using multiple libraries. I don't need to search between the different libraries. I'm probably not using virtual libraries to their fullest potential, but I can look at that later.

1. Do you use multiple libraries or/and virtual libraries?


2. How do you use multiple libraries or/and virtual libraries?

Multiple libraries - Do you use them to manage different types of libraries? I.e Books, Comics, Manuals, Journals etc...or for different reasons - To review etc...

Virtual libraries - How do you use virtual libraries? Why? List some examples.

The manual lists the following examples.

Books added to calibre in the last day:: date:>1daysago
Books added to calibre in the last month:: date:>30daysago
Books with a rating of 5 stars:: rating:5
Books with a rating of at least 4 stars:: rating:>=4
Books with no rating:: rating:false
Periodicals downloaded by the Fetch News function in calibre:: tags:=News and author:=calibre
Books with no tags:: tags:false
Books with no covers:: cover:false


3. What are the strengths and weaknesses of using multiple libraries or virtual libraries?

• Do the multiple libraries run faster if there are less books in each library compared to having all the books in the one virtual library?
• How easy is it to change from multiple libraries to one library with virtual libraries and vice versa?
• My library is only fairly small (1500) compared to what I have heard some other Calibre users have (10,000 +). As people are reading more ebooks, it is easier to collect a large quantity of books and keep them, however this means trying to find certain books and managing them can be even harder. For the people with really big libraries, do you prefer to use multiple libraries or a virtual library to manage your books?

Multiple libraries

Strengths

• You prefer to display certain columns for each of your libraries

Weaknesses

• You cannot search through multiple separate libraries simultaneously in calibre.
• You can only view view/upload one library to Calibre Companion (CC)

Virtual libraries

Strengths

• Preferred way of splitting up your library when you want to search through your entire collection
• It will restrict the list of books shown in a book list but it will also restrict the entries shown in the tag browser
• You can view your entire library in Calibre Companion (CC) using virtual libraries.

Sorry...lots of questions...but very keen to hear everyone’s thoughts.

Inukami

[BetterRed]

Inline comments in red

Originally Posted by Inukami

I have a few questions on multiple libraries v virtual libraries that I would appreciate any input on. I know there are many different opinions and preferences but I’m interested to know why you choose to manage your library a certain way etc…

I use both multiple libraries and virtual libraries, although thinking of really only using multiple libraries. I don't need to search between the different libraries. I'm probably not using virtual libraries to their fullest potential, but I can look at that later.

1. Do you use multiple libraries or/and virtual libraries?

Both

2. How do you use multiple libraries or/and virtual libraries?

Multiple libraries - Do you use them to manage different types of libraries? I.e Books, Comics, Manuals, Journals etc...

Yes - Books, Journals, Media, Comics physical libraries - because my custom column requirement for each are quite different. I want to avoid the trap of overloading columns with different usages depending on publication type

or for different reasons - To review etc...

Physical libraries for Intake, Duplicates, Test

Virtual libraries - How do you use virtual libraries? Why? List some examples.

Varies for each library - but primarily administrative and few in number.

The manual lists the following examples.

Books added to calibre in the last day:: date:>1daysago
Books added to calibre in the last month:: date:>30daysago
Books with a rating of 5 stars:: rating:5
Books with a rating of at least 4 stars:: rating:>=4
Books with no rating:: rating:false
Periodicals downloaded by the Fetch News function in calibre:: tags:=News and author:=calibre
Books with no tags:: tags:false
Books with no covers:: cover:false

Nothing like any of the above - mine are atypical, and often transient - i.e. project related


3. What are the strengths and weaknesses of using multiple libraries or virtual libraries?

• Do the multiple libraries run faster if there are less books in each library compared to having all the books in the one virtual library?
  
  No significant performance difference between my Books library of 3,500 books and my Journals library of 150,000 books
  
• How easy is it to change from multiple libraries to one library with virtual libraries and vice versa?
  
  Disparate custom columns would make it hard for me to merge my 4 main libraries into one - besides as I imagine Jane Seymour might have have said - Why would I want to do that
  
  I did split Comics out of my Media library recently. That was easy I just took all the custom columns from Media to Comics with Copy to Library, and then culled the columns I didn't want and added the columns I did want. Which reinforces my notion that the main reason for having different physical libraries is because of the need for different custom columns for different ''publication types'.
  
• My library is only fairly small (1500) compared to what I have heard some other Calibre users have (10,000 +). As people are reading more ebooks, it is easier to collect a large quantity of books and keep them, however this means trying to find certain books and managing them can be even harder. For the people with really big libraries, do you prefer to use multiple libraries or a virtual library to manage your books?
  
  I don't regard Real and Virtual libraries as either/or alternatives

Multiple libraries

Strengths

• You prefer to display certain columns for each of your libraries

Yes, but that's something I do via the View Manager plugin - all of my libraries (except Test) have a set of stable View Manager column layouts and sort orders

Weaknesses

• You cannot search through multiple separate libraries simultaneously in calibre.
  I don't do many searches in calibre - I do far more full text content searches (which usually result in creating/maintaining Reading Lists) than calibre metadata searches
  
• You can only view view/upload one library to Calibre Companion (CC)
  
  Don't use CC
  
  I only keep a few books on a tablet and/or a phone, which I manage myself

Virtual libraries

Strengths

• Preferred way of splitting up your library when you want to search through your entire collection
  Irrelevant to me
• It will restrict the list of books shown in a book list but it will also restrict the entries shown in the tag browser
  True - if I go to a branch bricks and mortar library I can't easily see what's on the shelves of the main library 10km away. There, I would fossick through the library's catalogue - with calibre I would do the same via the CalibreSpy plugin.
  
• You can view your entire library in Calibre Companion (CC) using virtual libraries.
  
  True - but that's irrelevant to me

Sorry...lots of questions...but very keen to hear everyone’s thoughts.

Inukami

[ilovejedd]

1. Do you use multiple libraries or/and virtual libraries?
Both.

2. How do you use multiple libraries or/and virtual libraries?

Multiple libraries - Do you use them to manage different types of libraries? I.e Books, Comics, Manuals, Journals etc...or for different reasons - To review etc...
I use separate libraries for commercial books and fanfiction. Actually, I even use separate Calibre Portable installs because I use different tweaks for the two (author sort is the biggie).

When I'm planning on making significant changes to the library metadata and/or structure, I'll work on a separate full copy that would either replace my current library or get scrapped.

Because of the way Calibre handles conversion, I also have a duplicate library where I keep my tweaked ePubs. Basically, I want to keep the originals but use tweaked ePubs for conversion but Calibre doesn't allow that so separate libraries was the way to go.

Virtual libraries - How do you use virtual libraries? Why? List some examples.
I use virtual libraries to isolate recently downloaded books that require metadata processing and cleanup. I have a VL for books with tag _new_ and the rest (cover check, etc), I just handle via saved searches. On my fanfiction library, I also use VLs to separate by fandoms/crossover.


3. What are the strengths and weaknesses of using multiple libraries or virtual libraries?

• Do the multiple libraries run faster if there are less books in each library compared to having all the books in the one virtual library?
  Not that I've noticed (5K fanfic library, 1K normal books).
• How easy is it to change from multiple libraries to one library with virtual libraries and vice versa?
  Author sort is an issue for me (comma on commercial books, copy on fanfics) so this one's a no go.
• My library is only fairly small (1500) compared to what I have heard some other Calibre users have (10,000 +). As people are reading more ebooks, it is easier to collect a large quantity of books and keep them, however this means trying to find certain books and managing them can be even harder. For the people with really big libraries, do you prefer to use multiple libraries or a virtual library to manage your books?
  I don't have a humongous collection but I prefer to use multiple libraries, anyway.

Multiple libraries

Strengths

• You prefer to display certain columns for each of your libraries

Weaknesses

• You cannot search through multiple separate libraries simultaneously in calibre.
  This is something I never do so not really a con for me.
• You can only view view/upload one library to Calibre Companion (CC)
  I don't use CC. I use either the content server or calibre-php and switching libraries is as simple as going to a different webpage.

Virtual libraries

Strengths

• Preferred way of splitting up your library when you want to search through your entire collection
  As mentioned, not something I do so not really a pro for me. Indeed, I would just need to create more VLs to facilitate filtering.
• It will restrict the list of books shown in a book list but it will also restrict the entries shown in the tag browser
  True, but it doesn't restrict entries for "Manage Tags" afaik.
• You can view your entire library in Calibre Companion (CC) using virtual libraries.
  As mentioned, I don't use CC.

[Sirtel]

I don't use virtual libraries. My books (more than 10 000) are categorized by various tags and custom column entries. To restrict my current view to a select group I use saved searches, of which I have several.

I do have a couple of different libraries. My main one and the one I use for sending books to my Kindles (all the books in that library are in azw3 format). The books in the Send-to-Kindle library are in the main library too, just not always in azw3. Strictly speaking I don't need a separate library for sending books to device, of course, but I've found it's easier to manage books currently on my Kindles this way.

I use Calibre Companion, but not very often, because I very rarely read books on my phone.

[theducks]

Originally Posted by Inukami

I have a few questions on multiple libraries v virtual libraries that I would appreciate any input on. I know there are many different opinions and preferences but I’m interested to know why you choose to manage your library a certain way etc…

I use both multiple libraries and virtual libraries, although thinking of really only using multiple libraries. I don't need to search between the different libraries. I'm probably not using virtual libraries to their fullest potential, but I can look at that later.

1. Do you use multiple libraries or/and virtual libraries?

Both
2. How do you use multiple libraries or/and virtual libraries?
Different columns, Different Audience (grand kid)

Multiple libraries - Do you use them to manage different types of libraries? I.e Books, Comics, Manuals, Journals etc...or for different reasons - To review etc...

Virtual libraries - How do you use virtual libraries? Why? List some examples.

The manual lists the following examples.

Books added to calibre in the last day:: date:>1daysago
Books added to calibre in the last month:: date:>30daysago
Books with a rating of 5 stars:: rating:5
Books with a rating of at least 4 stars:: rating:>=4
Books with no rating:: rating:false
Periodicals downloaded by the Fetch News function in calibre:: tags:=News and author:=calibre
Books with no tags:: tags:false
Books with no covers:: cover:false
The main use is status: Read, To Read: To Edit(touch up to my prefs): Wishlist (Empty books)

3. What are the strengths and weaknesses of using multiple libraries or virtual libraries?
Strength of multiple: Can be on different drives, can be moved/backed up independently

Weakness of VL:Easy to bloat (affects performance) with columns that don't apply to the majority of contents
Inukami

Answers inline

[BetterRed]

Originally Posted by ilovejedd

Author sort is an issue for me (comma on commercial books, copy on fanfics) so this one's a no go.

FYI - DaltonST's JobSpy plugin has a couple of features that may help

Per-Library Tweak for Calibre Global Tweak for Author Sort Method
Per-Library Tweak for Adding Books for 'read file metadata' from either file contents or file name
Per-Library Tweak for Book Automatic Adding Directory

I toyed with multiple instances of Portable for a while, but I found the lack of ability to quickly switch between libraries and the need to maintain the same settings/customisations plugins etc in the different configurations far too much of a nuisance. The per library tweaks in JobSpy solved some of that.

BR

[ekbell]

I have a separate library for fanfiction but mostly I use virtual libraries to organize my books. I have virtual libraries for my paper books, ebooks, children's books, adult books, fiction, nonfiction, public domain/creative commons, .... etc. etc. as well as the not infrequent temporary virtual library to fit a short term need.

I prefer virtual libraries because my categories overlap and I do not want to have to search more then one library to find all books fitting a particular category.

I don't use a lot of custom columns and those I use are applied to the majority of my books regardless of type

[ilovejedd]

Originally Posted by BetterRed

FYI - DaltonST's JobSpy plugin has a couple of features that may help

Per-Library Tweak for Calibre Global Tweak for Author Sort Method
Per-Library Tweak for Adding Books for 'read file metadata' from either file contents or file name
Per-Library Tweak for Book Automatic Adding Directory

I toyed with multiple instances of Portable for a while, but I found the lack of ability to quickly switch between libraries and the need to maintain the same settings/customisations plugins etc in the different configurations far too much of a nuisance. The per library tweaks in JobSpy solved some of that.

BR

I do know about Job Spy.

To be honest, I haven't run the Calibre Portable install for my normal book library in years. I only run Calibre GUI when I need to clean up metadata and do conversions for new acquisitions and I haven't purchased new "normal" books since, hmm, around 2014? Anyway, the library is accessible to my reading devices at any given time via calibre-php-server or Dropbox.

I'll probably add the libraries into one Calibre Portable install if/when the need arises. Right now, I'm more concerned with organizing and normalizing tags for my AO3 library (which I've just let accumulate on Marvin and/or Dropbox since 2014).

That said, while Job Spy's per library tweaks eliminates the need for separate Calibre Portable instances, it still requires the use of separate libraries. Can't use it for a single library that just make use of virtual libraries.

[Inukami]

Thanks for everyone's detailed responses.

After reading the various response, I decided to separate my libraries which is working better for me know. Most of you seemed to imply that separate libraries work better for different publication types (i.e Books v Comics), and this seems to apply to my case as well.


Thanks

Inukami

[harriska2]

I tried out Virtual Libraries and that seems to help with my problem of having my husband wade through my journal articles and curriculum to find stuff that he would like. I recently cleaned up my library with tags so it is easy to create the virtual library and use it via the Calibre web server.

[un_pogaz]

1. Do you use multiple libraries or/and virtual libraries? 2. How do you use multiple libraries or/and virtual libraries?

Multiple libraries
2 physical libraries with several virtual libraries inside : One for eBooks (is my main and the reference for others), the second for Comics and Mangas. A third 3 for non-French eBooks (because I'm), but rarely used (storage/archive).
Sometimes, I create a 4th physical libraries, for books coming from multiple sources and on which I will have to do a lot of sorting work, editing metadata, or even completely rework the ePubs. This physical libraries is temporary and the books will be added in the main libraries once the work is finished. But it takes a lot of books for such a method to be used (like Perry Rhodan's complete serie... 373 books in France)

Virtual libraries
In my main library, the books are structured around 2 axes

• Authors with several books
• Authors with only one book
  and
• Books that I am interested in and would like to read
• Books that I am not interested in and that I keep for archival purposes

This makes 4 main category/virtual libraries, in addition :
3 utilities, metadata not checked, not in one of the 4 main category and books to be rework.
And with a little trick, I created 2 virtual libraries that when I plug in my e-redear, Calibre check the books on it and list me at a glance the books added or deleted them.

3. What are the strengths and weaknesses of using multiple libraries or virtual libraries?

• It really takes a lot of books in a physical library to see a slowdown when loading it. A lot, a lot of pounds for a barely noticeable slowdown if you don't pay attention.
• VL to PL (Physical Libraries) really easy. But if you had books belonging to several VLs, be careful to duplicate the books.
• For PL to VL, it depends on the care with which you created and maintained the PL. Theoretically the biggest risk is to lose Custom Collums, but Calibre detects this and asks you if you wanted to add Custom Collums not present. No, the real problem will be the duplicates, fortunately the "Find Duplicate" plugin is a precious friend in this case.
• For a while, I had 2 physical libraries, but the bigger they get, the more glaring the defects of such a system became. Merging was difficult, but now a main physical libraries with several virtual inside is the best possible solution .

Multiple physical libraries
Strengths

• A very clear separation between several very different categories of books.

Weaknesses

• Book duplicated between several libraries. If you changed in one, you would have to copy in the other.
• Uncommon metadata. If you changed it in one, you had to do the same in the others.
• Switching from one physical library to another is slower than between 2 virtual libraries.
• Metadata management is becoming a living hell. Really.
• (the worst) You cannot search in several separate physical libraries simultaneously.

Virtual libraries
Strengths

• Everything you were told.
• Instant switching between 2 virtual libraries.
• Metadata standardization greatly facilitated.
• Once you understood how to do it, you could create incredibly complex virtual libraries.

Weaknesses

• The way to create Virtual libraries can be difficult to understand at first time

You marked as an advantage for Multiple physical libraries, the ability to choose and display several different columns depending on the libraries. Maybe. But in this case I present to you View Manager, a plugin that allows you to save, sort and modify the order of the displayed columns. And even to sort your libraries if necessary.

EDIT : S***, thank for the necropost, harriska2

[peaceridge]

I have 2 physical libraries set up much like you. One for Manga and comics [I call it archived&comics] and a Current one. I also keep my archival books on the archived&comics library. In my Current library, I tag them Archived, move them to the archived library, and then remove specific formats from the books on the Current library, but leave the book title in the Current library. If I need it, I get it from the archived. This allows me to keep a list of all my books on the Current library, but it is smaller data-wise and more quickly backed up.

[Fiat_Lux]

Originally Posted by Inukami

• Do the multiple libraries run faster if there are less books in each library compared to having all the books in the one virtual library?

When I moved all of my books that were 500 MB+ in size, to their own physical library, Calibre was noticeably faster, except when I was working with that specific library.

One of these days, I'll hand-edit the ePubs, and gets them down to something more reasonable. (Why on earth is an ePub 2 GB in size? Or a PDF 5 GB in size? I can understand CBZ and CBRs coming in at that size, because they are essentially photo-galleries in an archive file format.)

Keep all of you ebooks under 100 MB in size, and you won't notice a speed difference.

I'm almost at the point of splitting a couple of books of poetry into the individual poems, because the number of items doesn't appear to affect the speed of Calibre, and hunting for specific poems within books can be aggravating.

[theducks]

Calibres speed is mostly governed by the size of Metadata.db and RAM
Loads of custom fields, long winded comments will make the DB bigger.
The actual size of the book only matters when converting, editing, viewing and embedding metadata.

Do run the first phase of Library Maintenance if you have done some major pruning / renaming. Those entries are only inactive until the house is cleaned