Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 09-25-2018, 03:17 PM   #1
RobinLee
Junior Member
RobinLee began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Oct 2017
Device: Kindle Paperwhite 3
Virtual libraries

Hi

Will virtual libraries showing only a few columns make Calibre run faster?

Just over 5000 books
61 columns
45 plugins
60 icon rules

My computer has a SSHD with 16MG RAM but even just copying a book from anther library takes a lot longer than it used to.

I really don't want to split up the library physically, but if I were to use virtual libraries only showing specific columns would it run faster?
RobinLee is offline   Reply With Quote
Old 09-25-2018, 03:36 PM   #2
Tarana
Wizard
Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.Tarana ought to be getting tired of karma fortunes by now.
 
Tarana's Avatar
 
Posts: 1,868
Karma: 20068434
Join Date: Sep 2012
Location: Minneapolis
Device: Voyage, K3, PPW 2 3, HDX, KBasic 4, 5, 7 & 8, Nook ST, Glo, HD+, Echos
Wow, that's some library! I can't help you, alas. I only use the virtual library based on TAGS. So, one is for Scifi, one is for thriller/crime, that sort of thing. And yes, that runs better.
Tarana is offline   Reply With Quote
Advert
Old 09-25-2018, 03:42 PM   #3
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 22,112
Karma: 22087764
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: K4NT, Galaxy Tab A
Quote:
Originally Posted by RobinLee View Post
Hi

Will virtual libraries showing only a few columns make Calibre run faster?

Just over 5000 books
61 columns
45 plugins
60 icon rules

My computer has a SSHD with 16MG RAM but even just copying a book from anther library takes a lot longer than it used to.

I really don't want to split up the library physically, but if I were to use virtual libraries only showing specific columns would it run faster?
AFAIK VL's are just logical views that just make interaction faster.
AND there is no difference in the columns.There is ONLY ONE DB (Use the VIEW MANAGER plugin to adjust yhe colums to show )

Have you run Library Maintenance (at least, the first phase)? That compacts the DB (tosses the now dead records). ( make note of the metadata DB size before cleaning)

Lots of RAM is best as the DB is in RAM (and you have that) . , exclude the Library folder from active A/V (all books have been checked when first downloaded. Assumes the download folder has not been foolishly excluded)
theducks is offline   Reply With Quote
Old 09-25-2018, 03:55 PM   #4
Rellwood
Library Breeder
Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.
 
Rellwood's Avatar
 
Posts: 356
Karma: 1891742
Join Date: Apr 2015
Location: Fullerton, California
Device: Kobo Aura HD (2), PW2 (1), PW3 (4)
Quote:
Originally Posted by theducks View Post

Have you run Library Maintenance (at least, the first phase)? That compacts the DB (tosses the now dead records). ( make note of the metadata DB size before cleaning)

Lots of RAM is best as the DB is in RAM (and you have that) . , exclude the Library folder from active A/V (all books have been checked when first downloaded. Assumes the download folder has not been foolishly excluded)
I usually run the check library once a week, right now its 59MB and I just performed a check which made it go to 55.

What do you mean by excluding the folder from the active A/V?
Rellwood is offline   Reply With Quote
Old 09-25-2018, 03:57 PM   #5
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 22,112
Karma: 22087764
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: K4NT, Galaxy Tab A

All those rules and columns are taking a big hit.

The question is WHY did you do it the way you did and is there a better (more process efficient) way?

Will a saved search get the job done when you need THAT particular item?

Would putting multiple column values in a Tag like column get you there (every column bloats the DB structure)

and you really need that many Icon (or coloring) rules?
We have many users with 50K books. I am approaching 15K.
It takes 11 seconds to switch back to the Main Library after working in my Intake Library. I have 5 color rules, NO column rules, and 10 custom columns.
theducks is offline   Reply With Quote
Advert
Old 09-25-2018, 04:00 PM   #6
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 22,112
Karma: 22087764
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: K4NT, Galaxy Tab A
Quote:
Originally Posted by Rellwood View Post
I usually run the check library once a week, right now its 59MB and I just performed a check which made it go to 55.

What do you mean by excluding the folder from the active A/V?
Just that. Open you Anti virus preferences and exclude it from the real time scans (Leave the manual scans). That way, when you add, move, edit books or metadata, they are not scanned (and eating CPU cycles and time)
theducks is offline   Reply With Quote
Old 09-25-2018, 04:01 PM   #7
Rellwood
Library Breeder
Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.
 
Rellwood's Avatar
 
Posts: 356
Karma: 1891742
Join Date: Apr 2015
Location: Fullerton, California
Device: Kobo Aura HD (2), PW2 (1), PW3 (4)
@theducks,

I suppose I could weed it down. Its my own fault with the plugins. Its funny that you talk about not needing all that data because sometimes I come across an old library with much less data and end up wishing I could stay in it because it is so clean and easy to manage.
Rellwood is offline   Reply With Quote
Old 09-25-2018, 04:06 PM   #8
Rellwood
Library Breeder
Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.Rellwood ought to be getting tired of karma fortunes by now.
 
Rellwood's Avatar
 
Posts: 356
Karma: 1891742
Join Date: Apr 2015
Location: Fullerton, California
Device: Kobo Aura HD (2), PW2 (1), PW3 (4)
Quote:
Originally Posted by theducks View Post
Just that. Open you Anti virus preferences and exclude it from the real time scans (Leave the manual scans). That way, when you add, move, edit books or metadata, they are not scanned (and eating CPU cycles and time)
Oh, Antivirus...no its an ignored folder.
Rellwood is offline   Reply With Quote
Old 09-25-2018, 05:32 PM   #9
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,067
Karma: 10563148
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by RobinLee View Post
Hi

Will virtual libraries showing only a few columns make Calibre run faster?

Just over 5000 books
61 columns
45 plugins
60 icon rules

My computer has a SSHD with 16MG RAM but even just copying a book from anther library takes a lot longer than it used to.

I really don't want to split up the library physically, but if I were to use virtual libraries only showing specific columns would it run faster?
5,000 books is not many books, I have a library with 150,000

60+ columns seems excessive, especially if the library contains actual ebooks rather than 'foreign objects' such as music or movies. Composed columns (columns built from other columns) can have a -ve effect on performance

45 plugins seems excessive, although they should only effect start up times, but I would get rid of any I don't actually use.

60 icon rules also seems excessive, maybe you could reduce the number of rules by using Program Mode templates.

As has already been mentioned, use the View Manager (VM) plugin to limit what columns are shown in the Book List. VM will also define the sort order.

Regarding AntiVirus (AV): IMO if you need to exclude your calibre library from the AV you use, you need to get another AV. I run two, Defender and MalwareBytes, I exclude 1 DLL (a Word Addin) in MWB, and nothing in Defender. As far as I'm concerned tey have no discernible effect of calibre performance.

Added: Copying a typical book (1 cover, 1 PDF, 1 DOCX, 1 EPUB, 1 metadata.db) from my Intake library to my Books library takes a second or two, copying a 'book' which has audio/video content (FLAC, MP3/4) or hi-res comics (CBZ) take longer of course. The libraries are on a SATA 3.0 2TB HDD.

BR

Last edited by BetterRed; 09-25-2018 at 05:45 PM.
BetterRed is offline   Reply With Quote
Old 09-26-2018, 04:06 AM   #10
Sirtel
Guru
Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.Sirtel ought to be getting tired of karma fortunes by now.
 
Sirtel's Avatar
 
Posts: 996
Karma: 9212212
Join Date: Jan 2014
Location: Estonia
Device: Kindle Voyage, Kindle Oasis 1 & 2
Quote:
Originally Posted by BetterRed View Post
5,000 books is not many books, I have a library with 150,000

60+ columns seems excessive, especially if the library contains actual ebooks rather than 'foreign objects' such as music or movies. Composed columns (columns built from other columns) can have a -ve effect on performance

45 plugins seems excessive, although they should only effect start up times, but I would get rid of any I don't actually use.

60 icon rules also seems excessive, maybe you could reduce the number of rules by using Program Mode templates.

As has already been mentioned, use the View Manager (VM) plugin to limit what columns are shown in the Book List. VM will also define the sort order.

Regarding AntiVirus (AV): IMO if you need to exclude your calibre library from the AV you use, you need to get another AV. I run two, Defender and MalwareBytes, I exclude 1 DLL (a Word Addin) in MWB, and nothing in Defender. As far as I'm concerned tey have no discernible effect of calibre performance.

Added: Copying a typical book (1 cover, 1 PDF, 1 DOCX, 1 EPUB, 1 metadata.db) from my Intake library to my Books library takes a second or two, copying a 'book' which has audio/video content (FLAC, MP3/4) or hi-res comics (CBZ) take longer of course. The libraries are on a SATA 3.0 2TB HDD.

BR
Same here. Almost 20 000 books, 25 columns (with the built-in ones included), 10 plugins, 6 icon rules. 16 GB RAM, 500 GB SSD. Copying 20-50 books all at once takes 10-20 seconds (I only have standard epub/mobi/azw3 books).

I do use the View Manager plugin and several custom columns are not visible (I need them for calibre catalogs). Also I run Library Maintenance regularly.
Sirtel is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Virtual Libraries in WEB? hermann1514 Calibre 2 01-19-2018 01:38 PM
IF statement in virtual libraries sbin Library Management 5 11-01-2016 05:34 AM
Understanding Virtual Libraries Rellwood Library Management 13 11-17-2015 08:39 AM
Virtual Libraries scottmc Library Management 0 08-13-2014 07:15 PM
Virtual Libraries Ortep Library Management 68 12-21-2013 04:23 AM


All times are GMT -4. The time now is 05:56 PM.


MobileRead.com is a privately owned, operated and funded community.