Calibre GUI very slow while using calibredb

qwertyuio · 12-22-2020, 06:40 PM

I have the GUI open with server enabled. I have a script running that is adding books and setting a few fields using calibredb. I find the GUI is so slow and unresponsive that it is unusable.

I understand there is database locking going on while metadata is added But I would have expected the time to update a few tables is very short. Is Calibre locking the entire database while copying the books?

Would using the DB API be faster?

Also, when adding books the current search filter is ignored. For example if I have the search filter enabled with 5 books displayed, new books currently being added by calibredb will keep being added to the view even though they don't match the search filter.

JSWolf · 12-22-2020, 08:36 PM

How many eBooks do you have in your database?

davidfor · 12-22-2020, 09:19 PM

@qwertyuio: I have noticed this when book are added via the automatic adding. If I drop a few books into the folder, the UI displays the busy cursor and it is unresponsive while the the books are added. I think it will happen with all methods of adding books as calibre is copying the file to it's structure, opening it to get the metadata and updating the database.

kovidgoyal · 12-22-2020, 09:25 PM

Ignoring the search filter is by design and applies to any method of adding books. As for slowness, it is inevitable since adding takes place in the main GUI thread, so it will block. And adding involves FAR more than updating a few database tables.

qwertyuio · 12-23-2020, 12:12 AM

Is it possible to run calibre-server and connect the GUI to it? Or could the server in the GUI run on a separate thread? It feels like the MS-DOS days of only being able to perform a single task at a time. I have a 3Ghz quad core and I can not add books and work in the GUI at the same time.

qwertyuio · 12-23-2020, 12:14 AM

Quote:

Originally Posted by davidfor

I think it will happen with all methods of adding books as calibre is copying the file to it's structure, opening it to get the metadata and updating the database.

I'm wondering if "calibredb add" could have a "move" option to avoid the copying (assuming same filesystem). Since I delete the book after adding it so that would be one less thing for calibre to take up time doing.

qwertyuio · 12-23-2020, 12:16 AM

Quote:

Originally Posted by JSWolf

How many eBooks do you have in your database?

About 100k. Is there a limit at which point calibre becomes too slow and I should start using separate libraries?

kovidgoyal · 12-23-2020, 12:41 AM

Quote:

Originally Posted by qwertyuio

Is it possible to run calibre-server and connect the GUI to it? Or could the server in the GUI run on a separate thread? It feels like the MS-DOS days of only being able to perform a single task at a time. I have a 3Ghz quad core and I can not add books and work in the GUI at the same time.

You should not be adding books and working in the GUi at the same time anyway, since then any gui operations you perform are operating on unknown sets of books. Adding book sin calibre is a relatively rare operation, after the initial stages of library creation.

12-22-2020, 06:40 PM	#1
qwertyuio Enthusiast Posts: 28 Karma: 10 Join Date: Jan 2019 Device: none	Calibre GUI very slow while using calibredb I have the GUI open with server enabled. I have a script running that is adding books and setting a few fields using calibredb. I find the GUI is so slow and unresponsive that it is unusable. I understand there is database locking going on while metadata is added But I would have expected the time to update a few tables is very short. Is Calibre locking the entire database while copying the books? Would using the DB API be faster? Also, when adding books the current search filter is ignored. For example if I have the search filter enabled with 5 books displayed, new books currently being added by calibredb will keep being added to the view even though they don't match the search filter.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibredb not working when GUI is running	webdjoe	Calibre	9	07-26-2017 04:29 AM
calibredb set_custom(etc) GUI refresh?	JimmXinu	Library Management	7	09-12-2016 08:52 PM
Calibre GUI is very slow to update	Katsunami	Calibre	22	07-12-2014 08:26 AM
Option for 'calibredb' to not notify a running Calibre GUI?	mbovenka	Development	4	11-08-2012 04:30 AM
calibredb set_custom GUI question	Sabkor	Library Management	4	03-12-2011 08:51 PM

12-22-2020, 08:36 PM	#2
JSWolf Resident Curmudgeon Posts: 79,657 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	How many eBooks do you have in your database?

12-22-2020, 09:19 PM	#3
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	@qwertyuio: I have noticed this when book are added via the automatic adding. If I drop a few books into the folder, the UI displays the busy cursor and it is unresponsive while the the books are added. I think it will happen with all methods of adding books as calibre is copying the file to it's structure, opening it to get the metadata and updating the database.

12-22-2020, 09:25 PM	#4
kovidgoyal creator of calibre Posts: 45,307 Karma: 27111242 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Ignoring the search filter is by design and applies to any method of adding books. As for slowness, it is inevitable since adding takes place in the main GUI thread, so it will block. And adding involves FAR more than updating a few database tables.

12-23-2020, 12:12 AM	#5
qwertyuio Enthusiast Posts: 28 Karma: 10 Join Date: Jan 2019 Device: none	Is it possible to run calibre-server and connect the GUI to it? Or could the server in the GUI run on a separate thread? It feels like the MS-DOS days of only being able to perform a single task at a time. I have a 3Ghz quad core and I can not add books and work in the GUI at the same time.

Advert

Advert