Speed up adding books

democrite · 01-08-2020, 06:18 PM

Hello,

It would be nice of adding books could be faster. Small works on a decently quick SSD might take 3 or 4 seconds though larger books that I often add such as a test of a 400 meg EPUB takes 35 seconds. It seems that calibre is unzipping the whole file when I think all that's needed is the container.xml, OPF, and the cover. At least in cases when a large book takes a long time to add, the abort button doesn't respond until processing is completed, and in one test, the book is still added. As I often add large works, it would be nice if it were quicker.

A few bugs with 4.x. The adding books dialog on macOS has a minimum width of around 1050 points. Maybe that is 2x retina or something. And dragging out a multiple selection to copy books out of calibre no longer works.

JSWolf · 01-08-2020, 06:27 PM

On adding, Calibre is reading the metadata, extracting the cover, checking for possible duplicates, saving the cover, the eBook, and metadata.opf as a backup, and updating the database. So a lot is going on. And if you have any plugins that do something when you add a book, take that into consideration as well.

democrite · 01-08-2020, 07:04 PM

I have few plugins installed. Some to cleanup selected works, few that I can think of that would be used on import except for Apprentice Alf's. It just seems that unzipping only the needed files, parsing of metadata and updating of SQLite, copying the file, and running of plugins such as for DRM could be much quicker. If for example one adds several 100+ meg files, such can take several minutes. It seems more optimization could be done.

kovidgoyal · 01-08-2020, 08:34 PM

calibre does not unzip entire epub files to read metadata, unless you have some third party plugins installed, then all bets are off.

BetterRed · 01-08-2020, 08:56 PM

Quote:

Originally Posted by JaneSanchez

I have the same problem：D：D：D

What OS and File System, and what sort of disk drive.

BR

democrite · 01-08-2020, 09:17 PM

The DRM app itself takes maybe 1 or 2 seconds so I'd guess the plugin should be nearly as quick. I can't think of any other plugins that would do anything on import. Even adding a 400 meg EPUB without DRM takes 30+ seconds to import. I'm not sure what it is.

kovidgoyal · 01-08-2020, 09:26 PM

Disable your third party plugins then check the time.

BetterRed · 01-08-2020, 09:37 PM

Try adding a 400 MB video, then all calibre has to do is copy the file to the library folder. The times your're quoting are almost 10 times what I get on my Windows 10/NTFS/6Gbs SATA III system.

democrite · 01-08-2020, 11:06 PM

It was a plugin.

davidfor · 01-09-2020, 12:37 AM

Quote:

Originally Posted by democrite

It was a plugin.

Which one?

democrite · 01-09-2020, 12:52 AM

Apprentice Alf.

01-08-2020, 06:18 PM	#1
democrite Evangelist Posts: 440 Karma: 77256 Join Date: Sep 2011 Device: none	Speed up adding books Hello, It would be nice of adding books could be faster. Small works on a decently quick SSD might take 3 or 4 seconds though larger books that I often add such as a test of a 400 meg EPUB takes 35 seconds. It seems that calibre is unzipping the whole file when I think all that's needed is the container.xml, OPF, and the cover. At least in cases when a large book takes a long time to add, the abort button doesn't respond until processing is completed, and in one test, the book is still added. As I often add large works, it would be nice if it were quicker. A few bugs with 4.x. The adding books dialog on macOS has a minimum width of around 1050 points. Maybe that is 2x retina or something. And dragging out a multiple selection to copy books out of calibre no longer works.

01-08-2020, 06:27 PM	#2
JSWolf Resident Curmudgeon Posts: 79,442 Karma: 145491800 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	On adding, Calibre is reading the metadata, extracting the cover, checking for possible duplicates, saving the cover, the eBook, and metadata.opf as a backup, and updating the database. So a lot is going on. And if you have any plugins that do something when you add a book, take that into consideration as well. Last edited by JSWolf; 01-08-2020 at 07:44 PM.

01-08-2020, 11:06 PM	#9
democrite Evangelist Posts: 440 Karma: 77256 Join Date: Sep 2011 Device: none	It was a plugin. Last edited by democrite; 01-08-2020 at 11:50 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Tips to speed up adding books?	sealbeater	Library Management	5	07-10-2016 11:09 PM
Speed reading books recommendations	jocampo	Reading Recommendations	3	09-15-2011 12:37 PM
Remembering Books/Loading Speed When Connecting Device	jesscat	Calibre	19	04-17-2011 09:50 AM
Adding books speed question	karmalized	Calibre	3	03-02-2009 11:32 PM
Text to Speech and audio books - speed?	moz	Reading and Management	3	05-30-2008 02:02 PM

01-08-2020, 07:04 PM	#3
democrite Evangelist Posts: 440 Karma: 77256 Join Date: Sep 2011 Device: none	I have few plugins installed. Some to cleanup selected works, few that I can think of that would be used on import except for Apprentice Alf's. It just seems that unzipping only the needed files, parsing of metadata and updating of SQLite, copying the file, and running of plugins such as for DRM could be much quicker. If for example one adds several 100+ meg files, such can take several minutes. It seems more optimization could be done.

01-08-2020, 08:34 PM	#4
kovidgoyal creator of calibre Posts: 45,253 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	calibre does not unzip entire epub files to read metadata, unless you have some third party plugins installed, then all bets are off.

01-08-2020, 09:17 PM	#6
democrite Evangelist Posts: 440 Karma: 77256 Join Date: Sep 2011 Device: none	The DRM app itself takes maybe 1 or 2 seconds so I'd guess the plugin should be nearly as quick. I can't think of any other plugins that would do anything on import. Even adding a 400 meg EPUB without DRM takes 30+ seconds to import. I'm not sure what it is.

01-08-2020, 09:26 PM	#7
kovidgoyal creator of calibre Posts: 45,253 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Disable your third party plugins then check the time.

01-08-2020, 09:37 PM	#8
BetterRed null operator (he/him) Posts: 21,662 Karma: 29711016 Join Date: Mar 2012 Location: Sydney Australia Device: none	Try adding a 400 MB video, then all calibre has to do is copy the file to the library folder. The times your're quoting are almost 10 times what I get on my Windows 10/NTFS/6Gbs SATA III system.

01-09-2020, 12:52 AM	#11
democrite Evangelist Posts: 440 Karma: 77256 Join Date: Sep 2011 Device: none	Apprentice Alf.

Advert

Advert