Device management really slow when device = connected folder

nickdma · 11-15-2010, 10:58 AM

I've got a library of about 3k books in Calibre. It's located on a network share, which does slow things down considerably, so I'm aware that's part of the issue...

I use the Connect/share... connect to folder feature to attach a folder as a device to calibre, so that I can use "send to device" and take advantage of the rename/metadata plugin features. Yes, I know I could simply copy the files faster.

It seems like when I connect a folder, any operations on the device take an excessive amount of time to complete. The connected folder isn't nested, and it's practically empty, only containing a few books I happen to be reading at any given time.

For example, simply connecting to the folder causes calibre to stop responding for 2 minutes.

I figure this is probably not an often used feature, and maybe there's just some unnecessary processing that's happening, which when you add a few factors together, balloons into the performance issue I'm seeing.

Not a big deal, just thought I'd mention it in hopes there's some easy tweaking that could make it faster.

From an end user perspective, it just seems out of whack. You wouldn't think it would take much to "connect" to a local folder. I'm sure there's a ton of db queries happening in the background to determine the state of the folder, and that's the source of the slowdown.

Oh, this is with the latest .28 build. It's been this way for awhile now.

kovidgoyal · 11-15-2010, 11:31 AM

I don't see this slowdown connecting to a local (i.e. not networked folder). You should open a ticket and attach more details of your setup. What kind of networked folder, what operations exactly are slow and w=hat are not, what OS, etc.

itimpi · 11-15-2010, 12:03 PM

Are you running Calibre under Windows? If so you might want to look in the resources sub-folder of the Calibre install location and then examine the file calibre-portable.bat. Although that was targetted primarily at those who wanted a portable calibre it also shows how to run with the books on a network share and the metadata database held locally - that can provide a significant performance boost if the bottleneck is database access as sqlite (which is what Calibre uses) is known to be slow running over a network..

nickdma · 11-15-2010, 12:18 PM

Thanks, I'll look into opening a ticket. Like I said, it's not a big deal, but hopefully one of those cases where there's an opportunity for easy optimization.

Moving the metadata local would surely speed it up, but then I lose a central library. I use a running instance of calibre-server on my home server, to share out the books amongst the various deices in the house.

I'm fairly certain it's a limitation of sqlite over the network. I know I pay a performance penalty, and I expect it to be slow with various large metadata operations.

It's just that I wouldn't expect send to device with a connected folder to be a large metadata operation.

kovidgoyal · 11-15-2010, 12:19 PM

send to device causes an iteration over the the books in the db to update the on device column.

ChuckG · 12-27-2010, 02:41 PM

I am maintaining a ~8000 book library over the network as well, and I am seeing a similar slowdown when opening Calibre with my Kindle is attached, or when I send a book to the device. I assume this is due to the updating of the 'on device' column, and it makes Calibre almost unusable since it takes about 5 minutes to update each book, even though the status has only changed for one book.

Is there a way to optimize this, or perhaps disable the 'on device' feature?

11-15-2010, 10:58 AM	#1
nickdma Enthusiast Posts: 41 Karma: 12 Join Date: Oct 2010 Device: Tablet PC	Device management really slow when device = connected folder I've got a library of about 3k books in Calibre. It's located on a network share, which does slow things down considerably, so I'm aware that's part of the issue... I use the Connect/share... connect to folder feature to attach a folder as a device to calibre, so that I can use "send to device" and take advantage of the rename/metadata plugin features. Yes, I know I could simply copy the files faster. It seems like when I connect a folder, any operations on the device take an excessive amount of time to complete. The connected folder isn't nested, and it's practically empty, only containing a few books I happen to be reading at any given time. For example, simply connecting to the folder causes calibre to stop responding for 2 minutes. I figure this is probably not an often used feature, and maybe there's just some unnecessary processing that's happening, which when you add a few factors together, balloons into the performance issue I'm seeing. Not a big deal, just thought I'd mention it in hopes there's some easy tweaking that could make it faster. From an end user perspective, it just seems out of whack. You wouldn't think it would take much to "connect" to a local folder. I'm sure there's a ton of db queries happening in the background to determine the state of the folder, and that's the source of the slowdown. Oh, this is with the latest .28 build. It's been this way for awhile now.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
iPhone Cannot send. No device is connected.	stephenlcraig	Apple Devices	3	12-18-2010 06:55 AM
Folder Device Interface	Sweetpea	Calibre	3	10-02-2010 02:52 AM
On-Device library management (Hanlin V3+)	mistermouse	Calibre	4	06-18-2010 05:27 AM
No device connected error Sony PRS-505	b_knits	Calibre	7	02-23-2010 09:32 PM
No device is connected	pope_a	Calibre	18	09-20-2009 12:29 PM

11-15-2010, 11:31 AM	#2
kovidgoyal creator of calibre Posts: 43,826 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I don't see this slowdown connecting to a local (i.e. not networked folder). You should open a ticket and attach more details of your setup. What kind of networked folder, what operations exactly are slow and w=hat are not, what OS, etc.

11-15-2010, 12:03 PM	#3
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Are you running Calibre under Windows? If so you might want to look in the resources sub-folder of the Calibre install location and then examine the file calibre-portable.bat. Although that was targetted primarily at those who wanted a portable calibre it also shows how to run with the books on a network share and the metadata database held locally - that can provide a significant performance boost if the bottleneck is database access as sqlite (which is what Calibre uses) is known to be slow running over a network..

11-15-2010, 12:18 PM	#4
nickdma Enthusiast Posts: 41 Karma: 12 Join Date: Oct 2010 Device: Tablet PC	Thanks, I'll look into opening a ticket. Like I said, it's not a big deal, but hopefully one of those cases where there's an opportunity for easy optimization. Moving the metadata local would surely speed it up, but then I lose a central library. I use a running instance of calibre-server on my home server, to share out the books amongst the various deices in the house. I'm fairly certain it's a limitation of sqlite over the network. I know I pay a performance penalty, and I expect it to be slow with various large metadata operations. It's just that I wouldn't expect send to device with a connected folder to be a large metadata operation.

11-15-2010, 12:19 PM	#5
kovidgoyal creator of calibre Posts: 43,826 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	send to device causes an iteration over the the books in the db to update the on device column.

12-27-2010, 02:41 PM	#6
ChuckG Junior Member Posts: 3 Karma: 10 Join Date: Dec 2010 Device: Kindle 3	I am maintaining a ~8000 book library over the network as well, and I am seeing a similar slowdown when opening Calibre with my Kindle is attached, or when I send a book to the device. I assume this is due to the updating of the 'on device' column, and it makes Calibre almost unusable since it takes about 5 minutes to update each book, even though the status has only changed for one book. Is there a way to optimize this, or perhaps disable the 'on device' feature?

Advert

Advert