Calibre 64 - adding books gives errors on network drive

[canadianbill]

Hello,

calibre version 0.9.28 [64bit]

I use Windows 7 and I store my eBooks on an external server which is always accessible.

However when I try adding books or amending metadata for books I sometimes get:

ERROR: Failed(64, 'ReadFile', 'The specified network name is no longer available.')

Traceback (most recent call last):
File "site-packages\calibre\gui2\dialogs\metadata_bulk.py", line 125, in do_one_safe
File "site-packages\calibre\gui2\dialogs\metadata_bulk.py", line 177, in do_one
File "site-packages\calibre\library\database2.py", line 2526, in set_authors
File "site-packages\calibre\library\database2.py", line 652, in set_path
File "site-packages\calibre\library\database2.py", line 1527, in add_format
File "site-packages\calibre\library\database2.py", line 1374, in copy_format_to
File "site-packages\calibre\utils\filenames.py", line 326, in copy_path_to
error: (64, 'ReadFile', 'The specified network name is no longer available.')

At this point calibre 64 normally becomes 'stuck' and I have to close it down via Task Manager. Although sometimes when it looks like it is working again and I try to resave edits, I get:

ERROR: Unhandled exception

DatabaseException: database is locked RemoteTraceback (most recent last call): File "site-packages\calibre\library\sqlite.py", line 272, in run OperationalError: database is locked

Traceback (most recent call last):
File "site-packages\calibre\gui2\ui.py", line 819, in closeEvent
File "site-packages\calibre\gui2\ui.py", line 729, in write_settings
File "site-packages\calibre\gui2\init.py", line 306, in save_layout_state
File "site-packages\calibre\gui2\library\views.py", line 378, in save_state
File "site-packages\calibre\gui2\library\views.py", line 372, in write_state
File "site-packages\calibre\library\prefs.py", line 61, in set
File "site-packages\calibre\library\prefs.py", line 56, in __setitem__
File "site-packages\calibre\library\sqlite.py", line 315, in run
DatabaseException: database is locked
RemoteTraceback (most recent call last):
File "site-packages\calibre\library\sqlite.py", line 272, in run
OperationalError: database is locked


Any thoughts or suggestions as to how to deal with these issues?

Kind regards,
Bill

[itimpi]

Calibre does not officially support network drives - so if it works for you then you are lucky! Kovid has said that there are issues with some networked filesystems around database access and locking which seems to be consistent with your symptoms.

Having said that, I run with the library on a network drive (mapped to a drive letter), but the metadata.db file held locally and that seems to be reliable and give reasonable performance. It does mean that I need to remember to sync the local metadata.db file back to the network copy of the library to keep it complete but that is easy enough to automate.

[kovidgoyal]

http://manual.calibre-ebook.com/faq....rked-drive-nas

[canadianbill]

Ok thanks for those suggestions. I will use a local copy and use my network drive for backup purposes. At least most eBooks are not large in size like media files.

Regards,
Bill

[ScottAA]

Originally Posted by kovidgoyal

http://manual.calibre-ebook.com/faq....rked-drive-nas

Are there any plans to make Calibre friendly to network drives in the future?

Currently my calibre library takes up over 10% of the local hard drive, and ebooks are exactly the sort of static media files that are optimal for network drive storage.

I'd like to free up the space used by the actual ebooks, and keep the Calibre database itself on the local drive.

I see from the link above that there are issues with doing that - but are there any plans to ever tackle them and officially support this separation?

[itimpi]

I think that many of the issue with network drives relate to the handling of the metadata.db file over the network, and in particular record locking.

I run my system with the Calibre library on the network (via a mapped drive) , but my metadata.db file held locally with its location specified by the CALIBRE_OVERRIDE_DATABASE_PATH environment variable. I launch Calibre via a batch file that sets this up. This seems to work without any issues for me. I then have the batch file I use to launch Calibre set to copy the metadata.db file back from the local location to the Calibre library held on the network drive.

I am not sure if this is an officially supported way of working but it does seem to run error free,

[Adoby]

I use NFS to connect to my NAS, from Ubuntu, and calibre seems to work flawlessly with the library on a network drive that way. I have never noticed any problems. However, I still have metadata.db on a local filesystem, on a SSD drive. I am trying to convince myself that it improves performance.

[ScottAA]

Originally Posted by itimpi

I run my system with the Calibre library on the network (via a mapped drive) , but my metadata.db file held locally with its location specified by the CALIBRE_OVERRIDE_DATABASE_PATH environment variable. I launch Calibre via a batch file that sets this up. This seems to work without any issues for me. I then have the batch file I use to launch Calibre set to copy the metadata.db file back from the local location to the Calibre library held on the network drive.

That sounds like how I'd like it to work, I'll give it a try.

I'm wondering about that last step you mention, though. I'm on a Mac and there are issues with launching apps from shell scripts (batch file equivalent). Why do you need to copy the metadata.db back to the network drive?

[itimpi]

Originally Posted by ScottAA

Why do you need to copy the metadata.db back to the network drive?

I do not HAVE to do so. I like to do it so that calibre library folder on the network drive is self-contained for backup purposes.

[ScottAA]

Makes sense, thanks for the tips!

[theducks]

Originally Posted by ScottAA

Makes sense, thanks for the tips!

Calibre Library has two parts:
1)the metadata.db (SQL)
2)the book file and folder

These need to happen as a 'transaction'.
If something delays/blocks a file write, the linkage gets fubared

[ScottAA]

Originally Posted by theducks

Calibre Library has two parts:
1)the metadata.db (SQL)
2)the book file and folder

These need to happen as a 'transaction'.
If something delays/blocks a file write, the linkage gets fubared

So are you saying it's a bad idea to do what itimpi is doing?

[itimpi]

Originally Posted by ScottAA

So are you saying it's a bad idea to do what itimpi is doing?

The statement s true as far as it goes. It applies regardless of where the datbase is held. Holding the database locally means this is actually less likely to happen as it handling the database across a network connection that is suspect.