Calibre Network Request

[erwin1234]

Originally Posted by eschwartz

Lots of people doing multiple-computer-access sync the files, so only the changes need to be copied over.

Yes, that is the way to go, but that is not what kenmac999 is doing with his script. Read the thread before replying please.

[erwin1234]

Originally Posted by theducks

erwin
Timing, caching and locking are the biggest banes to a shared network

(this assumes a caseless solution to OS filenames will be implemented)

Any change requires a full lock on (server side) for all affected records AND FILES/Folders

All local caches of data for those must be flushed and Locked, (then refreshed at the end of the change transaction)

BTW the locking system needs to be OS independent

I am not sure why you post this, to what question is this an answer? Is this the way calibre is working? Is it transaction based? Does it do file locking? Are you familiar with the code?

It seems even simple file copy/move/rename operations fail if you operate calibre via a network mapped drive letter. That still strikes me as strange.

[eschwartz]

Originally Posted by erwin1234

Yes, that is the way to go, but that is not what kenmac999 is doing with his script. Read the thread before replying please.

Am I not allowed to suggest things now?

[theducks]

Originally Posted by erwin1234

I am not sure why you post this, to what question is this an answer? Is this the way calibre is working? Is it transaction based? Does it do file locking? Are you familiar with the code?

It seems even simple file copy/move/rename operations fail if you operate calibre via a network mapped drive letter. That still strikes me as strange.

Those are Requirement to be successful on a network.
AFAIK Calibre does none, which is why it is single user. It can't even abide delayed writes.

[CRussel]

Calibre works fine on a network share IF you understand that it must ALWAYS be single user. The issue isn't the network share, it's the multiple user access. Because, as theducks points out, you need to actually lock files and folders if you're going to allow multiple people to have access at the same time.

Doing this in a platform independent manner is usually done with a relational database where the smarts are built into the database. But changing to that kind of back end would turn Calibre into a very different program. Instead, to provide transparent access to your books at all times, even if Calibre goes away, Kovid and the developers have chosen what is, essentially, a flat-file system. I can perfectly understand that decision. Even though it's a minor nuisance for me personally. I'd still rather have it this way than to have all my books hiding inside a relational database.

[theducks]

Originally Posted by CRussel

Calibre works fine on a network share IF you understand that it must ALWAYS be single user. The issue isn't the network share, it's the multiple user access. Because, as theducks points out, you need to actually lock files and folders if you're going to allow multiple people to have access at the same time.

Doing this in a platform independent manner is usually done with a relational database where the smarts are built into the database. But changing to that kind of back end would turn Calibre into a very different program. Instead, to provide transparent access to your books at all times, even if Calibre goes away, Kovid and the developers have chosen what is, essentially, a flat-file system. I can perfectly understand that decision. Even though it's a minor nuisance for me personally. I'd still rather have it this way than to have all my books hiding inside a relational database.

I would not call Calibre a flatfile system, but some sort of hybrid.

It would take a lot (HUGE) of work, but Calibre could probably be turned into a true client-server.
OMG getting it to work on 3 OS

I can imagine the support nightmares for the basic (typical) desktop user, just setting up the client and the server (the can be on the same machine), then add the complexity of additional clients boggles.

It is quite possible that the GUI would not change, only what happens behind it.

[Adoby]

Perhaps the calibre database server backend could be optional and run as an appliance from inside a virtual machine. That way only the client(s) have to be multi platform.

[CRussel]

Originally Posted by theducks

I would not call Calibre a flatfile system, but some sort of hybrid.

Fair enough. Though it bears a startling resemblance from what I can see to an avowedly flatfile, client/server, system I supported in my past. I wasn't trying to denigrate, merely describe.

It would take a lot (HUGE) of work, but Calibre could probably be turned into a true client-server.
OMG getting it to work on 3 OS

I can imagine the support nightmares for the basic (typical) desktop user, just setting up the client and the server (the can be on the same machine), then add the complexity of additional clients boggles.

It is quite possible that the GUI would not change, only what happens behind it.

Oh, I absolutely agree. This is a completely non-trivial change. And I'm not at all sure it's worth the effort. Really. It is, after all, basically a single user app and I don't honestly have a problem with that. Yes, it would be nice to be able to stick it on the Terminal Server and let anyone in the house connect to it simultaneously. But it's hardly necessary.

One alternative might be to set a "read-only" flag for the second user connecting to a library. Sort of like Word does with a document that is open by another user. The second user could transfer stuff to their eReader or even read a book, but couldn't make changes to metadata or add books.

[erwin1234]

I appreciate the discussions and suggestions to make calibre multi-user and the difficulties to overcome when transforming calibre into a true networked client server architecture. These are recognized. But this discussion is off-topic and diverges from the main subject of the thread, which is not about multi user (the author is working on this) but about single user access to a networked drive.

So please let me repeat my observation in a different way: why has calibre problems to do simple file operarations on single files (the files and folders of the ebook data themselves) on a networked drive (or mapped -letter). Even so in a completely Win7 environment (so win7 client win7 server over network). I do not know any other software that does work on a local hardrive D:\ but not on a networked mapped network drive D:\. What is so special about calibre file I/O, please explain.

Originally Posted by CRussel

Calibre works fine on a network share IF you understand that it must ALWAYS be single user.

Well, this is exactly the point: it is NOT working fine! Please test it yourself and post here what your findings are!

[itimpi]

I have no problem holding the library book files on a networked drive and the metadata database locally. Things tend to go wrong if I also try to put the metadata database onto the network drive. This leads me to believe that it is something about the interactions between calibre and its metadata database that tend to go wrong when this database is on a network drive.

[Adoby]

Also, if/when calibre really becomes multiuser, then that implies that the problems with access over the network has been solved. Perhaps the two are not directly related, but multiuser support necessitate network access.

[CRussel]

Originally Posted by erwin1234

Well, this is exactly the point: it is NOT working fine! Please test it yourself and post here what your findings are!

I use it that way every day. My Library resides on K:\, which is a mapped network share to my Windows Storage Server.

[mbovenka]

Originally Posted by CRussel

I use it that way every day. My Library resides on K:\, which is a mapped network share to my Windows Storage Server.

Same here on my Synology NAS (Linux + Samba, effectively).

[theeye23]

I use calibre portable, after I put the calibre portable to our server and share the folder and mapped it to a network drive, I set the security permission of the Library folder only to read-only for everyone except me, that way, only I can change the book, though they can still change the preferences. It's ok, at least they will not ruin the e-books. It's still working fine.

[erwin1234]

Originally Posted by CRussel

I use it that way every day. My Library resides on K:\, which is a mapped network share to my Windows Storage Server.

Very interesting. I would like to know why it works with windows storage server and not with my config which is on a windows 7 ultimate mapped network share.

Just re-checked with latest calibre 1.22.0.
1. install calibre. Enable debug mode, debug file is in \Appdata\local\temp.
2. create new library on network share on map drive on server S:\, full access on share, full admin access on server
3. import some ebooks. This works fine. All folders and files are created allright
4. change metadata -> change title of book. Press ok. This does not work. I get the famous error again:

calibre, version 1.22.0
FOUT: Toegang geweigerd: Kan van dit boek de locatie op de harde schijf niet wijzigen. Is dit boek geopend in een ander programma?

Traceback (most recent call last):
File "site-packages\calibre\gui2\metadata\basic_widgets.py", line 102, in commit
File "site-packages\calibre\db\legacy.py", line 814, in func
File "site-packages\calibre\db\cache.py", line 57, in call_func_with_lock
File "site-packages\calibre\db\cache.py", line 969, in set_field
File "site-packages\calibre\db\cache.py", line 983, in update_path
File "site-packages\calibre\db\backend.py", line 1428, in update_path
File "site-packages\calibre\db\backend.py", line 1230, in copy_cover_to
File "site-packages\calibre\utils\filenames.py", line 397, in copy_path_to
File "site-packages\calibre\startup.py", line 155, in local_open
OSError: [Errno 13] Permission denied


I pressed cancel and did a second rename. In the library folder, the new title folders are all there (so three folders now for the same one book), but the files are not ok. In the first rename the epub has been renamed and the cover.jpg is there, the second time the rename of the epub failed and the third folder only has the cover.jpg. This file cover.jpg also remains in the first (original) folder.

So all sorts of renaming/copying/moving goes wrong. That is why I have my main library read-only on the network share now.

The debug mode does not help here, there are NO error messages in the debug file.

You are lucky it works for you..