MobileRead Forums - View Single Post - Keeping book on NAS but metadata.db on local (Ubuntu)

q345 · 05-26-2010, 06:15 PM

I shouldn't have used the word back-up, since what I've suggested is not.
It's just an implementation of having data and metadata of an e-book in one place, like in mp3 file.

I think everybody agree that some of the numerous e-book formats are deficient because they don't have place for metadata, and that the source of the problems. So instead of file let's allocate one directory per e-book in all formats ( as it now in Calibre ) or even one directory per a particular format of a particular e-book and keep there e-book itself and an additional small file with it's metadata.

Chaley, regarding the performance, indeed if one uses Calibre to maintain a huge library and performs constant massive tag editing, it might be a slight drop in performance ( remember those additionl metadata file are very tiny),
but, on my uneducated opinion and I might be wrong, for majority of people working with Calibre, frequency of read request to database is much higher then the write requests. Obviously, if you try to use Calibre to manage Library of Congress collection it might be very slow, but I guess big libraries uses different tools anyway.

Kovid, just to clairify that what I suggest is completely independent/orthogonal of back-up. Of course regular back-up are a must for ANYTHING, including Calibre. Still, a reasonable efforts are made in any application to make it more robust and fault-tolerant and as somebody else mentioned it's not for the e-books per se.

You are the owner of the code and you have much better perspective on the issue, but I think implementation of this feature is not too difficult and it doesn't even change anything in the design. Simply if you modify some rows in metadata.db , you also modify some of these small files in correspondent directories ( The easiest way just dump SQL for a certain row into a file if this row has been changed). It might even be controlled by the option. These files are not used for regular search, but for recovery only.

Of course, one might ask what if people start tampering with this small metadata files and move them. Well, even now I can tamper with big metadata.db file and screw everything with a very little effort. A single warning in the manual "Files are automatically generated. Don't move/edit them" should suffice.

05-26-2010, 06:15 PM	#24
q345 Member Posts: 23 Karma: 10 Join Date: May 2010 Device: none	Mea culpa ( back-up was a wrong choic of word) I shouldn't have used the word back-up, since what I've suggested is not. It's just an implementation of having data and metadata of an e-book in one place, like in mp3 file. I think everybody agree that some of the numerous e-book formats are deficient because they don't have place for metadata, and that the source of the problems. So instead of file let's allocate one directory per e-book in all formats ( as it now in Calibre ) or even one directory per a particular format of a particular e-book and keep there e-book itself and an additional small file with it's metadata. Chaley, regarding the performance, indeed if one uses Calibre to maintain a huge library and performs constant massive tag editing, it might be a slight drop in performance ( remember those additionl metadata file are very tiny), but, on my uneducated opinion and I might be wrong, for majority of people working with Calibre, frequency of read request to database is much higher then the write requests. Obviously, if you try to use Calibre to manage Library of Congress collection it might be very slow, but I guess big libraries uses different tools anyway. Kovid, just to clairify that what I suggest is completely independent/orthogonal of back-up. Of course regular back-up are a must for ANYTHING, including Calibre. Still, a reasonable efforts are made in any application to make it more robust and fault-tolerant and as somebody else mentioned it's not for the e-books per se. You are the owner of the code and you have much better perspective on the issue, but I think implementation of this feature is not too difficult and it doesn't even change anything in the design. Simply if you modify some rows in metadata.db , you also modify some of these small files in correspondent directories ( The easiest way just dump SQL for a certain row into a file if this row has been changed). It might even be controlled by the option. These files are not used for regular search, but for recovery only. Of course, one might ask what if people start tampering with this small metadata files and move them. Well, even now I can tamper with big metadata.db file and screw everything with a very little effort. A single warning in the manual "Files are automatically generated. Don't move/edit them" should suffice.