Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 05-04-2013, 08:51 PM   #1
offby1
Junior Member
offby1 began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2010
Device: iPhone w/ Stanza
Calibre content-server doesn't seem to see new books

I keep my ebook library in Dropbox, and on one of my machines I have a launchd daemon that runs the calibre content server:

calibre-server --with-library=$HOME/Dropbox/ebooks

I can confirm that the server serves my library, but it does not appear to detect changes to the library that happen on another system; I mainly run Calibre on another machine and manipulate the library there, relying on Dropbox to sync it.

I can tell (using calibredb list) that my newly added books are just not appearing. I've killed and restarted the calibre server, reloaded the launchd daemon, really everything I can think of, and no good has come of it.

How can I make the content server pick up these changes?
offby1 is offline   Reply With Quote
Old 05-04-2013, 08:59 PM   #2
chaley
Grumpy old git
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 8,999
Karma: 2737818
Join Date: Jan 2010
Location: UK
Device: Many android devices
Are you saying that the books do not appear if you restart the content server, but are there if you run calibredb with the same --with-library argument?
chaley is offline   Reply With Quote
Advert
Old 05-04-2013, 09:55 PM   #3
offby1
Junior Member
offby1 began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2010
Device: iPhone w/ Stanza
No, I am not seeing them either way. The only places I can see them are A) on the filesystem of the machine that is hosting the content server, and B) in the Calibre instance running on my other computer.
offby1 is offline   Reply With Quote
Old 05-04-2013, 10:31 PM   #4
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,732
Karma: 13739682
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
Quote:
Originally Posted by offby1 View Post
I can confirm that the server serves my library, but it does not appear to detect changes to the library that happen on another system; I mainly run Calibre on another machine and manipulate the library there, relying on Dropbox to sync it.
I haven't attempted this, but if calibre-server has the library opened in dropbox then the metadata.db will be locked and it will not be updated via any external device while locked.

If you search for conflicted copy in your dropbox folder I'm betting you find a few. These happen when another device updates something that is in use on the second device.

If you use dropbox to sync, only one device should be accessing the dropbox library at a time. You may want to consider a different method for syncing the folders. Preferably at a time when neither library is being used.
DoctorOhh is offline   Reply With Quote
Old 05-04-2013, 11:05 PM   #5
chaley
Grumpy old git
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 8,999
Karma: 2737818
Join Date: Jan 2010
Location: UK
Device: Many android devices
Quote:
Originally Posted by DoctorOhh View Post
I haven't attempted this, but if calibre-server has the library opened in dropbox then the metadata.db will be locked and it will not be updated via any external device while locked.
I use dropbox as my transport between the machine where I run calibre and the linux server that runs my content server. My experience indicates that dropbox uses a delete and create scheme on linux, which usually avoids the locked file problem. Unfortunately that also means that programs having the deleted file open (e.g., the content server) continue to see the deleted file, not the new file. I get around this by using monit to monitor the changed timestamp of metadata.db, restarting the content server. To be sure that I don't suffer from conflicts I copy the metadata.db file to /tmp and tell calibre to use the copy.

I agree with doctorohh - there must have been a conflict on metadata.db at some point. You need to ensure that all computers are seeing the same files.
chaley is offline   Reply With Quote
Advert
Old 05-05-2013, 02:28 AM   #6
offby1
Junior Member
offby1 began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2010
Device: iPhone w/ Stanza
There's no metadata.db conflict; I assumed that must have been the case, but it turns out no. For what it's worth, the content server _only_ reads the DB, it never (at least, not under my control) writes to the DB.

I have resolved the issue, however; it turns out I'd misunderstood how and when calibre writes to the DB; it writes on exit, not on edit, which meant that my changes had not persisted until I quit Calibre on my editing host.

*facepalm*

Thanks for the feedback.
offby1 is offline   Reply With Quote
Old 05-05-2013, 02:34 AM   #7
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,732
Karma: 13739682
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
Quote:
Originally Posted by offby1 View Post
There's no metadata.db conflict; I assumed that must have been the case, but it turns out no. For what it's worth, the content server _only_ reads the DB, it never (at least, not under my control) writes to the DB.

I have resolved the issue, however; it turns out I'd misunderstood how and when calibre writes to the DB; it writes on exit, not on edit, which meant that my changes had not persisted until I quit Calibre on my editing host.

*facepalm*

Thanks for the feedback.
Excellent, thanks for the update.
DoctorOhh is offline   Reply With Quote
Old 05-05-2013, 10:58 AM   #8
chaley
Grumpy old git
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 8,999
Karma: 2737818
Join Date: Jan 2010
Location: UK
Device: Many android devices
Quote:
Originally Posted by offby1 View Post
There's no metadata.db conflict; I assumed that must have been the case, but it turns out no. For what it's worth, the content server _only_ reads the DB, it never (at least, not under my control) writes to the DB.

I have resolved the issue, however; it turns out I'd misunderstood how and when calibre writes to the DB; it writes on exit, not on edit, which meant that my changes had not persisted until I quit Calibre on my editing host.

*facepalm*

Thanks for the feedback.
just for completeness because it might matter someday -- calibre does write to the db on each update. In most cases it opens a database transaction, writes everything it needs to write, then commits the transaction. However, even though the db is up-to-date, calibre is still holding the lock on the db. Depending on the OS and some other things, that lock will prevent other programs from opening and copying the db.

Also, even though the content server does not write to the db, it does open it for update. Python, calibre's implementation language, does not support read-only database access. Because the db is open for write there is a lock, which can block other programs from accessing the db.
chaley is offline   Reply With Quote
Reply

Tags
contentserver, dropbox, launchd, mac

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
is there a tool to open (read) books from the calibre's content server? tsolignani Related Tools 2 04-16-2013 07:14 AM
calibre-server: Doesn't server up newly added files kaistian Calibre 5 03-17-2013 11:54 AM
Calibre content server downloading doesn't include metadata cerasi Devices 0 09-08-2012 07:27 PM
500 Internal Server Error accessing content server Calibre 0.8.8 DaddyO57 Calibre 1 07-20-2012 06:08 PM
Content server download doesn't complete on Blackberry jsm555 Devices 0 03-13-2012 02:42 AM


All times are GMT -4. The time now is 10:48 AM.


MobileRead.com is a privately owned, operated and funded community.