Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Devices

Notices

Reply
 
Thread Tools Search this Thread
Old 12-27-2017, 06:59 PM   #1
dlfuller
Member
dlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enough
 
Posts: 24
Karma: 566
Join Date: May 2009
Device: none
Simple Approach to an External Library and Server using 3.14.0

I have the calibre app running on macOS with its library on a headless Ubuntu Server. The content server is never started from the menubar in the app.

Calibre is also installed on Ubuntu Server providing the calibredb command line interface to the calibre database and calibre-server which starts the server.

My simple approach with publication revisions is to modify them in the calibre app on the Mac then quit. Next stop and relaunch calibre-server on the Ubuntu Server.
Code:
sudo killall calibre-server
sudo calibre-server &
Those revised publications are then available to all devices on the server.

To add books, I add or drop them into the calibre app, check, modify, and convert if necessary. Quit the app. Again the same routine, stop and relaunch calibre-server on the Ubuntu Server.
Code:
sudo killall calibre-server
sudo calibre-server &
The publications are added to the library on the server and available to all devices.

The steps are easy to do and the calibre app keeps up to date. What might I be missing with this simple technique that would allow data corruption or loss?
dlfuller is offline   Reply With Quote
Old 12-27-2017, 09:08 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,485
Karma: 10205098
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The main concern with this is that macOS file systems and linux filesstems are different (one is case sensistive, one isn't) so directly sharing the same library between them can be problematic.
kovidgoyal is offline   Reply With Quote
Old 12-28-2017, 01:35 PM   #3
dlfuller
Member
dlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enough
 
Posts: 24
Karma: 566
Join Date: May 2009
Device: none
Thanks for the insight @kovidgoyal (and of course for all your ongoing efforts with calibre).

Is this a solvable situation?

As an example, what might happen the way it is? My use of the macOS calibre app is limited to adding books, format conversions, adding tags, revising metadata, etc. Once those publications are available on the server they are only read or downloaded through the calibre-server URL from multiple devices regardless of OS. Basically no reading in the macOS calibre app.
dlfuller is offline   Reply With Quote
Old 12-28-2017, 02:03 PM   #4
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 52,387
Karma: 47516399
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Aura H2O, Sony PRS-650, Sony PRS-T1, nook STR, iPad 4, iPhone 5
Given that the library is on Linux, use that and not OSX.
JSWolf is offline   Reply With Quote
Old 12-28-2017, 02:17 PM   #5
dlfuller
Member
dlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enough
 
Posts: 24
Karma: 566
Join Date: May 2009
Device: none
Fine @JSWolf, except this is a headless server. Those adding or editing activities for books are a pain just using the CLI.
dlfuller is offline   Reply With Quote
Old 12-28-2017, 06:15 PM   #6
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,281
Karma: 24761108
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD, Aura ONE, Clara HD
Quote:
Originally Posted by dlfuller View Post
Fine @JSWolf, except this is a headless server. Those adding or editing activities for books are a pain just using the CLI.
Calibre can be made to watch a directory and load any books found there. Try using the save-to-disk from the MAC and put them in a shared folder on the server to pickup. Or, save them on the MAC and use a script to process the folder running calibredb against each to send them to the server instance of calibre.

Actually, you could probably use the Open With plugin to add a menu item to call the calibredb command and send the book to the server. But, you would want to update the metadata and cover before doing this. The above method would do this as part of the save-to-disk.
davidfor is offline   Reply With Quote
Old 12-28-2017, 08:52 PM   #7
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,093
Karma: 10563148
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by davidfor View Post
Calibre can be made to watch a directory and load any books found there. Try using the save-to-disk from the MAC and put them in a shared folder on the server to pickup. Or, save them on the MAC and use a script to process the folder running calibredb against each to send them to the server instance of calibre.

Actually, you could probably use the Open With plugin to add a menu item to call the calibredb command and send the book to the server. But, you would want to update the metadata and cover before doing this. The above method would do this as part of the save-to-disk.
@davidfor - the first post has this

Quote:
Originally Posted by dlfuller View Post
I have the calibre app running on macOS with its library on a headless Ubuntu Server.
I interpret that as saying the Mac install of calibre is accessing a library on the server, which is presumably running Ext3 or Ext4. When calibre-server is run at the server it presumably uses the same library.

My question would be - will the files calibre creates and maintains from the MAC client, be named according to Ext3/4 rules (case sensitive) or HFS+ rules (case insensitive). I have no idea, so I would do some experiments

BR
BetterRed is offline   Reply With Quote
Old 12-28-2017, 09:03 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,485
Karma: 10205098
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
It's on my TODO list to add support for adding books/converting them and editing metadata to the browser interface, I just did not have time to finish it for calibre 3.x and unfortunately because of some stuff in my personal life, my calibre development time is severely curtailed at the moment, hopefully it will pick up steam again soon and the server will gain those features.
kovidgoyal is offline   Reply With Quote
Old 12-28-2017, 10:17 PM   #9
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,281
Karma: 24761108
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD, Aura ONE, Clara HD
Quote:
Originally Posted by BetterRed View Post
@davidfor - the first post has this



I interpret that as saying the Mac install of calibre is accessing a library on the server, which is presumably running Ext3 or Ext4. When calibre-server is run at the server it presumably uses the same library.
I misread the first post. And based on the comments made since calibre v3 was released I'm surprised it works. The OP seems to have two instances of calibre (one for the server, one for the GUI) running at the same time on the same library. I'm surprise it is working. If the OP was stopping the server before opening the GUI, then the only concern would be the file systems.
Quote:
My question would be - will the files calibre creates and maintains from the MAC client, be named according to Ext3/4 rules (case sensitive) or HFS+ rules (case insensitive). I have no idea, so I would do some experiments
Whether that is an issue depends on how the MAC is mounting the share from the server. It should be mounting it as a networked file system, not a native file system such as HFS+ or an EXT variant. I know Windows has no issues mounting EXT3/4 shares and using them. I can't check now, but I'm reasonably sure there is no way to tell what the remote file system is. It's probable that the client OS concerned won't support the full function of the underlying physical file system used on server, but the server should be returning errors to the client OS when it does something it shouldn't. The differences in case sensitivity mentioned by Kovid are a good example of this, and I don't remember how Windows handles this. If the server has files with the same name but different cases, I don't know if Windows will show in a directory listing. I have played with this in the past, but I don't remember exactly what happen, and I'm away from home and can't test what happens.

It will also be different depending on how the remote drive is accessed. Samba, NFS, FTP or something else will probably all give slightly different results.
davidfor is offline   Reply With Quote
Old 12-28-2017, 11:16 PM   #10
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,093
Karma: 10563148
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@davidfor - and I might have misread a different part of the opening post

I thought it said the calibre-server was being stopped before starting the MAC client. But on re-reading it would appear the OP leaves calibre-server running while updating the library from the MAC, then when the MAC is closed the server is stopped and started. My guess is that this sequence is used to minimise the server unavailability time.

Would calibre-server.exe running on the server against a library on a local drive detect that a MAC elsewhere on the network was running the calibre GUI against the same library - or vice versa? I am under the impression it wouldn't, but I'm not up to speed with the version 3 server - I use the old 2.85.1 server.

I recall reading of user implemented schemes where the presence of a 'library.lock' file in the server hosted library folder was used to prevent concurrent execution of calibre.exe on two or more workstations.

BR
BetterRed is offline   Reply With Quote
Old 12-29-2017, 12:04 AM   #11
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,281
Karma: 24761108
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD, Aura ONE, Clara HD
My use of the calibre server has always been pretty basic. It's enabled to run when I have calibre running on my laptop. I really only use it as one of the tests for the browser on my Kobo devices.

I doubt if a service running on a Linux server will be able to tell if a client machine is directly accessing a file on the server. It depends on what locking is done. And that depends on the applications, OSes and file systems involved.

From what I understand of what Kovid has said about the v3 server, what the OP is doing shouldn't work. Or will have problems. The calibre UI is updating the database, independently of the calibre server, and there is a potential for disaster. What the OP is doing sounds like something for calibre v2 and they haven't changed.

And yes, lock files have been a popular way of trying to prevent multiple updates or uses of an application. They can work but everyone (code or people) has always update them appropriately. Otherwise, all you hear in the office is someone searching for someone to remove the lock file. Or trying to find the person who went to lunch without closing the application and now their PC is locked.
davidfor is offline   Reply With Quote
Old 12-29-2017, 11:48 AM   #12
wanni
Junior Member
wanni began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Dec 2017
Device: Pocketbook Lux 3
I'm running as well 2 clients against 1 DB. One on the server (windows) where the books and the DB resides. And there the content-server then publishes all books to the outside so I can access it while traveling from my ebook-reader.
The other client runs on my PC as this is where I manage new book entries (so I don't need remote access). As you add new books they then are automatically stored on the server where the DB is (just using a network path).
However, same as above with the Linux version: Calibre on the server does not recognize new entries without a restart. A automatic "refresh" reading of the DB would be great so I don't need to restart Calibre anymore.
Has anybody an idea? Or would this be a new feature?
wanni is offline   Reply With Quote
Old 12-29-2017, 03:29 PM   #13
dlfuller
Member
dlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enoughdlfuller will become famous soon enough
 
Posts: 24
Karma: 566
Join Date: May 2009
Device: none
Thanks for all the feedback.

Quote:
I thought it said the calibre-server was being stopped before starting the MAC client. But on re-reading it would appear the OP leaves calibre-server running while updating the library from the MAC, then when the MAC is closed the server is stopped and started. My guess is that this sequence is used to minimise the server unavailability time.
First, I do make a lot of demands on the server. Simply a single user either accessing the server or using a single calibre Mac app on a single Mac. Not both simultaneously. No multiple Mac apps. Minimizing server availability is not a concern. Multiple Android devices do access the server simultaneously.

Yes, the calibre-server remains running, revisions are made to the library, and then the calibre-server is stopped and restarted.

Quote:
Whether that is an issue depends on how the MAC is mounting the share from the server. It should be mounting it as a networked file system, not a native file system such as HFS+ or an EXT variant.
The calibre library is mounted as a volume on the Mac using SFTP.

So far, this simple approach seems to be working. I have revised and renamed titles without issue. Outside of case sensitivity, I don't know what else to look for, but will still keep an eye out for any glitches.
dlfuller is offline   Reply With Quote
Old 09-02-2018, 10:22 AM   #14
twinkybot
Junior Member
twinkybot began at the beginning.
 
twinkybot's Avatar
 
Posts: 4
Karma: 10
Join Date: Jul 2017
Device: Tolino Vision 2
@kovidgoyal any news on "linking" remote libraries?

I also would like to have a Client which can connect to a (headless) server.
twinkybot is offline   Reply With Quote
Old 09-02-2018, 10:43 AM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,485
Karma: 10205098
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The server web interface can now do all the most important operations -- editing metadata, converting, etc.

As for non-browser client to connect to the server, see https://github.com/kovidgoyal/calibr...e/srv/TODO.rst
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
No external access to Content Server. Help please quoddle Devices 14 05-14-2016 05:35 PM
Run Calibre with Library on an external,How? beckyt Library Management 5 06-15-2015 03:14 PM
Storing library on external drive? csg1964 Library Management 2 03-21-2015 07:06 PM
SPCS: Simple PHP Content Server for Calibre Fmstrat Related Tools 6 05-29-2013 07:26 PM
Request Library with external storage ckole enTourage Archive 9 04-19-2010 11:36 PM


All times are GMT -4. The time now is 05:15 PM.


MobileRead.com is a privately owned, operated and funded community.