Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 09-05-2015, 07:57 PM   #1
mohataj
Junior Member
mohataj began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Oct 2012
Device: nook simple touch
Recreating my Library on my server takes weeks now

There is my huge library on my server connected via NFS.

OK, I know, don't use a network storage for your calibre library.

It killed the database once or twice a year, recreating with calibre on my macbook took 2-3 days - that's OK and adding new books worked flawless afterwards.

Now I'm at 4% after 3 days with calibre 2.31.0. (3rd run) - this will take months from now on.

Is there any way to recreate the database using calibre-server on my linux server?

So there is a faster access to my ebooks to a local harddisk than gbit ethernet?

Is there a way to connect the local calibre on my macbook to calibre-server to add new books?

Is there a way, like a watch folder, to add books on a headless calibre-server on my linux server?

My current situation is - there is no calibre-server running on my server.
There is my eBooks-folder on my server served by NFS to my macbook.
Calibre on my macbook is the only way to manage and add the ebooks.
Bicbucstriim http://projekte.textmulch.de/bicbucstriim/ as I understand it "reads" the database, is the access point for my iPhone, Android, S60, symbian, maemo, nook, kindle, readr and is working really well to transfer my ebooks.

Last edited by mohataj; 09-05-2015 at 08:28 PM. Reason: more info
mohataj is offline   Reply With Quote
Old 09-05-2015, 10:05 PM   #2
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 18,873
Karma: 17287248
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Astak Pocket Pro, K4NT,Galaxy Tab 2
Investigate the CLI options and run those from a (server) Terminal session


Use you MAC (standalone) to prepare Books for transfer. Cleanup edits, Polish...

Then Export the books and place the clean EPUBs into the Auto Add folder on the server
theducks is offline   Reply With Quote
 
Advertisement
Old 09-05-2015, 11:06 PM   #3
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,318
Karma: 76285381
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
calibre's Auto-add folder on the server only works if you have the calibre GUI running (headless).

I suggest exporting books to a temporary folder on the server (shared over NAS) and using a cron/systemd job backed by inotify to add all books added to that folder, and clean them up afterward.

calibre has many useful CLI tools you probably want `calibredb restore_database` to fix the remote library, and `calibredb add --one-book-per-directory` to add books exported from calibre on your Mac.
eschwartz is offline   Reply With Quote
Old 09-05-2015, 11:14 PM   #4
mohataj
Junior Member
mohataj began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Oct 2012
Device: nook simple touch
OK, calibre-server --with-library=/mnt/3TB/eBooks shows - nothing, there is even no load, is this command indexing my folder?

"calibre's Auto-add folder on the server only works if you have the calibre GUI running (headless)"

this is exactly what I want - how do I do this?
mohataj is offline   Reply With Quote
Old 09-05-2015, 11:23 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,941
Karma: 7303535
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibredb restore_database
kovidgoyal is online now   Reply With Quote
Old 09-05-2015, 11:45 PM   #6
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,318
Karma: 76285381
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by mohataj View Post
OK, calibre-server --with-library=/mnt/3TB/eBooks shows - nothing, there is even no load, is this command indexing my folder?
Well, if you were trying to launch the Content Server, that would totally do it.

But if you were trying to fix the library, following my advice by using the `calibredb restore_database` command, then the best way to start would be to reread my post. And read the manual: http://manual.calibre-ebook.com/gene...calibredb.html

Quote:
"calibre's Auto-add folder on the server only works if you have the calibre GUI running (headless)"

this is exactly what I want - how do I do this?
First you have to make sure your server has all the X11 libraries installed. Then you additionally make sure you have the X Virtual Framebuffer installed, and then you run calibre inside xvfb.
Sit back. Watch contentedly as your server's RAM and cpu get flushed down the toilet because you didn't follow my advice and use the CLI tools instead.


This is exactly what you DON'T want!!! Running GUI programs headless while piped to a dummy (virtual) screen is a weapon of last resort, for programs which have no sane way of being used in a server environment!
calibre, fortunately, has the backend implementation necessary to be used headless, without the GUI.
eschwartz is offline   Reply With Quote
Old 09-06-2015, 12:10 AM   #7
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,318
Karma: 76285381
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by eschwartz View Post
calibre's Auto-add folder on the server only works if you have the calibre GUI running (headless).

I suggest exporting books to a temporary folder on the server (shared over NAS) and using a cron/systemd job backed by inotify to add all books added to that folder, and clean them up afterward.

calibre has many useful CLI tools you probably want `calibredb restore_database` to fix the remote library, and `calibredb add --one-book-per-directory` to add books exported from calibre on your Mac.
What I would suggest doing, again, is cleaning up the books the way you want it on your Mac, then using Save-to-disk to save them in one-book-per-folder layout.
Save them inside a folder on your server, e.g. /tmp/calibre-intake/

Then adding them to your linux server using the CLI as I said above.
Code:
calibredb add -r --one-book-per-directory /tmp/calibre-intake/
rm -rf /path/to/ebook-folder/*
You can automate that in a script, which uses inotify to tell when the /tmp/calibre-intake/ folder changes:
Code:
while inotifywait -e create /tmp/calibre-intake; do
    calibredb add -r --one-book-per-directory /tmp/calibre-intake/
    rm -rf /tmp/calibre-intake/*
done
You can also convert that to a system service (now I think about it, NOT as a cron job). How to do this will depend on what init system your server uses.

Last edited by eschwartz; 09-06-2015 at 12:12 AM.
eschwartz is offline   Reply With Quote
Old 09-12-2015, 05:23 PM   #8
mohataj
Junior Member
mohataj began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Oct 2012
Device: nook simple touch
thanks for your replies, recreating the database by commandline took me 8 days now and it's reliable and nice to see the progress.

Is there any way to use calibre with a write protected metadata.db?

So I have a read only calibre for use with calibre companion.
mohataj is offline   Reply With Quote
Old 09-12-2015, 05:45 PM   #9
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,237
Karma: 901265
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by mohataj View Post
thanks for your replies, recreating the database by commandline took me 8 days now and it's reliable and nice to see the progress.

Is there any way to use calibre with a write protected metadata.db?

So I have a read only calibre for use with calibre companion.
I do not believe so - I am reasonably certain that the main Calibre program fails to run if the database is set read-only.

Calibre server does not require write access.
itimpi is offline   Reply With Quote
Old 09-12-2015, 11:17 PM   #10
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,318
Karma: 76285381
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
calibre-server will only ever write to the database when performing an automatic schema upgrade after a calibre update changes the layout of the database... which happens only rarely.

If you really want to prevent the database from being modified, you could always use the environment variable "CALIBRE_OVERRIDE_DATABASE_PATH" pointed at a temporary copy which can be trashed.
Don't modify any metadata while you do this calibre might lose track of your books!
eschwartz is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sync library between server and laptop cptnemo Library Management 4 05-05-2013 02:33 AM
Recreating library on new computer? jeather Calibre 5 01-31-2013 02:22 PM
central server library Dahaniel Library Management 7 09-03-2011 03:51 AM
Library on NAS server? bous Calibre 5 11-26-2009 05:03 AM
Library takes 5 minutes to open the reader! accipiter Sony Reader 1 02-02-2009 12:46 AM


All times are GMT -4. The time now is 12:21 AM.


MobileRead.com is a privately owned, operated and funded community.