Looking for Advice on system configuration

brawny · 12-17-2015, 06:32 PM

Greetings,

This is my first post here. I've done some research in preparation for setting up my library system by reading posts here and reading the calibre user manual, but really I'm looking for advice more than anything.

I've been playing with calibre for a bit now and have it working on two systems. Here's my environment:

Xubuntu server, used for media storage. calibre installed and working with local library.
Old laptop (Windows 7). calibre installed and working with local copy of library from Xubuntu server.
Kobo Aura H2O (recent birthday gift for my wife)
iPads (2) with Marvin installed
I expect I'll be receiving a Kobo for Christmas, and if not, then likely at a Boxing Day sale... :-)

My end state is hopefully:

ebook library that is visible from the internet, so that we can download books to our devices from our personal calibre library when away from home.
Access to the library from my laptop in order to load books directly (I believe the Kobo software isn't available for Linux)
An automated way of keeping the two copies of the library in sync. I have a NAS that I use to back up my media on the server nightly. I just need to find a way of keeping the laptop library in sync with the server.

So two issues to resolve:

How to keep laptop library synchronized with server library. I think FreeFileSync is probably a good candidate, but wondering if there are other / better methods.
How make server library visible on the internet. Options that I'm aware of is to expose a port on my server to the internet and use the calibre interface, or set up calibre2opds.

Looking for advice or feedback.

Thanks in advance,

Brawny

eschwartz · 12-17-2015, 06:53 PM

You could mirror your library onto the server, but I would be hesitant to do that across OSes.

Because... Windows-the-OS is case-insensitive (although case-preserving).

You may find that linux cannot find the files, because Windows will happily store "Author 1/Book 1" as "author 1/book 1" without telling calibre.

brawny · 12-17-2015, 08:19 PM

Thank you eschwartz for pointing that out.

Not being familiar with FreeFileSync, I wonder if I could synchronize the other way - keeping my linux version of the library as the master, and bringing updates to my laptop to get rid of any case sensitivity issues?

That would mean that any editing of the library would have to be done on the server, which is a bit awkward. Any ebooks downloaded to the laptop via the Kobo software (or Amazon, etc) might also be an issue. They'd need to be added on the server library rather than the local laptop copy.

Perhaps I should be looking at running some sort of graphical access to the server using X, so that I could run the server version of calibre remotely on the laptop and get rid of the laptop copy of everything? Still would have to figure out the best way to get purchased Kobo / Amazon books onto the server. And it would limit my ability to load books onto the Kobo to wifi only without the local copy of the library and access via usb.

I understand the limitation around not accessing a library via a networked drive. It's definitely an unfortunately limitation in my situation. It would certainly simplify things!

theducks · 12-17-2015, 08:29 PM

Some versions of VNC have a (windows) file transfer feature.

I am with Eschwartz. Don't mix OS

Why not VNC into your 'server' (assumes it is NOT Headless) and do Calibre there and also run the Calibre server / Wireless device connection (for CC Users) on that box.

All Library admin and GUI would be done via the 'remote' (Any PC with a VNC Client)
You would drop the files into the Auto-Add folder (VNC file transfer OR FTP...)
then tune the metadata

eschwartz · 12-17-2015, 09:40 PM

You could also manage books on your computer, and sync it up to the server, but finish by running

Code:

calibredb restore_database --really-do-it

to have calibre rebuild the database and fix all errors of that nature.

Or export "cleaned" books, with their proper metadata, and import the cleaned copy on the server via the CLI.

Or get Ubuntu on that laptop.

Aside: my favorite way of transferring data to another computer is the old-fashioned way, via rsync.
Heck, there are GUIs for rsync.

theducks · 12-17-2015, 09:43 PM

Do Mint, it is based upon the Ubuntu distro, but WAY cleaner

eschwartz · 12-17-2015, 10:17 PM

I am a bit disappointed actually, that Mint is basing every release off of Ubuntu 14.04 LTS which is incredibly out-of-date, rather than using newer releases as the base for non-LTS releases.

But hey, *buntu users should be used to out-of-date anyway.

Get Arch/Gentoo if you want up-to-date non-security-related software. (And it will be really clean.)

So I guess Mint is the best choice for *buntu users.

yonkyunior · 12-18-2015, 03:49 PM

Maintaining large library requiring a lot of time..
i found the convention by the bootstraps would be great use for long time..
here

How I manage eBooks with Calibre by the bootstraps
https://www.mobileread.com/forums/sho...d.php?t=185452

read it slowly

i not yet using Quarantine&Scrub or CALM plugin, could be useful if you can understand how to use it,

yonkyunior · 12-18-2015, 04:01 PM

Quote:

Originally Posted by yonkyunior

Maintaining large library requiring a lot of time..
i found the convention by the bootstraps would be great use for long time..
here

How I manage eBooks with Calibre by the bootstraps
https://www.mobileread.com/forums/sho...d.php?t=185452

read it slowly

i not yet using Quarantine&Scrub or CALM plugin, could be useful if you can understand how to use it,

Dammmmnn... wrong answer..wrong post..sorry

but i read on another post that someone using Duplicati to keep it sync, but i dont understand how. Maybe you should check it

brawny · 12-25-2015, 08:16 AM

Thanks for all the feedback and suggestions, everyone. I thought I'd provide an update on how I've decided to proceed.

For now, I've decided to start small, and focus on having a local library on my laptop. While having the library available on my server in case I need a book while I'm not at home sounds like a great idea, in practice, its probably not something that will be used very often, and certainly is a 'nice to have' rather than a 'need to have'.

I've started with a duplicate copy of the library from the server, and have started to clean it up using calibre running locally on my laptop against a local copy of the server. I have FreeFileSync running on my laptop to back the local database up to a NAS I have.

My plan is to take the existing library of just under 3000 books that I've inherited, and clean up the metadata, get rid of duplicates, etc. Once I have the library in a shape that I'm happy with, I'll revisit trying to make the library visible on the internet either using a second copy of calibre on the server, or by using calibre2opds.

Thanks to everyone for your advice and suggestions!

Brawny

eschwartz · 12-25-2015, 03:34 PM

Sounds reasonable.

Though I think linux on the laptop is even more reasonable.

Do keep an eye on the future for calibre v3.x -- with revamped server; hopefully may handle all this far more gracefully. Kovid has said in the past that the server will eventually be rewritten with multi-user/read-write capabilities.

brawny · 01-05-2016, 10:17 AM

Thanks eshwartz!

I may eventually have to put linux on my laptop, as it gets older and can no longer keep up with bloated Micro$oft OSes. I have some os specific software that I need, otherwise I would have switched long ago.

Looking forward to calibre v3.0 Multi-user would be fantastic, if that's in the cards.

Thanks again,
Brawny

12-17-2015, 06:32 PM	#1
brawny Junior Member Posts: 4 Karma: 10 Join Date: Dec 2015 Device: Kobo Aura H2O	Looking for Advice on system configuration Greetings, This is my first post here. I've done some research in preparation for setting up my library system by reading posts here and reading the calibre user manual, but really I'm looking for advice more than anything. I've been playing with calibre for a bit now and have it working on two systems. Here's my environment: Xubuntu server, used for media storage. calibre installed and working with local library. Old laptop (Windows 7). calibre installed and working with local copy of library from Xubuntu server. Kobo Aura H2O (recent birthday gift for my wife) iPads (2) with Marvin installed I expect I'll be receiving a Kobo for Christmas, and if not, then likely at a Boxing Day sale... :-) My end state is hopefully: ebook library that is visible from the internet, so that we can download books to our devices from our personal calibre library when away from home. Access to the library from my laptop in order to load books directly (I believe the Kobo software isn't available for Linux) An automated way of keeping the two copies of the library in sync. I have a NAS that I use to back up my media on the server nightly. I just need to find a way of keeping the laptop library in sync with the server. So two issues to resolve: How to keep laptop library synchronized with server library. I think FreeFileSync is probably a good candidate, but wondering if there are other / better methods. How make server library visible on the internet. Options that I'm aware of is to expose a port on my server to the internet and use the calibre interface, or set up calibre2opds. Looking for advice or feedback. Thanks in advance, Brawny

12-17-2015, 06:53 PM	#2
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	You could mirror your library onto the server, but I would be hesitant to do that across OSes. Because... Windows-the-OS is case-insensitive (although case-preserving). You may find that linux cannot find the files, because Windows will happily store "Author 1/Book 1" as "author 1/book 1" without telling calibre. Last edited by eschwartz; 12-17-2015 at 09:34 PM.

12-17-2015, 09:40 PM	#5
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	You could also manage books on your computer, and sync it up to the server, but finish by running Code: calibredb restore_database --really-do-it to have calibre rebuild the database and fix all errors of that nature. Or export "cleaned" books, with their proper metadata, and import the cleaned copy on the server via the CLI. Or get Ubuntu on that laptop. Aside: my favorite way of transferring data to another computer is the old-fashioned way, via rsync. Heck, there are GUIs for rsync.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
360 Plus Configuration of fbreader180	unkilbeeg	PocketBook	2	10-24-2012 07:12 PM
903 UMTS configuration	juggle	PocketBook	0	05-04-2011 09:07 AM
Calibre configuration help!	Iganius	Calibre	5	06-25-2010 11:10 AM
iPad configuration	Wordcrasher	Apple Devices	14	06-21-2010 06:59 AM
how to clean more disk space in root file system to upgrade system	chinaet	iRex	1	12-18-2006 03:54 PM

12-17-2015, 08:19 PM	#3
brawny Junior Member Posts: 4 Karma: 10 Join Date: Dec 2015 Device: Kobo Aura H2O	Thank you eschwartz for pointing that out. Not being familiar with FreeFileSync, I wonder if I could synchronize the other way - keeping my linux version of the library as the master, and bringing updates to my laptop to get rid of any case sensitivity issues? That would mean that any editing of the library would have to be done on the server, which is a bit awkward. Any ebooks downloaded to the laptop via the Kobo software (or Amazon, etc) might also be an issue. They'd need to be added on the server library rather than the local laptop copy. Perhaps I should be looking at running some sort of graphical access to the server using X, so that I could run the server version of calibre remotely on the laptop and get rid of the laptop copy of everything? Still would have to figure out the best way to get purchased Kobo / Amazon books onto the server. And it would limit my ability to load books onto the Kobo to wifi only without the local copy of the library and access via usb. I understand the limitation around not accessing a library via a networked drive. It's definitely an unfortunately limitation in my situation. It would certainly simplify things!

12-17-2015, 08:29 PM	#4
theducks Well trained by Cats Posts: 31,054 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Some versions of VNC have a (windows) file transfer feature. I am with Eschwartz. Don't mix OS Why not VNC into your 'server' (assumes it is NOT Headless) and do Calibre there and also run the Calibre server / Wireless device connection (for CC Users) on that box. All Library admin and GUI would be done via the 'remote' (Any PC with a VNC Client) You would drop the files into the Auto-Add folder (VNC file transfer OR FTP...) then tune the metadata

12-17-2015, 09:43 PM	#6
theducks Well trained by Cats Posts: 31,054 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Do Mint, it is based upon the Ubuntu distro, but WAY cleaner

12-17-2015, 10:17 PM	#7
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	I am a bit disappointed actually, that Mint is basing every release off of Ubuntu 14.04 LTS which is incredibly out-of-date, rather than using newer releases as the base for non-LTS releases. But hey, buntu users should be used to out-of-date anyway. Get Arch/Gentoo if you want up-to-date non-security-related software. (And it will be really clean.) So I guess Mint is the best choice for buntu users.

12-18-2015, 03:49 PM	#8
yonkyunior Cultivator Posts: 94 Karma: 216 Join Date: Feb 2015 Device: PRST2	Maintaining large library requiring a lot of time.. i found the convention by the bootstraps would be great use for long time.. here How I manage eBooks with Calibre by the bootstraps https://www.mobileread.com/forums/sho...d.php?t=185452 read it slowly i not yet using Quarantine&Scrub or CALM plugin, could be useful if you can understand how to use it,

12-25-2015, 08:16 AM	#10
brawny Junior Member Posts: 4 Karma: 10 Join Date: Dec 2015 Device: Kobo Aura H2O	Thanks for all the feedback and suggestions, everyone. I thought I'd provide an update on how I've decided to proceed. For now, I've decided to start small, and focus on having a local library on my laptop. While having the library available on my server in case I need a book while I'm not at home sounds like a great idea, in practice, its probably not something that will be used very often, and certainly is a 'nice to have' rather than a 'need to have'. I've started with a duplicate copy of the library from the server, and have started to clean it up using calibre running locally on my laptop against a local copy of the server. I have FreeFileSync running on my laptop to back the local database up to a NAS I have. My plan is to take the existing library of just under 3000 books that I've inherited, and clean up the metadata, get rid of duplicates, etc. Once I have the library in a shape that I'm happy with, I'll revisit trying to make the library visible on the internet either using a second copy of calibre on the server, or by using calibre2opds. Thanks to everyone for your advice and suggestions! Brawny

12-25-2015, 03:34 PM	#11
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Sounds reasonable. Though I think linux on the laptop is even more reasonable. Do keep an eye on the future for calibre v3.x -- with revamped server; hopefully may handle all this far more gracefully. Kovid has said in the past that the server will eventually be rewritten with multi-user/read-write capabilities.

01-05-2016, 10:17 AM	#12
brawny Junior Member Posts: 4 Karma: 10 Join Date: Dec 2015 Device: Kobo Aura H2O	Thanks eshwartz! I may eventually have to put linux on my laptop, as it gets older and can no longer keep up with bloated Micro$oft OSes. I have some os specific software that I need, otherwise I would have switched long ago. Looking forward to calibre v3.0 Multi-user would be fantastic, if that's in the cards. Thanks again, Brawny

Advert

Advert