Can I rsync my library?

[JSWolf]

Sure you can use rsync, but you'd be better off (IMHO) with FreeFileSync.

[Quoth]

Originally Posted by JSWolf

Sure you can use rsync, but you'd be better off (IMHO) with FreeFileSync.

Depends if Linux or Window or Mac what is best.
I use a GUI wrapper on rsync called grsync which allows "dry runs".
Also rsync is built-in on LinuxD and used by Timeshift, the current recommended system back-up restore.

The important things are only one-way and that no Calibre component is running.

[JSWolf]

Originally Posted by Quoth

Depends if Linux or Window or Mac what is best.
I use a GUI wrapper on rsync called grsync which allows "dry runs".
Also rsync is built-in on LinuxD and used by Timeshift, the current recommended system back-up restore.

The important things are only one-way and that no Calibre component is running.

FreeFileSync runs on Windows, MacOS, and Linux and has a GUI.

[ownedbycats]

*sighs*

[theducks]

Originally Posted by ownedbycats

*sighs*

Plus 1

[BetterRed]

Originally Posted by theducks

Plus 1

I'll raise you 3

[haertig]

Originally Posted by JSWolf

Sure you can use rsync, but you'd be better off (IMHO) with FreeFileSync.

I'm willing to bet that FreeFileSync is nothing more than a GUI on top of rsync internals. Not everybody needs a GUI. Some of use run servers that don't even support GUI's.

I have no problem with people using GUI's if they need or want that. Problems can occur if a GUI wrapper project is later abandoned and not readily available for a platform where you need it. If that happens, the people who know how to use the underlying program(s) that the GUI wrappers hide are going to be miles ahead in the game. But still, nothing wrong with using a GUI, especially if you know the underlying program(s) as well. Then you're really set.

I see GUI's as best for (1) Convenience (sometimes), and (2) New users or users that rarely use that functionality and thus benefit with point-n-click, menu drop-downs and other handholding and help to get them through the process.

That being said, I'm certainly using a GUI right now to respond to this post - a web browser. I'm most certainly not invoking HTTP links manually using wget or curl or something like that. Web browsers are "convenient" like that.

[ownedbycats]

There's also some shadiness of FreeFileSync possibly violating GPL, which is why I'm very reluctant to recommend it.

[JSWolf]

Originally Posted by ownedbycats

There's also some shadiness of FreeFileSync possibly violating GPL, which is why I'm very reluctant to recommend it.

Are you sure it's still that way as that post is from 2016 and FFS is now up to version 13.x?

[ownedbycats]

There's a thread on the FFS site from this year.

[haertig]

Originally Posted by ownedbycats

There's also some shadiness of FreeFileSync possibly violating GPL, which is why I'm very reluctant to recommend it.

This sounds vaguely reminiscent of the DeDRM vs. Epubdor conversation, although that conversation is not about licensing. But it's similar in that Person B is possibly taking Person A's code and using in in a way not desired by Person A.

[Quoth]

Originally Posted by ownedbycats

*sighs*

Originally Posted by theducks

Plus 1

Originally Posted by BetterRed

I'll raise you 3

Yes, me also.

[Quoth]

Originally Posted by haertig

I'm willing to bet that FreeFileSync is nothing more than a GUI on top of rsync internals. Not everybody needs a GUI. Some of use run servers that don't even support GUI's.

I have no problem with people using GUI's if they need or want that. Problems can occur if a GUI wrapper project is later abandoned and not readily available for a platform where you need it. If that happens, the people who know how to use the underlying program(s) that the GUI wrappers hide are going to be miles ahead in the game. But still, nothing wrong with using a GUI, especially if you know the underlying program(s) as well. Then you're really set.

I see GUI's as best for (1) Convenience (sometimes), and (2) New users or users that rarely use that functionality and thus benefit with point-n-click, menu drop-downs and other handholding and help to get them through the process.

That being said, I'm certainly using a GUI right now to respond to this post - a web browser. I'm most certainly not invoking HTTP links manually using wget or curl or something like that. Web browsers are "convenient" like that.

Yes, I've used rsync directly, but grsync is widely supported/distributed and is a very handy GUI for rsync. So there is less to mess up. I can't think why anyone on Linux would want some other solution.

Similarly I use Filezilla on Linux, though the functionality of SFTP / FTP is available without it.

[Cactus Chef]

So, went back and did it again (because I bought a couple books in the past few days and needed to back them up), and, yeah, I had "--ignore-existing" on in the rsync arguments, and that's why the metadata file wasn't coming over. I copied the metadata.db and metata.db backup files manually, and all is good now.

That makes me even more confused, though, as to why I had so many books that rsync needed to re-copy when I did my last sync. I did a full backup and restore in August 2023 (which included full library and config files). When I did the first rsync a couple weeks ago, a bunch of books that were already there were picked up by rsync again. Did Calibre make some change between versions that renamed some of the library folders on the filesystem such that rsync saw them as new files and needed to copy them? That's the only thing I can think of that would explain it.

Otherwise, I probably need to do one more rsync and turn off --ignore-existing just to make sure I'm truly getting everything. If I'm remembering my rsync-fu, even if it flags my entire library as needing to copy, it should basically skip the files and transfer 0/0 bytes on each one when it determines that the hashes between the source and destination of each EPUB are the same, right?

[JSWolf]

Originally Posted by Cactus Chef

So, went back and did it again (because I bought a couple books in the past few days and needed to back them up), and, yeah, I had "--ignore-existing" on in the rsync arguments, and that's why the metadata file wasn't coming over. I copied the metadata.db and metata.db backup files manually, and all is good now.

That makes me even more confused, though, as to why I had so many books that rsync needed to re-copy when I did my last sync. I did a full backup and restore in August 2023 (which included full library and config files). When I did the first rsync a couple weeks ago, a bunch of books that were already there were picked up by rsync again. Did Calibre make some change between versions that renamed some of the library folders on the filesystem such that rsync saw them as new files and needed to copy them? That's the only thing I can think of that would explain it.

Otherwise, I probably need to do one more rsync and turn off --ignore-existing just to make sure I'm truly getting everything. If I'm remembering my rsync-fu, even if it flags my entire library as needing to copy, it should basically skip the files and transfer 0/0 bytes on each one when it determines that the hashes between the source and destination of each EPUB are the same, right?

You need to not tell rsync to ignore existing. What you did with the manual copying could still have a library that's not updated. Just let any new/updated files be copied. If you don't, then don't bother to backup anything as it's a waste of time.