Sync Libraries

[BetterRed]

@Tanjamuse - Then I would suggest you use Free File Synch, I don't use it myself because when I selected a file sync program FFS did not have the facilities I need, they're obscure file system issues that you wouldn't want to know about. The product I use, GoodSynch, is not free. For your purposes FFS will be more than good enough - probably better because its not full of stuff you don't need.

It would be better if someone who actually used FFS in anger chimed in at this point, I'll rattle a couple cages for you.

BR

[itimpi]

I use FreeFileSynch to back up my Calibre library to a USB hard disk. It is easy to use and is one of the fastest sync programs that I have come across.

[copyrite]

Yep, I used to use MS's SyncToy, but found FFS does a speedier job once it does the initial synch. I use "mirror" mode, so it mirrors my Calibre library exactly... deleted files from the hard drive are deleted in the backup... otherwise there's a bunch o' extra files hanging about that can cause confusion.

[AnotherCat]

Originally Posted by Tanjamuse

Would you mind giving me a guide as to how I sync? And does it pick up changes in the metadata or added metadata or custom columns ?

I am assuming that you want a backup of your library only in order to protect against loss of your library through disk failure or similar, and that you are not looking to backup Calibre's own Preferences settings.. I don't know how much of a guide you are wanting so if I go overboard or do not cover enough, apologies.

I have used Free File Sync for many years and it has proved itself to be totally reliable and it has a very good reputation. That said, one has to be aware that error on the user's part can result in loss of data, and it can be a challenge initially for those not particularly computer literate.

My suggestion would be to download Free File Sync ( http://freefilesync.sourceforge.net/ ) and create a test folder with some files copied into it (it could be a copy of your Calibre Library) and play with backing those up to your SD card (or USB drive should you get one) in order to get an understanding of how it works.

To set up FFS you will see a tabular region where on the left you enter files or folders you want backed up or synched (in your case I am assuming that will be just your library folder) from the source drive and on the right the destination drive and folder for each left hand side item.

In each case there is a choice of how one wants the synchronisation to occur (a little gearwheel icon); for a backup one may select "Mirror" as Copyrite suggests which will only write or delete from the left hand items (your hard drive) to the right (your SD card).

However, if you are making a backup only i.e. you will never be manually adding, deleting or editing files on that, for the sake of simplicity and learning until you gain familiarity you can just leave the sync option at the default (the gearwheel icon stays grey) if you wish; ordinarily this syncs both ways, but if nothing is ever going to be changed by you on the backup there will be no changes to be synced back to your hard drive (so the result is you get a "Mirror").

When done the synchronisation setup you have created can be saved.

A problem with backups of files such as books where you may not notice a book is missing from the library for many months, or even years, through accidental deletion, corruption or whatever, is that unless one makes periodic archived backups which are stored forever that one can go back to, ones next backing up will just replicate the library sans the missing book to ones backup and the book is lost.

So, on the top left hand side of the FFS window is a Compare button, this will compare the files on both drives (the hard drive and the SD card) and tabulate all changes that it will make on both drives. It pays to run down this and check to see that the books that FFS says will be deleted in the backup are indeed ones that you have yourself deleted out of your library and so want to be removed forever. When absolutely happy hit the Synchronise button at top right of the FFS window; pretty obviously, don't use or have Calibre doing anything during the above process (Calibre can be open but inactive, however).

Back to Calibre itself, as long as you have selected your library folder (or each library should you have several) this will backup the changes in the metadata for each book and the information in columns (with respect to columns that has been my experience, including user created columns; however I am not familiar enough with the innards of Caibre's coding to know if this is always so, but I have never noticed an exception).

You will also notice that FFS places a small file in both the originating and backup folders in order to keep track from session to session. If you do Library Maintenance in Calibre you will find that while these files show up as invalid authors they should just be ignored and no maintenance action taken.

[BetterRed]

IMO its 'best practice' to close all applications that may be accessing a file that you're backing up with a sync tool and don't restart them until the sync is finished.

For calibre that would include the GUI, the Editor if you use it on files in the library, the Viewer if you keep track of reading position, and any command line tasks that operate on the libraries.

If I used FFS I would not have the issue of the small control files FFS puts into library folders because my libraries (Main, Journals, Media, Prepare and Test) are all within in an outer folder - CalibreData. I've also moved calibre's configuration folder into CalibreData via see CALIBRE_CONFIG_DIRECTORY see ==>> http://manual.calibre-ebook.com/cust...ment-variables. So the FFS control files would be in CalibreData rather than the Library folders themselves.

on the need to check deletions before committing the sync. Absolutely essential if the sync tool doesn't squirrel away a copy of them, mine does (and before images of changes) but I still examine the results of the Compare before I go ahead. I sync all my data to a USB3 external disk every day rarely takes more than 5-10 minutes.

BR

[judyni]

The word "backup" is very general and there are a myriad of choices to make. I personally do mirror backups (except my cloud backup) (and in some case have my tool also keep a copy of deleted files which I can delete manually later). I never ever have backups run automatically. Should you make a mistake and delete files by accident and are making a mirror copy, an automatic backup would delete those things from your backup and you end up with nothing (except maybe something in the recycle bin). I do backups manually when I've finished with some work, at least once a day on projects I'm working on.

But I want mirror because I want an exact copy of my primary directories in case of disk failure. And yes, I have had disk failures.

My tool does not show deletions before it runs. That is the one lacking feature with what I use and I keep thinking I should get a tool that does give a list first. On the other hand, would I have the patience to wait for the list... I often queue up several backups and then go to bed.

[AnotherCat]

BR

Yes, I agree it is best practice to shut down applications before syncing in both direction and sometimes when backing up their files. (EDIT: for the sake of clarity I am only talking about an applications own product files, such as Calibre's libraries, so nothing to do with the operating system, for example).

But is not a problem, as far as I can see, with Calibre and FFS when the purpose is backing up as if a file is in use at the time of syncing it FFS gives the opportunity to leave it out, or to retry after one shuts the offending application down or waits for it to finish (as one could in Calibre's case as it does not permanently lock library files throughout a session, unlike, say, as MSOutlook does with its .pst files).

I, personally, would not rely on Calibre maintaining database integrity if syncing libraries in both directions (so not a simple backup) over multiple PCs if the synced libraries are to be edited on multiple computers, and such multiple PC editing is the only reason that comes to my mind for wanting to sync. in both directions between PCs in the first place. So, for myself, the question of shutting Calibre down while doing such a sync is not something I have ever had to contemplate .

Yes, if one has all the libraries under a parent folder and then syncs that parent folder, FFSs own tracking files are invisible to Calibre (and hence to its Library Maintenance tool).

For myself, while I do have all the libraries under single a parent folder I have each library folder itself stacked separately within in a single FFS job, the decision making behind that being driven only by my desire that if something gets screwed up during a sync it will be likely limited to one library, not all. Entirely a personal decision and perhaps ultra conservative.

[cager]

I keep a copy synchronised on a USB stick and it works fine for me. I use the free MS program Syntoy 2.1.

I also loaded calibre-portable onto the USB stick so I can use the library on another computer without calibre installed.