Library crash - unable to rebuild

[darryl]

I can vouch for BTRFS snapshots. Easy and versatile. Regular snapshots make recovery from a problem such as set out in this thread quick and trivial. But remember, snapshots do not replace the need for a full backup on different media preferably kept at a different location.

[eschwartz]

darryl, you use btrfs?

I haven't done that, it doesn't sound stable enough for me. And I have heard at least one ominous report about pacman (ArchLinux) not playing well with free space accounting...

[darryl]

Originally Posted by eschwartz

darryl, you use btrfs?

I haven't done that, it doesn't sound stable enough for me. And I have heard at least one ominous report about pacman (ArchLinux) not playing well with free space accounting...

I've been using btrfs on my Notebook (running Sabayon) for at least 8 months now and have never had a problem with it. I can't say the same of my experiences years ago with Reiserfs or XFS. It was a bit of a pain to get the sub-volumes set-up the way I wanted them, but well worth the effort. No problems so far even with encrypted root and home partitions. I like Arch and did use it for a while before settling on Sabayon. However, I can't vouch for how Pacman works with this filesystem. I have had no problems with either Entropy or Portage on Sabayon. In fact, I have taken to making a snapshot before any major upgrade. I have experimented with booting into these snapshots and can confirm that this does work as advertised. I haven't had an upgrade cause major problems since I have been using btrfs, but if this does happen it is nice to know that I can boot back into the system as it was before the upgrade with a minimum of fuss. And, of course, back again to the post-upgrade system to fix when convenient and after having been able to research the problem. And, of course, you don't have to boot into a snapshot to benefit from it. All the snapshots and their files are accessible as part of your normal file system. So if, for instance, you accidentally delete a file you can simply copy it back from a recent snapshot. It is really that simple!

[cybmole]

Originally Posted by LadyKate

Isn't there a problem with the metadata.db backing up?

if calibre is idle, then there shoudl be no issue synching as the file metadata.db is not being accessed or altered during the sync.

AFAIK, calibre does not do any automated routine maintenance, so as long as there are no jobs running when you are syncing ( which should only take a couple of minutes ) all should be OK ( but see my caveat below) . all or almost all of the library can be rebuilt by re-importing all the e books + covers +their OPF records, if only the .db file is bad ?


I must admit that i have never lost an entire library yet, so I have not stress tested the restore process. ( but I have retrieved individual books and searched within my dropbox backup via the calibre cloud app & that has always worked OK )
I would never sync if calibre was in the middle of some add book or convert book operation, but it's no strain to wt for one of those to complete.
I sync only manually, it's is not on any auto-schedule, and every time I re-sync , metadata.db should get re-synced anyways, so unless I mess up the one that I actually then need for a restore...

but feel free to completely close calibre to be extra safe, I think I said that before.

PS >harryt "As discussed earlier in the thread, backing up "metadata.db" while Calibre is running".... do you class idling ( i.e. no active jobs & no use of GUI ) as running ? ). If sync while idling & minimised to system tray is unsafe, I'd like to understand why that is please.

[eschwartz]

Even when you think calibre isn't doing anything, it may, for example, be dealing with a backlog of metadata.opf files that need to be written after a bulk job of several sorts. And calibre keeps a file lock on the metadata.db, ,which could lead to quarrels between it and other programs.

i.e. why risk it for no good reason?

[Ellachanted]

I haven't had time to change out my hard drive. Unfortunately, cloning a Windows 8 drive tends to be a pain (something to do with how the BIOS is set up) and I haven't had the time. So, I got another error message on the new library. Fortunately, updating Calibre fixed it this time. I guess I know what I am doing this weekend....


Anyway, I just really appreciate being able to come back to this thread before I mess anything up.

So thanks again for all the replies!

[cybmole]

Marcum reflect free will clone the whole windows drive, but a 32 or 64 gb usb drive backup of calibre library only is advised, while all is good,
cheap and quick, buy two and alternate them for extra backup insurance.

[Ellachanted]

I actually had planned to clone the drive prior to updating to 8.1 which I had been putting off because I had heard mixed stuff about it. I've done it before and I forgot to write down the process. I always think this will be the last time I do it. Very optimistic of me LOL!

[cybmole]

i cloned my "upgraded to win 10 " drive with Macrum reflect free. I wanted to move the original drive into a suspension cradle so I cloned it as a safety precaution before doing that, using a sata drive in a USB caddy to clone to. took about 3 hours on USB2, copied about 600GB.
the only side effect I now see, from running reflect, is that I have extra drive letters assigned to 3 -4 small previously hidden partitions on the original dell source drive - i assume these are recovery partitions put there by dell or by win 8. shoudl he harmless so long as I ignore them.

so I did not need to actually install the clone drive, but I now have the original in a sharkoon HD stabiliser in the spare 5.25 bay, where is is definitely quieter. dell's hard drive rail mounts leak hum into the case.
but for calibre libraries, I still advocate, sending a mirrored sync copy to whatever cloud services you can get for free, and/or to USB sticks. If, like me, you tinker a lot with your books, then the more backups the better!
a disconnected USB copy protects against ransomware ever stricking

[phossler]

a disconnected USB copy protects against ransomware ever stricking

Just a 'BTW' -- I use a robocopy vbs script to backup from my desktop PC to another PC in my Homegroup

Ransom-ware looks for drives mapped to letters (e.g. N but doesn't seem to go after UNC drives (e.g. \\DENPC2\HD2\Username\Backup) but robocopy can handle UNC drives

[Ellachanted]

Yeah, those unnamed drives in Windows. Don't ever name them unless you want to crash your OS. Started with Windows 7, they are boot drives that contain hidden OS files. I found out about them when I was trying to set up a dual boot system with XP and 7, as a number of my programs hadn't worked well on Vista and weren't upgraded to 7 yet.

It turns out, last time I changed out the hard drive on this computer, I made a system image after loading the majority of the programs. So I just put the new drive in, ran the recovery disc, and loaded the image. Still takes forever to transfer files, even using a 3.0 USB enclosure for the old drive. But so far so good.

[BetterRed]

Originally Posted by Ellachanted

Still takes forever to transfer files, even using a 3.0 USB enclosure for the old drive. But so far so good.

A USB 3.0 enclosure wont deliver USB 3.0 performance if its not plugged into a USB 3.0 controller

BR

[theducks]

Originally Posted by Ellachanted

Yeah, those unnamed drives in Windows. Don't ever name them unless you want to crash your OS. Started with Windows 7, they are boot drives that contain hidden OS files. I found out about them when I was trying to set up a dual boot system with XP and 7, as a number of my programs hadn't worked well on Vista and weren't upgraded to 7 yet.

It turns out, last time I changed out the hard drive on this computer, I made a system image after loading the majority of the programs. So I just put the new drive in, ran the recovery disc, and loaded the image. Still takes forever to transfer files, even using a 3.0 USB enclosure for the old drive. But so far so good.

Now that you mention it. The upgrade reLABELED my (c Partition
No wonder some mapped drives failed until I remapped .
Dang you M$

[cybmole]

as it played out , that drive went belly up anyway - maybe cos it overheated up high, andway windows crashed told me " a paramater has exceeded its value" - dell code for overheated maybe ? and would not then reboot. So In a mild panic I put in the clone, in the original HD location - it works fine, and does not have any extra weird drive letters either. I planned to hustle dell for a new HD but as they were shut for the weekend I put it in a sata-USB caddy, read the smart data, and then ran a 7 hour full media scan via another PC that had WD HD diagnostic software installed.- after 7 hours teh extended test said that it was all good!
Historic smart data showed a max temp of 51 degrees and i think dell bios steps in at 50 degrees to say that there's a problem. so no more HD in top 5.25 bays,except where I run with PC case side off.
Smart data Value threshold worst
airflow temp 70 45 49
HDA temp 30 0 51


given what you say about " dont ever name them" I wonder if W10 tried to do something helpful, like an auto-defrag of the extra partitions, & kicked of the whole debacle. - guess I'll never know. more likely it was overheat - read/write errors- crash - panic - but hardware OK once it has cooled down again.
anyway I can now assert that Macrium makes a very good clone!
I now am staying with " if it ain't broke don't fix it" so I will use that other drive some place else, sometime. I have system imaged the current drive OK

[theducks]

Cybe
Dell has some very poor consumer Computer 2nd HD configurations
Dimension 4600 piggybacks the 2nd HD on a vertical (flat side to front.) 1st HD.
Almost no breeze on the rear HD. HH <- air
My 2nd HD is in the lowest CD bay (in adapter)