Splitting Calibre library based on content type

[haertig]

I currently have a Calibre library with several thousand books. I store each book in both azw3 and epub format. Yes, I know this is unnecessary and redundant, I do it on purpose anyway - this is not my question. Some of the books I have in audiobook format as well. I have these in Calibre stored as zip files (they contain a sequence of MP3 files and cover.jpg).

I now want to split the audiobooks out to their own separate Calibre library, leaving the azw3 and epub versions of the books in the original Calibre library.

I want to know if my plan to do this is good, or if there is a better way. FWIW, there are about 250 books in my current library that have an audiobook version, out of many thousand total books. The zip files for these 250 books take up a little over 80Gb.

My plan is to:

1. Create a new, empty library named "Audiobooks" from within Calibre
2. Select and COPY all entries in the original library that contain an audiobook version (easy, because they are currently tagged with "audiobook") to the NEW library
3. Go to the NEW library (after the COPY is complete), select every entry, and delete all files of AZW3 and EPUB format
4. Go to the ORIGINAL library, select all entries with the "audiobook" tag, and delete all files of format ZIP
5. I may or may not delete the audiobook tags, haven't decided yet, because that might be a help in knowing that I have an audiobook (albeit in a different library). The downside is that this would "break relational integrity" in database terms, because I would have to remember to do something in two separate libraries as an ongoing step when adding new audiobooks, although that would not be a catastrophic error to make should I fail to keep the tags in sync.

The above should keep all existing metadata and book covers in place in both ORIGINAL and NEW libraries (I have customized this data heavily after initially populating it from Calibres "Download metadata").

The reason why I want to split out the audiobooks is for ease/convenience of copying or backing up my Calibre database(s) to multiple different locations (offsite). The breakdown of my current library disk usage is:

Total size    98Gb
zip           81.3Gb
epub          7.2Gb
azw3          8.8Gb
overhead      1Gb

Almost 100Gb is kind of a lot of data to be slinging around on a semi-frequent basis. Since I rarely listen to audiobooks, I would not be heartbroken if I lost those, so I would probably be less diligent about backing them up to multiple outside locations. Maybe just one location, and call it good.

Is there a better way to do what I'm wanting? Additionally, are there any comments on why I might NOT want to do what I'm wanting? I realize that storage space is cheap, and 100Gb is nothing these days. Also, after initially populating a remote device, then it's quick to update it. I use rsync for that, not a full Calibre export/import. I do make sure that Calibre is not running, and nothing is touching the library on either end when an rsync is in process. I know not to run Calibre over a network to it's library as well. My goal is not to run Calibre over a network, only to backup to remote locations over a network (or to a microSD/HDD/SSD that I hand carry to somewhere offsite). Currently I backup Calibre to a separate computer via rsync, but that computer is onsite. I want to up this backup strategy to include offsite as well. It is the hand carrying of hardware offsite that has the penalty of copying 100Gb of data. I would use new media for each trip, which knocks out the efficiency of rsync and results in a full data copy each time.

Thanks in advance for any suggestions to improve (or criticisms) of my plan.

[ownedbycats]

The main downside I can see to this is that if you want to change the tags/description/whatever on your main copy, it'll be kind of a pain to propagate that to your audiobook copy. Depending on how much you curate your metadata, this may or may not be an issue.

[theducks]

Reverse you plan logic.
Calibre can Copy to Library (delete...) Book Formats, leaving behind the extra stuff (Audio)

But other than a small amount of space in the DB, there is not much of a penalty.

Free file sync only transfer changed (status) files, so again the time to scan for needed copy penalty is low

[haertig]

Good points. Thanks.

Since I really only care about this for making remote copies (backups), I could configure the rsync to just not copy zip files. But that would copy Calibres metadata.db file to the remote, and it would still think that the zip files were there, when they were not. I don't know how Calibre would react to this (if a copy were every started up on the remote data). I can't imagine Calibre would be terribly happy about some of it's previously known user library files just up and disappearing. But maybe it deals with that in a sane manner. In theory, couldn't you simply delete the metadata.db file and have Calibre recreate it, or some usable semblance of it, based on all the individual directory opf files? I don't know the exact procedure to do this, of if it's considered only an emergency move, not really for routine use. In the case where Calibre recreates it's metadata.db file, does ALL the original metadata get resurrected from the opf files? Or does some metadata not make it? If some things don't make it, does anybody know, specifically, what those "some things" are?

[ownedbycats]

Restore database will rebuild it from the OPF files. But thats kind of an extreme measure, and some plugins keep info in the metadata.db which might get lost if you delete the file (unless it's stored somewhere else, I never checked).

Could you get around the database inconsistency with the data folders? That's not stored in the database, so if you can set rsync to ignore zip files in the data folders that could work without getting the database out-of-sync.

[kovidgoyal]

having missing format files is not a big deal, calibre will still generally function fine, unless you try to actually do something with a zip file, in which case you will get an error. The main wrong thing will be the value in the size column. This should be easy for you to verify by making a local copy of the calibre library and deleting all zip files from it and running with it and doing your usual tasks and see if there are any issues.

[Georgie W]

Hi - I have a related question. Like @Haertig, I have typically 2-3 formats for each of my 6300 Calibre books. I want to copy only the mobi format to my Google Drive. But I can't work out how to isolate the mobi files in bulk. "Send Specific Format to" is sort of what I want, only I can't see how to make Google Drive into a Device. Is there a way of doing what I want? Thanks.

[BetterRed]

Originally Posted by haertig

. . .

Thanks in advance for any suggestions to improve (or criticisms) of my plan.

Rather than critique your plan I'll tell how I manage audio editions.

I put the audio files (i.e. the MP3s) into an audio folder within the book's data folder, and a playlist file into the book folder that references them.

I use GoodSynch (similar to rsynch and Free file synch) to do mirrored backups of all my data, the daily backup is to local storage, the weekly is to remote offsite storage. In 12 years of usage I have never had to recover or rebuild an entire calibre library. On occasions I have needed to retrieve individual books or formats from local backup because of my own blunders.

BR

[DNSB]

Originally Posted by Georgie W

Hi - I have a related question. Like @Haertig, I have typically 2-3 formats for each of my 6300 Calibre books. I want to copy only the mobi format to my Google Drive. But I can't work out how to isolate the mobi files in bulk. "Send Specific Format to" is sort of what I want, only I can't see how to make Google Drive into a Device. Is there a way of doing what I want? Thanks.

You might try selecting on the books in your library, right clicking, select Save to Disk and then save Single Format to disk. Save to disk and then move those files to Google Drive. You could save to a directory that is being automatically synced to Google Drive but I prefer to avoid automatic syncing.

Personally, I would save ePub or azw3 not mobi since it is the least capable format of that trio.

[theducks]

Originally Posted by DNSB

Y

Personally, I would save ePub or azw3 not mobi since it is the least capable format of that trio.

Seconded. Mobi is an older format with less capability (thus all the work arounds found in the books HTML code)

[Georgie W]

Originally Posted by DNSB

You might try selecting on the books in your library, right clicking, select Save to Disk and then save Single Format to disk. Save to disk and then move those files to Google Drive. You could save to a directory that is being automatically synced to Google Drive but I prefer to avoid automatic syncing.

Duh! Yes, that would work. I should have thought of that. Thanks.

[Georgie W]

Originally Posted by theducks

Seconded. Mobi is an older format with less capability (thus all the work arounds found in the books HTML code)

I don't disagree, but since I have an older (jailbroken) Kindle that never leaves airplane mode, it's the mobi format I need to back up.

[theducks]

A Jail Broke K4? And it only does Mobi now?
Another reason mine (K4nt) will stay stock (I have not seen a FW update for it in years, anyway)