MobileRead Forums - View Single Post - Splitting Calibre library based on content type

haertig · 10-14-2023, 07:17 PM

I currently have a Calibre library with several thousand books. I store each book in both azw3 and epub format. Yes, I know this is unnecessary and redundant, I do it on purpose anyway - this is not my question. Some of the books I have in audiobook format as well. I have these in Calibre stored as zip files (they contain a sequence of MP3 files and cover.jpg).

I now want to split the audiobooks out to their own separate Calibre library, leaving the azw3 and epub versions of the books in the original Calibre library.

I want to know if my plan to do this is good, or if there is a better way. FWIW, there are about 250 books in my current library that have an audiobook version, out of many thousand total books. The zip files for these 250 books take up a little over 80Gb.

My plan is to:

Create a new, empty library named "Audiobooks" from within Calibre
Select and COPY all entries in the original library that contain an audiobook version (easy, because they are currently tagged with "audiobook") to the NEW library
Go to the NEW library (after the COPY is complete), select every entry, and delete all files of AZW3 and EPUB format
Go to the ORIGINAL library, select all entries with the "audiobook" tag, and delete all files of format ZIP
I may or may not delete the audiobook tags, haven't decided yet, because that might be a help in knowing that I have an audiobook (albeit in a different library). The downside is that this would "break relational integrity" in database terms, because I would have to remember to do something in two separate libraries as an ongoing step when adding new audiobooks, although that would not be a catastrophic error to make should I fail to keep the tags in sync.

The above should keep all existing metadata and book covers in place in both ORIGINAL and NEW libraries (I have customized this data heavily after initially populating it from Calibres "Download metadata").

The reason why I want to split out the audiobooks is for ease/convenience of copying or backing up my Calibre database(s) to multiple different locations (offsite). The breakdown of my current library disk usage is:

Code:

Total size    98Gb
zip           81.3Gb
epub          7.2Gb
azw3          8.8Gb
overhead      1Gb

Almost 100Gb is kind of a lot of data to be slinging around on a semi-frequent basis. Since I rarely listen to audiobooks, I would not be heartbroken if I lost those, so I would probably be less diligent about backing them up to multiple outside locations. Maybe just one location, and call it good.

Is there a better way to do what I'm wanting? Additionally, are there any comments on why I might NOT want to do what I'm wanting? I realize that storage space is cheap, and 100Gb is nothing these days. Also, after initially populating a remote device, then it's quick to update it. I use rsync for that, not a full Calibre export/import. I do make sure that Calibre is not running, and nothing is touching the library on either end when an rsync is in process. I know not to run Calibre over a network to it's library as well. My goal is not to run Calibre over a network, only to backup to remote locations over a network (or to a microSD/HDD/SSD that I hand carry to somewhere offsite). Currently I backup Calibre to a separate computer via rsync, but that computer is onsite. I want to up this backup strategy to include offsite as well. It is the hand carrying of hardware offsite that has the penalty of copying 100Gb of data. I would use new media for each trip, which knocks out the efficiency of rsync and results in a full data copy each time.

Thanks in advance for any suggestions to improve (or criticisms) of my plan.

10-14-2023, 07:17 PM	#1
haertig Wizard Posts: 1,899 Karma: 31522252 Join Date: Sep 2017 Device: PW3, Fire HD8 Gen7, Moto G7, Sansa Clip v2, Ruizu X26	Splitting Calibre library based on content type I currently have a Calibre library with several thousand books. I store each book in both azw3 and epub format. Yes, I know this is unnecessary and redundant, I do it on purpose anyway - this is not my question. Some of the books I have in audiobook format as well. I have these in Calibre stored as zip files (they contain a sequence of MP3 files and cover.jpg). I now want to split the audiobooks out to their own separate Calibre library, leaving the azw3 and epub versions of the books in the original Calibre library. I want to know if my plan to do this is good, or if there is a better way. FWIW, there are about 250 books in my current library that have an audiobook version, out of many thousand total books. The zip files for these 250 books take up a little over 80Gb. My plan is to: Create a new, empty library named "Audiobooks" from within Calibre Select and COPY all entries in the original library that contain an audiobook version (easy, because they are currently tagged with "audiobook") to the NEW library Go to the NEW library (after the COPY is complete), select every entry, and delete all files of AZW3 and EPUB format Go to the ORIGINAL library, select all entries with the "audiobook" tag, and delete all files of format ZIP I may or may not delete the audiobook tags, haven't decided yet, because that might be a help in knowing that I have an audiobook (albeit in a different library). The downside is that this would "break relational integrity" in database terms, because I would have to remember to do something in two separate libraries as an ongoing step when adding new audiobooks, although that would not be a catastrophic error to make should I fail to keep the tags in sync. The above should keep all existing metadata and book covers in place in both ORIGINAL and NEW libraries (I have customized this data heavily after initially populating it from Calibres "Download metadata"). The reason why I want to split out the audiobooks is for ease/convenience of copying or backing up my Calibre database(s) to multiple different locations (offsite). The breakdown of my current library disk usage is: Code: Total size 98Gb zip 81.3Gb epub 7.2Gb azw3 8.8Gb overhead 1Gb Almost 100Gb is kind of a lot of data to be slinging around on a semi-frequent basis. Since I rarely listen to audiobooks, I would not be heartbroken if I lost those, so I would probably be less diligent about backing them up to multiple outside locations. Maybe just one location, and call it good. Is there a better way to do what I'm wanting? Additionally, are there any comments on why I might NOT want to do what I'm wanting? I realize that storage space is cheap, and 100Gb is nothing these days. Also, after initially populating a remote device, then it's quick to update it. I use rsync for that, not a full Calibre export/import. I do make sure that Calibre is not running, and nothing is touching the library on either end when an rsync is in process. I know not to run Calibre over a network to it's library as well. My goal is not to run Calibre over a network, only to backup to remote locations over a network (or to a microSD/HDD/SSD that I hand carry to somewhere offsite). Currently I backup Calibre to a separate computer via rsync, but that computer is onsite. I want to up this backup strategy to include offsite as well. It is the hand carrying of hardware offsite that has the penalty of copying 100Gb of data. I would use new media for each trip, which knocks out the efficiency of rsync and results in a full data copy each time. Thanks in advance for any suggestions to improve (or criticisms) of my plan.