Automized Splitting of (TOO) Large Libraries

[Gary_M_Mugford]

I have a large central library. Too large. And no, I won't embarrass myself by detailing how Too Large became Too Large. It's impeding file management processes and backups on that drive because there are a LOT of files, many of which are of the sub-1MB variety. And Windows 7 seems to delight at finding such file-filled folders and immediately slowing down to a speed a sloth would laugh at.

A year or two ago, I started handling all incoming files through a Management version of the library (process, dole out the reports based on a lot of User management desires, save to DropBox folders for each reader and then move all the books from the Management library to the central one. Where they get looked at AT LEAST two, three times a year. I'm being facetious. BUT NOT BY MUCH.

I also have a Junk library and I am getting LOTS of junk, but when I seem to go look for it with the specific goal of moving it out of central, the numbers of found 'new' junk files seems to be minuscule compared to what I saw in my passing perusal of the covers during the processing stage. I move it out, but it's like removing a hair from an elephant. It does reduce the total weight, but ... well you get the drift.

So, I want to break up central. Divide it up into sections that I should have done years ago. Books, Unpublished Papers, Comics, AudioBooks, categories WITHIN those groups. There are a LOT of files to deal out. It would be relatively simple to do a multi-column search (shout out to DaltonST) and then copy-move the found results to the new libs. But I'd like to find a way to automate the whole process. Decide which chunks to go to which new SemiCentral library in what order, push a button and let the computer run all weekend ... or all week as I suspect it would (creating the new sub-libraries, if not found to exist already) .

Now, before anybody asks why I would not just do it manually, the first answer is that complicated manual processes are why we invented computers to prevent feeble humans from screwing up the sequences. AND, you remember the Management intake library? Well, I will have to chop THAT up in a similar way as the final monthly step for the new stuff. Otherwise, I'm back here asking about automation in about ten years.

So, is there something within the plugin system that fills the bill? Within the main core of Calibre? Or is manual my only non-programming solution?

Thanks, GM

[kovidgoyal]

I doubt you'll find an out of the box solution for this. When you start wanting custom automation, you have to be able to program/write scripts. calibre has a command lien tool calibredb to aid with the latter https://manual.calibre-ebook.com/gen...calibredb.html

[Gary_M_Mugford]

Thought that that would be the case. I've opened the calibre databases in my Delphi programming environment a time or two. I'd rather do the chunking from THERE, pre-making the SemiCentral databases ahead of time. Scared to do it, cuz things DO work, however slowly that that makes everything, from the Central Lib itself to Windows ops on that drive. But time is running out for me NOT to do something. Sigh.

I've got the relationships mapped out. The side-benefit is that I can fix up the occasional book/paper title that has information that more properly goes into the author, the series or the comments. IF I can fix THAT all in one fell swoop, while chunking the library, I might have something I can do to manage the monthly Management library too.

Keep doing what you're doing Kovid and I'll attend to my own specialized needs.

Best regards, GM