MobileRead Forums - View Single Post - Automized Splitting of (TOO) Large Libraries

Gary_M_Mugford · 11-20-2017, 01:31 AM

I have a large central library. Too large. And no, I won't embarrass myself by detailing how Too Large became Too Large. It's impeding file management processes and backups on that drive because there are a LOT of files, many of which are of the sub-1MB variety. And Windows 7 seems to delight at finding such file-filled folders and immediately slowing down to a speed a sloth would laugh at.

A year or two ago, I started handling all incoming files through a Management version of the library (process, dole out the reports based on a lot of User management desires, save to DropBox folders for each reader and then move all the books from the Management library to the central one. Where they get looked at AT LEAST two, three times a year. I'm being facetious. BUT NOT BY MUCH.

I also have a Junk library and I am getting LOTS of junk, but when I seem to go look for it with the specific goal of moving it out of central, the numbers of found 'new' junk files seems to be minuscule compared to what I saw in my passing perusal of the covers during the processing stage. I move it out, but it's like removing a hair from an elephant. It does reduce the total weight, but ... well you get the drift.

So, I want to break up central. Divide it up into sections that I should have done years ago. Books, Unpublished Papers, Comics, AudioBooks, categories WITHIN those groups. There are a LOT of files to deal out. It would be relatively simple to do a multi-column search (shout out to DaltonST) and then copy-move the found results to the new libs. But I'd like to find a way to automate the whole process. Decide which chunks to go to which new SemiCentral library in what order, push a button and let the computer run all weekend ... or all week as I suspect it would (creating the new sub-libraries, if not found to exist already) .

Now, before anybody asks why I would not just do it manually, the first answer is that complicated manual processes are why we invented computers to prevent feeble humans from screwing up the sequences. AND, you remember the Management intake library? Well, I will have to chop THAT up in a similar way as the final monthly step for the new stuff. Otherwise, I'm back here asking about automation in about ten years.

So, is there something within the plugin system that fills the bill? Within the main core of Calibre? Or is manual my only non-programming solution?

Thanks, GM

11-20-2017, 01:31 AM	#1
Gary_M_Mugford Groupie Posts: 180 Karma: 299 Join Date: Jul 2010 Location: Brampton ON Device: Kobo, Kindle3	Automized Splitting of (TOO) Large Libraries I have a large central library. Too large. And no, I won't embarrass myself by detailing how Too Large became Too Large. It's impeding file management processes and backups on that drive because there are a LOT of files, many of which are of the sub-1MB variety. And Windows 7 seems to delight at finding such file-filled folders and immediately slowing down to a speed a sloth would laugh at. A year or two ago, I started handling all incoming files through a Management version of the library (process, dole out the reports based on a lot of User management desires, save to DropBox folders for each reader and then move all the books from the Management library to the central one. Where they get looked at AT LEAST two, three times a year. I'm being facetious. BUT NOT BY MUCH. I also have a Junk library and I am getting LOTS of junk, but when I seem to go look for it with the specific goal of moving it out of central, the numbers of found 'new' junk files seems to be minuscule compared to what I saw in my passing perusal of the covers during the processing stage. I move it out, but it's like removing a hair from an elephant. It does reduce the total weight, but ... well you get the drift. So, I want to break up central. Divide it up into sections that I should have done years ago. Books, Unpublished Papers, Comics, AudioBooks, categories WITHIN those groups. There are a LOT of files to deal out. It would be relatively simple to do a multi-column search (shout out to DaltonST) and then copy-move the found results to the new libs. But I'd like to find a way to automate the whole process. Decide which chunks to go to which new SemiCentral library in what order, push a button and let the computer run all weekend ... or all week as I suspect it would (creating the new sub-libraries, if not found to exist already) . Now, before anybody asks why I would not just do it manually, the first answer is that complicated manual processes are why we invented computers to prevent feeble humans from screwing up the sequences. AND, you remember the Management intake library? Well, I will have to chop THAT up in a similar way as the final monthly step for the new stuff. Otherwise, I'm back here asking about automation in about ten years. So, is there something within the plugin system that fills the bill? Within the main core of Calibre? Or is manual my only non-programming solution? Thanks, GM