Simple script for cleaning Calibre office empty folders

phiro · 09-15-2018, 12:50 AM

I had a lot of empty folders after a while when using Calibre over many upgrades. The folders only had the cover and the calibre library files. So there were a lot of books in the library with no content.
I made a script to remove these entries and then refreshed/repaired the database from Calibre. For this interested this is the script at https://github.com/phiroict/calibre-ebook-cleaner

Have fun!

theducks · 09-15-2018, 02:05 AM

It is HIGHLY discouraged, to use any other method to Delete or clean a Calibre Library.

If you are ending up with unintended 'empty books' (Some of us create those to document our Paper Library), then your process is incorrect.

Calibre cleans up old News (if configured),

All other corruption is caused by OTHER programs interfering with file/folder operations. Most common culprits: CLOUD sync of the Library while Calibre is running. Over active, A/V scans is a close second

itimpi · 09-15-2018, 02:08 AM

If you think you have empty folders in the Calibre library that should not be there then the correct way to proceed is to run the Calibre Library Maintenance option. This can identify mismatches between the Calibre metadata database and the associated file system.

DNSB · 09-15-2018, 03:22 AM

Would not a simple format:false in the calibre search bar and then deleting the selected books have the same effect without taking chances on modifying the calibre library from outside calibre?

Divingduck · 09-15-2018, 03:59 AM

You will find with this method only records with no format files but not out of sync folders and files (because calibre maybe hadn't recognize manual manipulations from outside). In this case I would suggest the maintenance tool too.

theducks · 09-15-2018, 01:09 PM

Quote:

Originally Posted by DNSB

Would not a simple format:false in the calibre search bar and then deleting the selected books have the same effect without taking chances on modifying the calibre library from outside calibre?

As the other Duck points out: That uses what Calibre knows were 'empty books' (and won't find those in never knew about)
Run Library Maintenance: Fix ALL issues (Some fixes include using the OS to move unknown files and folders out of the Library so they can later be added the traditional Calibre way)

Adoby · 09-15-2018, 05:32 PM

If you have empty folders then you may have orphan books as well. Non-empty folders that calibre has lost track of. This can be caused by having the calibre library on a Linux pc or nas and accessing it from a windows computer over the network. Or mixing case sensitivity / not case sensitivity in some other way. It may slowly corrupt the database and cause calibre to loose track of files and folders.

phiro · 09-15-2018, 05:39 PM

Hi all, thanks for all the feedback!
Most of these library actions from Calibre I had tried to use before with no success. Adoby's response was close to my own situation (There was sharing of a linux based NAS with windows machines) so that may have caused my original problem. The ebook directory is many years and versions old and that may have added to the issues as well. The script resolved it for me. But if internal tools would remedy it better, I'd certainly try that first.

BetterRed · 09-15-2018, 08:42 PM

@phiro - my suggestion is you (and everyone else) should run the Check Library option periodically (e.g. weekly, monthly, quarterly, whatever). Check Library is also an option for the calibredb command, it outputs to a CSV.

I do it as part if my 'weekly housekeeping'. If it reports anything, then while I'm fixing the problem I'll probably recall how I created the problem, which acts as a timely reminder to 'stop doing that'

BR

Divingduck · 09-16-2018, 04:53 AM

An other way you can force those problems is with automatic import via folder and a wrong setting how to generate metadata with mass imports. In case you set up metadata read by book content rather than file name and in addition a not so good setup for adding books, you sometimes can come in trouble if metadata are not available or scrap (saw this mostly in combination with PDF formats). If I use this way, I often change the behavior to generate a title from the file name as these are more likely unique (in my use cases) and better in handling especially with automated unsupervised background tasks.

As was mentioned, a bit regular housekeeping is always a very good idea to keep everything smooth with no surprises.

09-15-2018, 12:50 AM	#1
phiro Junior Member Posts: 2 Karma: 10 Join Date: Sep 2018 Device: IPhone	Simple script for cleaning Calibre office empty folders I had a lot of empty folders after a while when using Calibre over many upgrades. The folders only had the cover and the calibre library files. So there were a lot of books in the library with no content. I made a script to remove these entries and then refreshed/repaired the database from Calibre. For this interested this is the script at https://github.com/phiroict/calibre-ebook-cleaner Have fun!

09-15-2018, 08:42 PM	#9
BetterRed null operator (he/him) Posts: 20,565 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	@phiro - my suggestion is you (and everyone else) should run the Check Library option periodically (e.g. weekly, monthly, quarterly, whatever). Check Library is also an option for the calibredb command, it outputs to a CSV. I do it as part if my 'weekly housekeeping'. If it reports anything, then while I'm fixing the problem I'll probably recall how I created the problem, which acts as a timely reminder to 'stop doing that' BR Last edited by BetterRed; 09-15-2018 at 08:44 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Empty folders when adding books	LucyOne	Calibre	0	07-15-2017 06:27 AM
Librarian Sync creates empty collections folders on Kindle	Trups	Kindle Developer's Corner	3	01-26-2017 08:38 AM
Simple script to saves Series to disc	Mike.Gallagher.1	Calibre	2	03-23-2016 06:01 PM
Calibre with "empty" folders (no ebook file)	Orionpkmyz	Calibre	1	11-13-2015 10:20 AM
.SDR folders empty when loading MOBI covers on Kindle Paperwhite	abarnard	Library Management	1	12-25-2012 09:57 PM

09-15-2018, 02:05 AM	#2
theducks Well trained by Cats Posts: 29,791 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	It is HIGHLY discouraged, to use any other method to Delete or clean a Calibre Library. If you are ending up with unintended 'empty books' (Some of us create those to document our Paper Library), then your process is incorrect. Calibre cleans up old News (if configured), All other corruption is caused by OTHER programs interfering with file/folder operations. Most common culprits: CLOUD sync of the Library while Calibre is running. Over active, A/V scans is a close second

09-15-2018, 02:08 AM	#3
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	If you think you have empty folders in the Calibre library that should not be there then the correct way to proceed is to run the Calibre Library Maintenance option. This can identify mismatches between the Calibre metadata database and the associated file system.

09-15-2018, 03:22 AM	#4
DNSB Bibliophagist Posts: 35,352 Karma: 145435140 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos	Would not a simple format:false in the calibre search bar and then deleting the selected books have the same effect without taking chances on modifying the calibre library from outside calibre?

09-15-2018, 03:59 AM	#5
Divingduck Wizard Posts: 1,161 Karma: 1404241 Join Date: Nov 2010 Location: Germany Device: Sony PRS-650	You will find with this method only records with no format files but not out of sync folders and files (because calibre maybe hadn't recognize manual manipulations from outside). In this case I would suggest the maintenance tool too.

09-15-2018, 05:32 PM	#7
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	If you have empty folders then you may have orphan books as well. Non-empty folders that calibre has lost track of. This can be caused by having the calibre library on a Linux pc or nas and accessing it from a windows computer over the network. Or mixing case sensitivity / not case sensitivity in some other way. It may slowly corrupt the database and cause calibre to loose track of files and folders.

09-15-2018, 05:39 PM	#8
phiro Junior Member Posts: 2 Karma: 10 Join Date: Sep 2018 Device: IPhone	Hi all, thanks for all the feedback! Most of these library actions from Calibre I had tried to use before with no success. Adoby's response was close to my own situation (There was sharing of a linux based NAS with windows machines) so that may have caused my original problem. The ebook directory is many years and versions old and that may have added to the issues as well. The script resolved it for me. But if internal tools would remedy it better, I'd certainly try that first.

09-16-2018, 04:53 AM	#10
Divingduck Wizard Posts: 1,161 Karma: 1404241 Join Date: Nov 2010 Location: Germany Device: Sony PRS-650	An other way you can force those problems is with automatic import via folder and a wrong setting how to generate metadata with mass imports. In case you set up metadata read by book content rather than file name and in addition a not so good setup for adding books, you sometimes can come in trouble if metadata are not available or scrap (saw this mostly in combination with PDF formats). If I use this way, I often change the behavior to generate a title from the file name as these are more likely unique (in my use cases) and better in handling especially with automated unsupervised background tasks. As was mentioned, a bit regular housekeeping is always a very good idea to keep everything smooth with no surprises.

Advert

Advert