Auto-remove empty .sdr folders from Kindle?

[tomsem]

I wonder if it would be possible to have an option to remove the empty .sdr folders when calibre connects to a Kindle. It seems these never get cleaned up, short of Reset.

[kovidgoyal]

Whenever calibre deletes things itself, it removes empty folders. If you manage files outside of calibre on your kindle then calibre is not going to clean up for you since that would require a full scan of all folders on the device to find empty folders.

[tomsem]

Originally Posted by kovidgoyal

Whenever calibre deletes things itself, it removes empty folders. If you manage files outside of calibre on your kindle then calibre is not going to clean up for you since that would require a full scan of all folders on the device to find empty folders.

If it just removed the empty .sdr folders from the folder used for wireless downloads (/documents/Downloads/Items01/ on Scribe, might be different on older Kindles), that would be sufficient. Or an optional setting could specify this folder (or a list of them).

And while they can appear elsewhere, that's only when I've side loaded without using calibre and just dropped them in /documents. I don't do it a lot so can just clean those when I notice them.

[kovidgoyal]

That might be sufficient for you but it wouldnt be sufficient in general and I dont want to implement things in half baked ways.

[JSWolf]

Originally Posted by tomsem

I wonder if it would be possible to have an option to remove the empty .sdr folders when calibre connects to a Kindle. It seems these never get cleaned up, short of Reset.

What could be done is that could be added to the Kindle driver as an option or a separate plugin to do it.

For example, the Kobo Utilities plugin has an option to remove leftover covers that aren't being used.

[theducks]

Look at Quality Check Plugin.
Over a decade ago, Kiwidude added Fix: Cleanup files and folders

Since all plugins are mainly ZIP folders, you can look at / tweak the code as needed

[tomsem]

Originally Posted by kovidgoyal

That might be sufficient for you but it wouldnt be sufficient in general and I dont want to implement things in half baked ways.

I trust full folder scan is too 'expensive', and would not want to pay it every time I plug in.

I'm sure I've noticed this before, and just shrugged it off. But for some reason now I cannot not see them every time I go into look at what is on there, and I find it annoying. Chalk it up to retirement, my attention latches on to trivial things and magnifies them.

It is not even just Kindles with this issue. I plugged in my Pocketbook Era, and it has folders devoid of content, with orphaned .sdr folders as well as completely empty folders (this is in the folder managed by calibre).

In the grand scheme of things not a big deal. But it would feel good if there were a Clean Up button somewhere.

[kovidgoyal]

Since the kindle just appears as a a disk it should be trivial to write a script that scans and deletes all empty folders on it. In fact here is one:

import os, sys, contextlib

def rmdir(x):
  with suppress(OSError):
     os.rmdir(x)

for (dirpath, dirnames, filenames) is os.walk(sys.argv[1]):
   for x in dirnames:
      rmdir(os.path.join(dirpath, x))

save it as clean.py and run it as

calibre-debug clean.py E:/

replace E:/ with the drive your kindle is mounted as.

[tomsem]

Originally Posted by kovidgoyal

Since the kindle just appears as a a disk it should be trivial to write a script that scans and deletes all empty folders on it. In fact here is one:

import os, sys, contextlib

def rmdir(x):
  with suppress(OSError):
     os.rmdir(x)

for (dirpath, dirnames, filenames) is os.walk(sys.argv[1]):
   for x in dirnames:
      rmdir(os.path.join(dirpath, x))

save it as clean.py and run it as

calibre-debug clean.py E:/

replace E:/ with the drive your kindle is mounted as.

It's not Windows, but trivial to modify to take a path. Thanks.

[tomsem]

Turns out it is not as simple as removing empty folders, though that takes care of most of the lint. (This will also delete empty non-.sdr folders, but that's probably a plus, or at least not likely to cause trouble.)

What is left over are some 'orphaned' (non empty) .sdr folders and temporary files with file extensions of pattern r"tmp\d*_(ASC|PHL)".

I'm working on a standalone version that handles these.

[JSWolf]

Originally Posted by tomsem

Turns out it is not as simple as removing empty folders, though that takes care of most of the lint. (This will also delete empty non-.sdr folders, but that's probably a plus, or at least not likely to cause trouble.)

What is left over are some 'orphaned' (non empty) .sdr folders and temporary files with file extensions of pattern r"tmp\d*_(ASC|PHL)".

I'm working on a standalone version that handles these.

A plugin would be nice to have to clean up a Kindle.

[tomsem]

This is what I came up with (not tested on Windows or Linux):

PHP Code:

from logging import info, basicConfig, INFO
from pathlib import Path
from re import fullmatch
from sys import argv

CONTENT_EXTENSIONS = {'.kfx', '.azw', '.azw1', '.azw3', '.pdf', '.txt'}
TMP_REGEX = r'\.tmp\d*_(ASC|PHL)'
basicConfig(level=INFO)


def remove_dir(folder):
    try:
        Path(folder).rmdir()
        info(f'Removed {folder}')
    except OSError as ex:
        info(f'** {ex} **')


def rmtree(root):
    for p in root.iterdir():
        if p.is_dir():
            rmtree(p)
        else:
            p.unlink()

    root.rmdir()


scan_folder = Path('/Volumes/Kindle/documents' if len(argv) == 1 else argv[1])
if not scan_folder.is_dir():
    info(f'Path {scan_folder} does not exist, quitting.')
    exit(1)

content_file_stems = set()
tmp_files = []
sdr_folders = []
for path_ in scan_folder.glob('**/*'):
    if '.sdr' in str(path_.parent) or 'updates' in path_.parent.parts:
        # take no action on content in .sdr folders or /updates folder
        continue
    if path_.is_dir():
        if path_.suffix == '.sdr':
            sdr_folders.append(path_)
        else:
            remove_dir(path_)
    else:
        if path_.suffix in CONTENT_EXTENSIONS:
            content_file_stems.add((path_.parent, path_.stem))
        elif fullmatch(TMP_REGEX, path_.suffix):
            tmp_files.append(path_)

for tmp_file in tmp_files:
    if (tmp_file.parent, tmp_file.stem) not in content_file_stems:
        Path(tmp_file).unlink()
        info(f'Removed orphaned temp file {tmp_file}')

for sdr_folder in sdr_folders:
    if (sdr_folder.parent, sdr_folder.stem) not in content_file_stems:
        rmtree(sdr_folder)
        info(f'Removed orphaned .sdr folder {sdr_folder}') 


All standard library, no external dependencies. Might need Python 3. On Mac you can just type 'python clean.py' once Kindle is plugged in. On Windows, you can use calibre-debug as interpreter (while supplying path argument like E:/documents).

It runs almost instantaneously, so I don't see any particular penalty to running it every time you plug Kindle in.

I haven't seen the tmp files except on one of my Kindles, so it's possible there are other patterns to look for.

I saw updates/ folder on a couple of them, with an .sdr folder inside. Not sure what it is for, but thought it best to leave it alone.

I think only /documents needs to be a target for cleanup. I would leave it to user to clean up screen captures in root, for example.

And I haven't seen any 'turds' in .documents folder so far (Scribe).

[Update] re-factored to use (preferred in Python 3) pathlib.