I didn't do another factory reset, but I did try a few things and it seems to be related somehow to the SD card.
Via the Reader, I formatted the SD card, erasing all of the files that were on it. I checked my YA Fantasy/Urban Fantasy collection again (this collection is shown in the photos of my original post illustrating the problem I'm having). The "removing" spots were gone.
I loaded a handful of files back onto the SD card and checked the collection again. None of the files on the SD card go in the YA Fantasy/UF collection, but the "removing" spots were showing up there again. They showed up in a few main memory collections and some of the SD-only collections.
So I popped the SD card out and checked the YA Fantasy/UF collection again and the "removing" spots were gone again. I don't know why it's happening, but I think it's safe to say that it has something to do with the SD card.