Some things to try if not tried already:
1. If just using the tools available in Windows probably the first thing I would try would be to open the cmd: prompt and type "rd [Drive:]path /s"
So, for example, if the folder you are having problems removing is D:\myfolder\books type after the cmd: prompt
rd D:\myfolder\books /s
It is case insensitive.
2. Because I have it already installed, I would myself try the application Unlocker. While this is oriented to deleting known to be locked files and folders it will often delete them if there is no apparent reason for the problem. If wanted to try it is at
http://emptyloop.com/unlocker/ . To use just highlight the troublesome folder in Explorer, right click, select Unlocker and follow the instructions to delete. It is safe to use.
3. Sometimes you may find that even though you cannot delete a file or directory in Explorer that you can rename it and it may then be able to be deleted.
4. Ensure that the problem folder is empty (even if it looks empty). Open the cmd: prompt and type the path to the folder followed by del *.* . Then try to delete the folder.