You mean thumbnails ? You could use the print screen button and Gimp to turn the covers into icons.
If you're using Windows XP, save them as "Microsoft Windows icon (*.ico)". I think they need to be 96x96. I have no idea about Vista or 7. Probably 256x256... Experiment and find out. Or you can use Google.
If you're using Linux you can save them any way you'd like, JPG, PNG, doesn't matter. It will even take SVG. Especially SVG ! No matter the format, Dolphin (the default file manager from KDE) will adjust the quality on-the-fly when you zoom in on the files/folders so you don't have to bother with various sizes like 16x16, 32x32, 48x48, etc.
Exporting the cover is easy. Print screen is one way, but you can also click the cover in Adobe Reader (just tried it on Linux), choose "Copy Image" and paste it into Gimp. Then save it as .ico or .png, whatever.