You have to go to each page and click the "Mark" button again to remove them. I try to remove them as I go along, so that I only have one mark per book at any given time, except in a few specific cases.
In the situation it sounds like you're in, having a bunch of bookmarks, you might just try deleting the book from your Reader and reloading a 'clean' copy.
Hope one of those helps.
