I doubt that that is the Collections Manager's fault, but you can remove all your collections from the context menu and choosing "Delete all collections".
On an unrelated note: ixtab, do you know of a way to create nested collections from an import from Calibre? Currently I have a custom column called "Kindle Collection" for example with the value "Books, Fantasy, The Hollows" -- how would I go about modifying this value so that Collections Manager would automatically create a nested collection from that while importing? E.g. "Books/Fantasy/The Hollows"?