Just to clarify HarryT's comment - the user should still be using the Remove menu option to make sure the calibre database is updated. It is then a second step required (if you have the buggy version) to open the folder directly and delete the same book formats you chose from the file system. Not something I would personally "encourage" given the room for error but HarryT is certainly correct in that the combination will act as a workaround.
Another option (after waiting for the next release) is to use the library maintenance function to "check" the library. That will show you the orphaned file formats, and you just tick the checkboxes and click the delete button on that screen.
As an aside to Kovid - some form of multi-select would be very welcome on that screen to allow checking lots of rows at once. I had well over a hundred files that were orphaned due to this bug, and it was a bit painful scrolling and clicking each one...