A Kindle won't change its collection files by itself
. You haven't said what you "did" to your Kindle. Did you perhaps use the Kindle Collections plugin? Or are you using some PC based software?
The first step any time collections get changed is to restart your Kindle - Home -> Menu -> Settings -> Menu -> Restart. let the Kindle restart and then it still takes a few seconds after that to get the collection numbers updated.
Only once you have done that will you see exactly what the state of collections on your Kindle are.
And you can sort your collections alphabetically too...