First let me apologise for the somewhat pedantic tone of this post, if it seems like I feel I know all the answers, it's because I do
Let's take a step back and ask a slightly metaphysical question:
"Why do you store files in folders?"
I'm betting the answer to that is some version of the following two themes
a) I want to be able to find them easily
b) I want to categorize them
Now lets address these themes vis-a-vis calibre vs. the file system
a) When you want to find a file, 99% of the time, what you actually want is the book that file contains. In other words rather than searching for a file you are searching for a book title or author or genre or whatever. Now that kind of search is significantly more efficient when using a dedicated interface for it, like calibre.
b) Folders are really just a system of hierarchical tags. So when you put a file in the folder /mystery/asimov/foundation/ what you're saying is that the file is a mytery written by asimov and part of the foudation series. But the book could in general belong to many overlapping categories. For example the book is also scifi and future history and a hugo winner and written by a dead author and so on. There is no way you can express these kinds of relationships using hierarchical tags, but you can using arbitrary, i.e. non hierarchical tags, which is what calibre gives you.