Flip the thinking around.
Which way
is easier to manage?
If you put a book on a shelf.
Is it easier to find:
if the shelf is in your house?
or
In a open access (more public) area?
In the user area of your disk(s), there many things (usually the user

) that can modify the file(name) and location.
Calibre would have to search the entire filesystem to rediscover books, THEN, try and match them to existing entries.