YOU CANNOT DO THAT!!!!
realtime syncing of calibre is subject to race conditions -- calibre and dropbox fighting over who gets to modify the database and which copy is the real copy.
Especially if you run calibre at the same time on both computers!
calibre keeps the database in-memory, and it won't pick up on outside changes until it restarts -- by which time you will either have an irreparably damaged database or a bad copy from the aforementioned fight.
Run calibre on
one computer, with Dropbox
paused. Close calibre. Sync Dropbox.
Switch to other computer. Sync Dropbox fully. Pause Dropbox. Run calibre. Close calibre. Sync Dropbox.
Repeat ad nauseam.
...
What you saw, files on disk that calibre doesn't know about, is the classic, canonical example of not following the above directions, and as a result, corrupting the database.
To fix:
Run Library Maintenance ==> Restore database.
Then run Check library. Verify that you don't have loose files floating around -- if you do, move them out of the library and re-add them.