I copied the Calibre Library file onto the external. I had thought I had updated the metadata through the 'metadata backup' in library maintenance before I copied the files though, so what actually happened was me copying the old data.
The other mistake I made was using the add option:
'Add books from directories, including sub-directories (Multiple books per directory, assumes every ebook file is a different book)'
I payed around with the app after the feedback from you guys, saved my library with 'save to disk' and then used 'Add books from directories, including sub-directories (One book per directory, assumes every ebook file is the same book in a different format).' to add the books from the file containing the saved ebooks.
All the editing I did to the metadata was saved and loaded properly this time! No loss of data! And with over 20 000 books, I'm rather glad about that little fact.