Adding books has to:
1) read metadata from the files, which for some file formats can be ver slow, for example, epub can require rendering of html in a browser to generate covers.
2) Search the entire existing library for duplicates
3) Update lots of fields in the database
There is no way that adding books is every going to come close to the performance of copying files in the filesystem.