I think that's using MTP. There is a horrible amount of overhead from that.
Are you on Windows? You can try my adbsync.exe (in my sig).
If you have the room, you can just make a new directory on /sdcard and copy your entire library there.
Then you can see how fast it goes and then just delete this whole duplicate directory.
Code:
C:\>adb shell
$ cd /sdcard
$ mkdir Test
$ ^D
C:\>adbsync C:\Caliber-whatever sdcard/Test /ascu /s
...
C:\>adb shell
$ cd /sdcard
$ rm -r Test
$ ^D
C:\>
There may be some difficulty in referring to sdcard.
You may have to use "storage/emulated/0/Test" or "mnt/sdcard/Test".
This stuff should be put in a makefile or a batch file so you don't have to remember the details.
What is described above doesn't modify anything on the desktop at all.