You might be encountering the FAT32 "number of files in a folder" performance problem. This CC FAQ might provide some help:
[Android] I have added a lot of books and now adding books takes forever. What do I do?.
As for manually copying the files: (assuming the current devs haven't change this) yes, you can do it. The steps:
- Copy the directory structure exactly as it is. However, if you are running into the FAT32 performance problem then this operation might also be very slow.
- Delete or rename the source directory so CC won't try to move the files in the next step.
- In CC Settings / Formats and folders, point CC to the new directory.
This works because for each book CC keeps the path from the parent directory to the book in its database. The path to the parent is kept elsewhere.