Can't help with the permission problems, although I would guess that it is forcing the login because calibre is asking to talk to the screen while calibre-server does not.
As regards running the server standalone, the
documentation you are looking for is here. The "missing books" probably happen because the instance of calibre-server is looking at the default (empty) library instead of the "real" one you have put somewhere else. You have two choices:
1) use the --with-library option to calibre-server
2) use the CALIBRE_CONFIG_DIRECTORY environment variable to tell the content server to use the same config that GUI calibre uses.
The first is almost certainly easier.