Yes, it's all automated. The current state of
https://github.com/koreader/koreader-translations is automatically pulled in when the nightly is created, but depending on specifics it may take up to 24 hours for Weblate to push the latest changes to that repo. So in some kind of worst case scenario where the timing badly aligns it might potentially take up to 2 days to actually show up in a build. But most of the time it'll just be the day after you made your changes.
Which languages are exposed in the menu to be selected is manual.