All right, I think I've fixed it. I had to rewrite
a lot of code around the sending of metadata at the beginning of the connection process, but it syncs without issue for me with a library of 4500 books. The thing to check in this version is the process of adding books from non-Calibre sources (e.g. download from your browser and then tap on "Open in CC"), and sync'ing metadata that's been changed. Ideally both of those with a library of at least a few hundred books.
Hopefully it will still work for all of you too
