[bigwoof]

so to summarise, I think I found two different bugs.

1. SQLITE_MAX_VARIABLE_NUMBER is set to 999 or lower (did not verify the exact number)

symptoms: if your collection has more books than this number, it will crash during the update with a "too many variables" error

solution: fix sqlite to have a higher limit (not sure how to do this myself)

my hacked solution (see previous post for details):
i) patch generate_collections.py to generate update sets that are 400 entries or lower
ii) comment out all the delete commands in change.lua

2. if you have a lot of books and collections (I am updating ~160 collections with about 4,000 books), the kindle will reboot during the update (out of memory I suspect)

Solution: patch cc_update.py to send only 100 SQL commands at a time to change.lua. this works for me and my collections updated

my fix is below (change the code at the bottom of cc_update.py)

            for i in range(0,len(self.commands),100):
                new_commands = self.commands[i:i+100]
                full_command = {"commands": new_commands,
                                "type": "ChangeRequest", "id": 1}
              
                r = requests.post("http://127.0.0.1:9101/change",
                                  data=json.dumps(full_command),
                                  headers={'AuthToken': self.session_token, 'con
tent-type':  'application/json'},
                                   proxies={'no': 'pass'})
                if r.status_code == requests.codes.ok:
                     log(LIBRARIAN_SYNC, "cc_update", "Success.")
                else:
                     log(LIBRARIAN_SYNC, "cc_update", "Oh, no. It failed.", "E")