Sync and Collections

[meeera]

Originally Posted by ownedbycats

I wonder if it has trouble with collections made up of entirely sideloaded books. This would match certain patterns I've noticed.

If it does, it's not a consistent trouble. All of my collections have always been made up of entirely sideloaded books.

[ownedbycats]

Yeah. Chances are it might just because my sideload-only collections are more frequently updated.

[ownedbycats]

Originally Posted by ownedbycats

Interesting thing I've noticed: Sync was stalling out. So I deleted a suspect collection (recently reached a certain number of books) and this fixed it. Then I let KoboTouchExtended re-create the collection - the sync issues didn't reoccur. Perhaps something similar to that 'differing case' mentioned earlier?

I wonder if it has trouble with collections made up of entirely sideloaded books. This would match certain patterns I've noticed.

It's happened twice more since (including today). Doing the same process fixes it; deleting the collections, syncing, then letting KoboTouch regenerate.

As usual, it mostly affects my sideload-only collections. Maybe the sync process doesn't like 'empty' ones?

The tricky part is identifying which collection is the glitchy ones. Nuking them all is easier.

[compurandom]

Originally Posted by ownedbycats

It's happened twice more since (including today). Doing the same process fixes it; deleting the collections, syncing, then letting KoboTouch regenerate.

I think I might be able to confirm this. I haven't been able to consistently sync for almost a year. After seeing discussion about collection size, I deleted several large collections (all >400 items) and blocked them from recreation in the kobo extended plugin, and then trimmed down a few more so that all of them are now under 150 items (all but one under 100 items) and it seems to be syncing reliably again...

[Lys]

Originally Posted by ownedbycats

It's happened twice more since (including today). Doing the same process fixes it; deleting the collections, syncing, then letting KoboTouch regenerate.

As usual, it mostly affects my sideload-only collections. Maybe the sync process doesn't like 'empty' ones?

The tricky part is identifying which collection is the glitchy ones. Nuking them all is easier.

When I want to sync, my new routine is just to delete all > 100 collections I have on my Libra, sync, and let Calibre recreate them on the next connection. This method never failed so far.

That said, I think the more people reach out the CS about this issue, the more probable is their tech team will spend time over trying to fix this issue. They told me they had the bug logged in their system, but if it's something not widely reported, any fix for it will be very low on their priority list.

[Sirtel]

Originally Posted by Lys

That said, I think the more people reach out the CS about this issue, the more probable is their tech team will spend time over trying to fix this issue. They told me they had the bug logged in their system, but if it's something not widely reported, any fix for it will be very low on their priority list.

Well, many of us here never sync and so we have nothing to report. I'm guessing there are more Kobo users who sideload everything than there are sideloading Kindle users, for example.

[Lys]

Originally Posted by Sirtel

Well, many of us here never sync and so we have nothing to report. I'm guessing there are more Kobo users who sideload everything than there are sideloading Kindle users, for example.

Of course, I meant the people that are routinely dealing with this issue, not the whole of people using Kobos at large

[ownedbycats]

My main issue is that I can't see a way to send in a bug report without the hellish chat support and the scripted checklist responses.

[Lys]

Originally Posted by ownedbycats

My main issue is that I can't see a way to send in a bug report without the hellish chat support and the scripted checklist responses.

I found the support through twitter a tiny bit better than through the chat box you get on the website.
You still have to go through the usual checklist (including things that have 0 to do with your current issue), but I managed to skip to the actual point quite quickly every time I used it, and with this particular issue, I was then escalated to a more technical level of support.

[milady133]

Originally Posted by Sirtel

Well, many of us here never sync and so we have nothing to report. I'm guessing there are more Kobo users who sideload everything than there are sideloading Kindle users, for example.

For the bug to appear there has to be a combination of this three types of users:
- Has more than 100 books
- Combines sideloaded and store bought books
- Has synching on

Just the first type is not that frequent, so I suppose there aren't much bug reports, and since restoring account+signing off then on restores the functionality temporarily, it's difficult to get the chat support to file a report since their checklist solution "works".

For my part, I tried to file a report years ago, I suspect the chat support just close my ticket with a solved despite my efforts to try to get them to acknowledge my problem, and I have been trying to live with the resetting of my account every few months until a couple of months ago.

I don't need synching so much, so I just created a new account without books, registered the Kobo to that account, and transferred the statistics of the old stored bought books to the new sideloaded books. If I ever have the urgent need of reading a recently bought book before I have time to process it in Calibre, I'll just use the web reader on my phone.

[compurandom]

Originally Posted by milady133

For the bug to appear there has to be a combination of this three types of users:
- Has more than 100 books
- Combines sideloaded and store bought books
- Has synching on

I think it's not "has more than 100 book" but "has more than 150(?) books in one category".

Dunno what the number is, but I'd say it's between 150 and 110.

[jmw1011]

This issue seems to have gone away for me. I posted in this thread about how I could only stop the sync issue by archiving all my kobo books and sideloading them on my Kobo so that my collections only contained side loaded books.

A few months back kobo had an issue where they lost everyone' s archived books, so my entire library was gone. I contacted customer support and they restored my all my books to me. I was afraid to re-archive them until kobo had fixed the problem, so I decided to try downloading them back to my kobo and adding them into collections again (I deleted the side loaded copies first). It has now been a few months since this happened. I've added new downloads to my collections and done several syncs without any issues. It seems deleting books from my account and restoring them was the fix I needed. This would be a pain if you have a lot of kobo books, but you could delete your owned books, find them in the store, and add them back into your account. Kobo will not make you repurchase them.

[compurandom]

Playing with this for months. I'm starting to think the real issue is just the sqlite database itself being slow. Slow can be caused by too many entries in the table, or it can be caused by too much churn of books being added and deleted, leaving the database fragmented.

I've noticed that when sync repeatedly fails and I do a soft database reset (by deleting it and letting all my sideloaded books rescan and then restoring bookmarks and categories with calibre), the calibre metadata sync itself gets much faster and also kobo sync stops failing.

I have to wonder if deleting large categories works because the database is smaller, or if it removes a bunch of fragments, or perhaps both. If it is database access time rather than database size threshold, this might explain some of the seeming randomness of the issue.

So next time I see a sync fail, I'm going to try compressing the database and see if that fixes it and see if it also shortens the calibre metadata sync too.

[JSWolf]

The Kobo Utilities plug for calibre can compress the database and get rid if the churn.

[compurandom]

Originally Posted by JSWolf

The Kobo Utilities plug for calibre can compress the database and get rid if the churn.

Oh thank you, master of the obvious. I would never have considered that calibre could cause churn by dumping hundreds of blogs on my device and then would also have an easy way of defragmenting the database.

Less sarcastically, at least I know someone heard my whimper in the wilderness.