Quote:
Originally Posted by Cootey
Alright. I'm back with another database corruption. For the record, I never have these issues with my Kobo Clara.
I believe that the recent firmware updates released for the Kobo have improved things. I've gone 22 days without a database corruption—until earlier this morning. Unfortunately, I did not remember to quit Calibre before ejecting as you had suggested (something I had been forgetting to do recently), but I do have more information that might help us.
My notes on my latest Elipsa glitch:
1. Checked database using Kobo Utilities in Calibre. ✔
2. Imported about 21 manga through Calibre. Database ✔
3. Imported another 9 manga. Checked database again. It was corrupt.
|
How much time was between these? It shouldn't matter, but, see my comment on item 10.
Quote:
|
4. On a whim, I deleted all 30 manga manually off the Elipsa before ejecting it from desktop. This avoided the import/database destruction I’ve noticed in the past. All purchased books remained downloaded. No re-importation of all previously imported books occurred.
|
That makes sense. If the last books added before the database corruption, then removing them any added afterwards will mean the import is not triggered. Though there is a dependency on where in the database the problem is.
Quote:
|
5. Repaired Kobo account. Plugged Elipsa back into Mac. Checked database. It remained corrupt.
|
That is expected. The repair is fixing the differences between the data the device knows about your purchased books and what the Kobo server knows. While it should update the database, it won't actually attempt to repair the corruption.
Quote:
6. Backed up my .nebo journal files, then I restored from older backup via Terminal. This has ALWAYS worked before, but this time…
7. …Elipsa .jpg & .svg files (the stylus notations and their page screenshots) from the .kobo/markups directory were now loaded as individual files. Several Kobo pages worth in My Books. Even repairing the account didn’t properly categorize them. I thought perhaps they might be newer out of sync annotations that weren't part of the older database, but the rsync string I use to restore a backup deletes everything off the Elipsa that isn't present in the backup.
|
I don't know about this. I don't have an Elipsa to play with, so I don't know what to expect. The images being imported as "books" means that the device saw them as books. Is it possible you got the directory name wrong? Nothing under ".kobo/markups" should be imported. If you made a typo with the name, it would explain it.
Quote:
|
8. Logged out of my account on my Kobo, then logged back in. Markup files were still listed individually as "books". I checked the database with Kobo Utils plug-in in Calibre, and the database had been repaired as expected. Since I wasn’t sure which markup files were being categorized correctly and which were not, I…
|
That is consistent with point 7. It really does sound like the files were in the wrong place.
Quote:
9. Did a factory reset (with current firmware).
10. Annotations were restored. I don't know if I lost any, but I keep WiFi on all the time, so it's not likely. (Turns out I lost annotations from just the past 24 hours before corruption.) Database is not corrupt. I'm ready to rebuild my setup again. But I've lost time, I'll be losing more time getting things back up and running, and I'm grumpy about it. Why is this happening to my Elipsa? ����*♂️
|
The lost annotations should be those made after the last sync. But, the annotations made while reading should be synced shortly after you made them. But, it is possible that would happen at most every 30 minutes. By that I mean if you made an annotation, it will probably sync. But, the next might not be done for 30 minutes and all made in that time period will then be synced. I don't know for sure as I have never tested this.
But, I suspect that annotations sync stopped when the database became corrupted. That's why I asked about the time between notes 2 and 3. If there was some time between them, the corruption might have happened shortly after the first books were imported. But, you didn't see it until the trying to import the next set of books.
Quote:
|
Here are my observations. I’ve used Calibre to import manga before since the last Elipsa database corruption with no issues. Yet whenever the database is corrupted, Calibre seems to be involved. (I run Calibre in debug mode, so please tell me what you folks need to see to help identify what is happening. I'll send it to you since it likely contains names of my local drives and network). When I sideload hundreds of books by dragging and dropping onto the mounted Elipsa, I have ZERO database issues.
|
But, how do you know? The reason you know about the database corruption is that calibre discovers it. All that tells you is that between the last time and this time, something happened. And even that isn't accurate. Unless the problem is really bad, calibre will only report a problem if the parts it accesses have problems. Getting the list of books reads two tables. If there are no problems with those tables, then no error will be reported. But, if there are problem elsewhere, things could be going wrong elsewhere. And maybe they will affect the part that calibre usually sees later. This is why I sometimes tell people to check the database.
But, the basic problem is that calibre does not treat the Elipsa differently to any other Kobo device. If calibre is the cause, why aren't we seeing the same problem with all devices?
Quote:
|
I did notice, however that the Elipsa was constantly being ejected this morning. THIS behavior is consistent with corruption events. Whether I was plugged into a hub or directly into the back of my 2014 Mac Mini, the Elipsa would connect, then suddenly eject itself. This is very irregular behavior, and the Elipsa was the only device having this issue. The Mac did not display any warning dialogues as it usually does if you unplug a USB device without ejecting through the Finder first. That seems to indicate that it was rejected properly. When the Elipsa seemed to be stably connected for a few minutes, I imported the manga through Calibre. The Elipsa was not ejected during this process. However, after the transfer was complete (done in two steps BTW with me checking the database after each batch) the database was corrupt after the second, smaller batch. Then I did step 4 above to minimize the damage, but to no avail.
|
What else was going on? Did you have calibre or something else running that would have been accessing the device?
Quote:
After resetting the firmware, my Elipsa was connected for half a day without being ejected from my Mac. I can make sure that I quit Calibre and Kobo Desktop in the future. I have put in an order for a different USB hub. But neither of those things may be the problem. In the meantime, something became corrupt on that second Calibre batch. The Elipsa hadn't technically imported anything yet since it was still mounted to my desktop. I'd like to figure this out. I'm not the only person running into corrupt database issues with the Elipsa.
The following is the PRAGMA integrity check when I discovered the database was corrupt. I don't think this is any different than any other integrity check dump that I've shared. It just means the database is now corrupt:
|
That really just tells me that the database is corrupted an it is unlikely to be recoverable. The only times I have found that corrupt databases have been recoverable is when the problems are only in the indexes or they are only freespace errors. It won't harm to try a VACUUM, but, I expect you will get an error during this and it won't be able to complete.