Mini help please - Kobo not syncing

[paola]

Hi all,

all good with my mini until today - last night while reading I synced the mini because there were some previews I wanted to download. I powered off without problems, and the cover of the book I was reading was showing on the device. The next time I powered it on, the book I was reading was not there!

I was on the road, so quite upset. I also noticed that some other books I had added relatively recently (up to about a month ago) were also not showing on the device.

I got back home, connected the mini to my laptop, all books are on the device (they show up on my windows explorer), but I could not access them from the mini. I did a sync, and they reappeared. But after each power off, they disappear again.

An additional issue is that Calibre no longer recognises the device - this is the message I get:

calibre, version 0.9.22
ERROR: Error: Error communicating with device

database disk image is malformed

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 85, in run
File "site-packages\calibre\gui2\device.py", line 470, in _books
File "site-packages\calibre\devices\kobo\driver.py", line 1658, in books
DatabaseError: database disk image is malformed

I tried a pinhole reset, nothing changes. that is books are not accessible in the device library. I can sync to my heart's content, and all books are back, but after a power off all the recent books disappear. I am loathe to do a factory reset as I don't want to lose all my annotations.

Any suggestions?

[davidfor]

The calibre error means the database on the device is corrupted. The other problems are symptoms of this. How, it was corrupted is a question. The most likely way is not ejecting the device properly when connected to a PC.

The simplest fix is to logoff on the device. This will wipe the old database and create a new one. It will restart, let you login again and download books from Kobo. Then it will add all the sideloaded books into the library. Unfortunately, you will lose the reading state for sideloaded books.

[paola]

Originally Posted by davidfor

The calibre error means the database on the device is corrupted. The other problems are symptoms of this. How, it was corrupted is a question. The most likely way is not ejecting the device properly when connected to a PC.

The simplest fix is to logoff on the device. This will wipe the old database and create a new one. It will restart, let you login again and download books from Kobo. Then it will add all the sideloaded books into the library. Unfortunately, you will lose the reading state for sideloaded books.

davidfor, many thanks - I do not know how the corruption happened, as I am very careful about ejecting the device properly, but somehow it must have happened.

Before I go for your suggestions, since I have a week old backup of the database, is it sensible to use it to replace the corrupted one (to salvage my annotations on sideloaded books), or would I be making things worse?

[robko]

Originally Posted by paola

Before I go for your suggestions, since I have a week old backup of the database, is it sensible to use it to replace the corrupted one (to salvage my annotations on sideloaded books), or would I be making things worse?

At this point you have nothing to lose by trying the backup. If it works great. If it doesn't you're looking at a factory reset or sign-in/out which is where you are now. The only risk is that it's possible that their could be corruption in the backup so that is waiting for a final trigger to manifest, so it may work for a bit and land you back where you are.

[paola]

davidfor, robko, thanks for the advice - I am now starting to wonder whether the problem is with Calibre, and I wonder whether I could pick your brains on this one.

The reason I am starting to suspect Calibre is that while it is true that I noticed the problem first when syncing, it is also true that I had added a book from Calibre in the same sessions, and problems started with the next power on.

I had solved the problem after copying the backed up database. More precisely:
1. I overwrote the sqlite database with a previous working version, about a week old;
2. I synced the device - at this point things seemed exactly as before, i.e. most recent books not showing;
3. powered off;
4. powered on;
and after this all was working, meaning I had almost all the books there, bar the last sideloaded kepub.epub. I read a book, powered off, powered on, and still all working. The last sideloaded kepub.epub was showing as present on the device upon connecting it to the laptop, but would not appear in any view on the device. So I connected to calibre, which could see the said book; deleted it and resend it again.

Then I disconnected the Mini, and the sideloaded book now showing in my mini library. Powered off, then on, and back to the original problem: all the most recent books not showing, including the just sideloaded one. I connected to calibre again, and got the same error message reported above, i.e. it cannot connect to the device.

So I started again process from 1. , this time first deleting the newly sideloaded kepub.epub: i.e. I deleted the kepub.epub, overwrote the database with the backed up file from last week, synced the device, powered off, then on, and again had the mini back in working order again with all my books (sideloaded included) except from the last kepub.epub - but now I don't want to re-send the book via calibre, as I am afraid it will mess it up again.

Any ideas on what could be going wrong? I have been supercareful (as always, I should say) about disconnecting the device properly.

[tshering]

Originally Posted by paola

I did a sync, and they reappeared. But after each power off, they disappear again.

You might try this: Sync the device. If those books appear, delete them via the menu. Then connect to the pc and add those books anew. Maybe you should also delete the related files in the manifest-folder, if they are then still there.
Good luck!

This was meant to refer to your first post!

[paola]

Originally Posted by tshering

You might try this: Sync the device. If those books appear, delete them via the menu. Then connect to the pc and add those books anew. Maybe you should also delete the related files in the manifest-folder, if they are then still there.
Good luck!

This was meant to refer to your first post!

tshering, thanks for this, but the situation has changed with respect to the first post, in the sense that I have now re-constituted the mini as I wanted it, with the exception of a single epub that I now suspect be the source of the problems - it looks like if I sideload it with Cailbre as kepub.epub, something "breaks" - not sure whether it is the kepub.epub or the book itself, or what it is.

So I'd say now my problem is to reload this book, but this seems now impossible to do!

[davidfor]

Originally Posted by paola

davidfor, robko, thanks for the advice - I am now starting to wonder whether the problem is with Calibre, and I wonder whether I could pick your brains on this one.

The reason I am starting to suspect Calibre is that while it is true that I noticed the problem first when syncing, it is also true that I had added a book from Calibre in the same sessions, and problems started with the next power on.

I had solved the problem after copying the backed up database. More precisely:
1. I overwrote the sqlite database with a previous working version, about a week old;
2. I synced the device - at this point things seemed exactly as before, i.e. most recent books not showing;
3. powered off;
4. powered on;

Step 2: was this using WiFi after disconnecting from the PC? Or while connected using the desktop app?

and after this all was working, meaning I had almost all the books there, bar the last sideloaded kepub.epub. I read a book, powered off, powered on, and still all working. The last sideloaded kepub.epub was showing as present on the device upon connecting it to the laptop, but would not appear in any view on the device. So I connected to calibre, which could see the said book; deleted it and resend it again.

Are you using the extended driver that supports kepub.epub files? And if so, which version? And which version of calibre?

Then I disconnected the Mini, and the sideloaded book now showing in my mini library. Powered off, then on, and back to the original problem: all the most recent books not showing, including the just sideloaded one. I connected to calibre again, and got the same error message reported above, i.e. it cannot connect to the device.

So I started again process from 1. , this time first deleting the newly sideloaded kepub.epub: i.e. I deleted the kepub.epub, overwrote the database with the backed up file from last week, synced the device, powered off, then on, and again had the mini back in working order again with all my books (sideloaded included) except from the last kepub.epub - but now I don't want to re-send the book via calibre, as I am afraid it will mess it up again.

Any ideas on what could be going wrong? I have been supercareful (as always, I should say) about disconnecting the device properly.

It sounds like there is a problem with that book. Or with the driver getting it onto the device. Have you tried sending another book?

As I ask above, it sounds like you are using the extended version of the driver by Joel. I haven't looked at it for a few weeks, so I don't know what state it is in. There was a version earlier in the year that did more updates to the database than. Joel withdrew that because he couldn't get them right.

It is possible it is a problem with the book. Any chance I can look at it? If it is legal to attach it here, do so. Otherwise upload it somewhere and PM me with a link. Or PM for an email address to send it to. If you kept a copy of the corrupted database, I would also be interested in looking at it.

[paola]

Dear Davidfor:

Originally Posted by davidfor

Step 2: was this using WiFi after disconnecting from the PC? Or while connected using the desktop app?

via wifi

Originally Posted by davidfor

Are you using the extended driver that supports kepub.epub files? And if so, which version? And which version of calibre?

yes, I am using Joel's version, which has been working well until last week. I donwloaded it on February 16th (I think it is the one for Calibre 0.9.19), and I am on Calibre 0.9.22

Originally Posted by davidfor

Have you tried sending another book?

no I have not, you are right I should try, but I am a bit weary of having to repeat all this again...

Originally Posted by davidfor

It is possible it is a problem with the book. Any chance I can look at it? If it is legal to attach it here, do so. Otherwise upload it somewhere and PM me with a link. Or PM for an email address to send it to. If you kept a copy of the corrupted database, I would also be interested in looking at it.

just sent you a PM - thanks a lot for your help!

[davidfor]

Originally Posted by paola

yes, I am using Joel's version, which has been working well until last week. I donwloaded it on February 16th (I think it is the one for Calibre 0.9.19), and I am on Calibre 0.9.22

You might want to update. I can see some code changes in Joel's driver to match the changes that were in calibre 0.9.21.

no I have not, you are right I should try, but I am a bit weary of having to repeat all this again...

I don't blame you. I hate it when it gets to this sort of thing.

just sent you a PM - thanks a lot for your help!

I've replied to the PM.

[paola]

Originally Posted by davidfor

You might want to update. I can see some code changes in Joel's driver to match the changes that were in calibre 0.9.21.

davidfor, many thanks - I have just updated joel's plugin, but it shows version 2.0.7 as the one I already had - anyhow, you never know!
As soon as I find the heart (and finish the book I am currently reading) I'll try the upload again!

[davidfor]

Originally Posted by paola

davidfor, many thanks - I have just updated joel's plugin, but it shows version 2.0.7 as the one I already had - anyhow, you never know!

Joel's driver doesn't have a separate version number. It is just showing the version from the main Kobo driver.

I'm looking at the book and I'll report later.

[davidfor]

I have had a look at the book and a test and I didn't have any problems.

Firstly I loaded the epub as an epub and had a quick look. Not issues.

Then I used Joel's driver to send it as a kepub. Again, no problems. I downloaded the driver yesterday, so it is the latest version.

I also opened both epub and generated kepub in Sigil with no problems. Sigil did object the the property added to the cover in the OPF for the kepub, but that is expected. I also ran epubcheck against both versions with the same result.

I loaded it onto both my Glo and my Touch. I checked the database on both, and there are no problems with them.

I don't know what is happening. It might be something in the version of Joel's driver that you were running. I can see that he has made some changes since then, so maybe he has fixed the problem.

Before trying this book again, could you take a copy of the database. Then, if it messes up the database, take a copy of the corrupt database and send them both to me. I can compare them and maybe see if something else is happening.

Also, I can't see it mentioned above: what firmware version are you on, and how did you get to it? Was the upgrade via the deskopt/WiFi or sideloaded?

[paola]

Originally Posted by davidfor

I have had a look at the book and a test and I didn't have any problems.

Firstly I loaded the epub as an epub and had a quick look. Not issues.

Then I used Joel's driver to send it as a kepub. Again, no problems. I downloaded the driver yesterday, so it is the latest version.

I also opened both epub and generated kepub in Sigil with no problems. Sigil did object the the property added to the cover in the OPF for the kepub, but that is expected. I also ran epubcheck against both versions with the same result.

I loaded it onto both my Glo and my Touch. I checked the database on both, and there are no problems with them.

I don't know what is happening. It might be something in the version of Joel's driver that you were running. I can see that he has made some changes since then, so maybe he has fixed the problem.

Before trying this book again, could you take a copy of the database. Then, if it messes up the database, take a copy of the corrupt database and send them both to me. I can compare them and maybe see if something else is happening.

Also, I can't see it mentioned above: what firmware version are you on, and how did you get to it? Was the upgrade via the deskopt/WiFi or sideloaded?

Dear davidfor,
I am afraid the situation has worsened: I did backup everything, then uploaded the book in question via Calibre with Joel's updated plugin - unlike the previous attempts, now the book does not even show in the device library (though the file is there, as I could check when connecting with the laptop). I powered off, then on, with no joy. The only way I can access the Mini is via windows explorer, as now Calibre cannot communicate with the device:

calibre, version 0.9.22
ERROR: Error: Error communicating with device

database disk image is malformed

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 85, in run
File "site-packages\calibre\gui2\device.py", line 470, in _books
File "site-packages\calibre\devices\kobo\driver.py", line 1658, in books
DatabaseError: database disk image is malformed

I've taken a copy of the post- transfer database, will PM you the link in a second.

Again, thank you all for your help!

EDIT: forgot to add - my firmware is 2.4.0, and I obtained it with a wireless sync.

[PeterT]

Maybe it's just me, but I really wonder why people want to risk using the totally unsupported .kepub.epub approach. It strikes me as adding in one unneeded complexity to the entire purpose of the device; namely reading books.

Yes there might be some bugs that are not present in the Access renderer on the Kobo; however, as people are seeing, there are additional issues that are raised by using it.