How to manually fix a (probably) corrupted database?

Lucas Malor · 05-08-2013, 05:15 AM

I have some problems using the ToC of an epub. I've read this could be caused by a databse corruption. The last time I tried to fix it logging out and relogging in, but the Kobo won't start any more and I had to completely reset it.

So this time I want to try to fix the database manually. Do you know how to do it?

davidfor · 05-08-2013, 06:07 AM

It depends on the problem. I have fixed mine a couple of times, but in those cases a couple of indexes were corrupt. Forcing a rebuild was enough.

Every other time, it has been inaccessible pages in the database. I haven't found a method to fix this. But, I haven't tried that hard. The logout has fixed things most times. And I have no hassle with an occasional factory reset, the other times.

I also backup the database when I am doing something that might be dangerous to the database. Then I can simply restore it.

If you want to see what the problem is, you need to run the command "PRAGMA integrity_check" against the database. If you have an SQLite browser or other SQLite tool, that should be able to run it.

Lucas Malor · 05-08-2013, 06:42 PM

Thank you for the tip. It seems I have no problems with the db. Maybe it's the book itself. Do you know if there's a way to check/repair an epub with calibre? The problems are with the ToC.

davidfor · 05-08-2013, 08:58 PM

Running a conversion on it will build the TOC from scratch. There was also a TOC editor added to calibre in recent weeks. I haven't played with it much, but it has options to generate the TOC from headings and other things

You could also use epubcheck to validate it or open it in Sigil to check it. Sigil also has options to rebuild the TOC.

And I'm always interested in seeing what causes these problems. Any chance of sending me the book to look at? If so, PM me with a download link or an email to send it to.

Arios · 05-09-2013, 12:13 AM

+ 1 David

Lucas you may have a look here to use easily epubcheck (https://www.mobileread.com/forums/sho...ht=epubchecker) and find epubcheck directly here (http://code.google.com/p/epubcheck/d...0.zip&can=2&q=). Indeed, with Sigil you will surely be able to find the problem with your TOC (http://code.google.com/p/sigil/downloads/list).

I'm happy to have both here at the same time, and forgive me Lucas to hijack for a while your post. I don't want to create another post just for that

.

David, we had a brief discussion some weeks ago about the Kobo bug with long paragraphs (https://www.mobileread.com/forums/sho...d.php?t=197956) and I still try (not very hard though) to find a solution about that. A solution is certainly very possible because with the same parameters, if we rename an epub with the kepub extension and set widows and orphans to 0 in the CSS, it works without a hitch. What we have to know is what "kepub" triggers into the Kobo device to solve the problem. Lucas, perhaps you have and idea also?

Anyway, presently I think that this bug is the most annoying of all in the Kobo ecosystem, and thanks for your patience

PeterT · 05-09-2013, 12:18 AM

@Arios: The rename forces the Kobo to use a totally different rendering engine for displaying the content. .ePub uses an ADE RMSDK based engine, while .kepub.epub uses a NetFront ACCESS based engine.

Two totally different implementations, each with their own idiosyncrasies.

Lucas Malor · 05-09-2013, 03:31 AM

Thank you all, I used epubcheck and calibre to explode and rebuild. Probably the problem was caused by a very irregular doctype in toc file.

For the sake of curiosity, exploding and rebuilding an epub is simply unzip and zip?

About the bug, it was reported in the Bugs thread. If I had Sigil I'd make some testcase. Unluckily I can't compile it on my pc since I'm proudly using Ubuntu 12.04 yet. Thank you Unity.

Lucas Malor · 05-09-2013, 03:49 AM

@davidfor: I forget, for your curiosity:

Code:

ERROR: a.epub/OEBPS/content.opf: Obsolete or irregular DOCTYPE statement. The DOCTYPE can be removed.
WARNING: a.epub/OEBPS/toc.ncx: Irregular DOCTYPE: found '-//W3C//DTD XHTML 1.1//EN', expecting '<!DOCTYPE ncx PUBLIC "-//NISO//DTD ncx 2005-1//EN" "http://www.daisy.org/z3986/2005/ncx-2005-1.dtd">'.
WARNING: a.epub/OEBPS/sitepoint.css(31): Token ';' not allowed here, expecting a property name
ERROR: a.epub/OEBPS/ch09.html(1357,49): value of attribute "dir" is invalid; must be equal to "ltr" or "rtl"

This is what I fixed. There are also a bunch of warning about spaces in URIs and not used files that I ignored. The book is "PHP & MySQL: Novice to Ninja" (ok, the title sounds stupid, but the book seems good

)

davidfor · 05-09-2013, 04:05 AM

Did fixing them work? The first two probably cause the OPF and NCX not to be read correctly. If not, it could be with the spaces in the URIs depending on where they are.

davidfor · 05-09-2013, 04:11 AM

Quote:

Originally Posted by Arios

David, we had a brief discussion some weeks ago about the Kobo bug with long paragraphs (https://www.mobileread.com/forums/sho...d.php?t=197956) and I still try (not very hard though) to find a solution about that. A solution is certainly very possible because with the same parameters, if we rename an epub with the kepub extension and set widows and orphans to 0 in the CSS, it works without a hitch. What we have to know is what "kepub" triggers into the Kobo device to solve the problem. Lucas, perhaps you have and idea also?

As PeterT said, there are two renderers involved, each with their own "features".

Quote:

Anyway, presently I think that this bug is the most annoying of all in the Kobo ecosystem, and thanks for your patience

What are you reading? I didn't see this bug until it was reported nearly a year after I got my Touch. Then I had to download a sample to see it. I think I have only actually seen it in three books I have been reading. Though one of them had a lot of paragraphs that were long enough. But, that author needed an good editor.

Lucas Malor · 05-09-2013, 04:21 AM

Yes, the problem was gone

Uschiekid · 05-09-2013, 11:14 AM

Quote:

Originally Posted by Lucas Malor

Thank you all, I used epubcheck and calibre to explode and rebuild. Probably the problem was caused by a very irregular doctype in toc file.

For the sake of curiosity, exploding and rebuilding an epub is simply unzip and zip?

About the bug, it was reported in the Bugs thread. If I had Sigil I'd make some testcase. Unluckily I can't compile it on my pc since I'm proudly using Ubuntu 12.04 yet. Thank you Unity.

I'm on Ubuntu 12.04 as well and got a working older version of Sigil up and running...version 0.6.2
I got it from here (pretty sure that's the right link...definitely got my build from the gilbert dude):
https://launchpad.net/~rgibert/+arch...+build/4045186

Works great, although every time I start it, it wants me to upgrade using the Sigil homepage, which as you know doesn't have a build for ubuntu.

Arios · 05-09-2013, 07:47 PM

@PeterT

Many thanks for this information Peter,

Should I assume that the problem will not be resolved before that Kobo changes the rendering engine?

@David

You're right David, it is not too frequent, but I guess this is my favorite bug

@ Lucas

You can use Sigil with Ubuntu through wine (32b dependancies) or have a look here:
https://www.mobileread.com/forums/sho...d.php?t=211754

I dont know for Ubuntu, but it works very well with LMDE (64b)

Lucas Malor · 05-09-2013, 08:01 PM

Thank you guys, I'll try them

05-08-2013, 05:15 AM	#1
Lucas Malor Pain in the arse Posts: 758 Karma: 77856 Join Date: Apr 2013 Device: Kobo Aura One, Kindle 4	How to manually fix a (probably) corrupted database? I have some problems using the ToC of an epub. I've read this could be caused by a databse corruption. The last time I tried to fix it logging out and relogging in, but the Kobo won't start any more and I had to completely reset it. So this time I want to try to fix the database manually. Do you know how to do it?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Corrupted database	Qylie	Calibre	8	08-28-2014 02:02 AM
How do you fix a corrupted Zip header	Nathan Hale	Introduce Yourself	3	12-23-2011 12:27 AM
Corrupted database	garyft	Calibre	3	05-13-2011 02:10 PM
Manually add/delete ebooks to a database	sariel2413	Calibre	3	03-18-2010 04:45 PM
manually editing database	teje	Calibre	9	02-05-2010 02:11 AM

05-08-2013, 06:07 AM	#2
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	It depends on the problem. I have fixed mine a couple of times, but in those cases a couple of indexes were corrupt. Forcing a rebuild was enough. Every other time, it has been inaccessible pages in the database. I haven't found a method to fix this. But, I haven't tried that hard. The logout has fixed things most times. And I have no hassle with an occasional factory reset, the other times. I also backup the database when I am doing something that might be dangerous to the database. Then I can simply restore it. If you want to see what the problem is, you need to run the command "PRAGMA integrity_check" against the database. If you have an SQLite browser or other SQLite tool, that should be able to run it.

05-08-2013, 06:42 PM	#3
Lucas Malor Pain in the arse Posts: 758 Karma: 77856 Join Date: Apr 2013 Device: Kobo Aura One, Kindle 4	Thank you for the tip. It seems I have no problems with the db. Maybe it's the book itself. Do you know if there's a way to check/repair an epub with calibre? The problems are with the ToC.

05-08-2013, 08:58 PM	#4
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Running a conversion on it will build the TOC from scratch. There was also a TOC editor added to calibre in recent weeks. I haven't played with it much, but it has options to generate the TOC from headings and other things You could also use epubcheck to validate it or open it in Sigil to check it. Sigil also has options to rebuild the TOC. And I'm always interested in seeing what causes these problems. Any chance of sending me the book to look at? If so, PM me with a download link or an email to send it to.

05-09-2013, 12:13 AM	#5
Arios A curiosus lector! Posts: 463 Karma: 2015140 Join Date: Jun 2012 Device: Sony PRS-T1, Kobo Touch	+ 1 David Lucas you may have a look here to use easily epubcheck (https://www.mobileread.com/forums/sho...ht=epubchecker) and find epubcheck directly here (http://code.google.com/p/epubcheck/d...0.zip&can=2&q=). Indeed, with Sigil you will surely be able to find the problem with your TOC (http://code.google.com/p/sigil/downloads/list). I'm happy to have both here at the same time, and forgive me Lucas to hijack for a while your post. I don't want to create another post just for that . David, we had a brief discussion some weeks ago about the Kobo bug with long paragraphs (https://www.mobileread.com/forums/sho...d.php?t=197956) and I still try (not very hard though) to find a solution about that. A solution is certainly very possible because with the same parameters, if we rename an epub with the kepub extension and set widows and orphans to 0 in the CSS, it works without a hitch. What we have to know is what "kepub" triggers into the Kobo device to solve the problem. Lucas, perhaps you have and idea also? Anyway, presently I think that this bug is the most annoying of all in the Kobo ecosystem, and thanks for your patience

05-09-2013, 12:18 AM	#6
PeterT Grand Sorcerer Posts: 13,309 Karma: 78876004 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	@Arios: The rename forces the Kobo to use a totally different rendering engine for displaying the content. .ePub uses an ADE RMSDK based engine, while .kepub.epub uses a NetFront ACCESS based engine. Two totally different implementations, each with their own idiosyncrasies.

05-09-2013, 03:31 AM	#7
Lucas Malor Pain in the arse Posts: 758 Karma: 77856 Join Date: Apr 2013 Device: Kobo Aura One, Kindle 4	Thank you all, I used epubcheck and calibre to explode and rebuild. Probably the problem was caused by a very irregular doctype in toc file. For the sake of curiosity, exploding and rebuilding an epub is simply unzip and zip? About the bug, it was reported in the Bugs thread. If I had Sigil I'd make some testcase. Unluckily I can't compile it on my pc since I'm proudly using Ubuntu 12.04 yet. Thank you Unity.

05-09-2013, 04:05 AM	#9
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Did fixing them work? The first two probably cause the OPF and NCX not to be read correctly. If not, it could be with the spaces in the URIs depending on where they are.

05-09-2013, 04:21 AM	#11
Lucas Malor Pain in the arse Posts: 758 Karma: 77856 Join Date: Apr 2013 Device: Kobo Aura One, Kindle 4	Yes, the problem was gone

05-09-2013, 07:47 PM	#13
Arios A curiosus lector! Posts: 463 Karma: 2015140 Join Date: Jun 2012 Device: Sony PRS-T1, Kobo Touch	@PeterT Many thanks for this information Peter, Should I assume that the problem will not be resolved before that Kobo changes the rendering engine? @David You're right David, it is not too frequent, but I guess this is my favorite bug @ Lucas You can use Sigil with Ubuntu through wine (32b dependancies) or have a look here: https://www.mobileread.com/forums/sho...d.php?t=211754 I dont know for Ubuntu, but it works very well with LMDE (64b)

05-09-2013, 08:01 PM	#14
Lucas Malor Pain in the arse Posts: 758 Karma: 77856 Join Date: Apr 2013 Device: Kobo Aura One, Kindle 4	Thank you guys, I'll try them