Glo Database malformed but signout does not work...

[koboy]

Originally Posted by davidfor

I doubt this is the case. I have deliberately filled the device to test a couple of things. I had no problems except if a firmware update was available.

not showing some book covers.

it's happened to me on all 3 of my e-readers, Aura 6",Mini,touch, if i go under 100mb, so i thought it was worth mentioning.



best wishes koboy

[davidfor]

Originally Posted by koboy

not showing some book covers.

it's happened to me on all 3 of my e-readers, Aura 6",Mini,touch, if i go under 100mb, so i thought it was worth mentioning.

How many books are you talking about? I did my test artificially by dropping large non-book files on the device to fill the space. If there are a lot of books and no free space, the issue could be that the database gets big and there is a problem writing updates to the database. I'll have to think about how to test that.

[Mrs_Often]

Originally Posted by GeoffR

It depends on what changes you make. If you change the file structure of the book (add, rename or delete files) or if you change parts the OPF or NCX or the places in the book that are linked to in the NCX then you should delete the book before reloading it, otherwise the database will still have the info from the old book which won't match the new book. This can cause unpredictable effects while trying to navigate the book.

If you just change the content of the book, fixing spellng mistakes or replacing images (without changng their filenames) etc., then it is okay to resend the updated book to the device without deleting it first.

A perfect, exactly what I wanted to know, thanks

Originally Posted by GeoffR

So in short, it is probably possible to fake the statistics, but it would be a lot of effort, I think you would have to insert a lot of fake reading sessions into the database.

Aww that's a shame! Thank you for explaining

Originally Posted by koboy

regarding the book covers, do you have more than 100mb of free space on the Glo? if not delete some books until you have at least 150mb spare, and then see if the book covers are showing.
best wishes koboy

I'm not actually sure how much space I have left, but interesting point, something I should keep an eye on indeed. Thanks!

Originally Posted by davidfor

For the firmware to get the kepub cover, the important bit is the "properties". The id has to there, but it is used as a cross-reference to other entries in the OPF. It doesn't matter what the value is.

If you have kepubs without the properties for the cover image that are showing covers, the covers probably got there some other way. Either the driver has been configured to send the covers or you used the Kobo Utilities to send them.

Ah good to know. I was wondering about that because all the books that I checked that were showing covers didn't have the properties tag so I was confused. But indeed, I have the magic Kobo Utilities set to stick its nose into cover things.

Originally Posted by davidfor

If he did a check on the database and kept the list of errors, I'd be interested in seeing them. I have fixed databases in the past, but with very limited type of errors.

I'm not sure if he saved anything, I'll ask and post the file if he saved it.

Originally Posted by davidfor

It really depends on when the cover didn't show up. If it never did, then it probably means the driver couldn't identify the cover and set the property when the book was sent. If it disappeared after a while, I don't know. There is a discussion on this in the extended driver thread with no conclusion. I did have some problems with firmware 3.16.10 and 3.17.0 that meant the stored cover image file was empty. In these cases, the cover was not displayed after I restarted the device. Removing the files and restarting the device generated correct covers. I think I have seen disappearing covers with 3.17.3, but haven't had one since the discussion started.

I don't think the covers ever showed up. Why would the driver not be able to identify the cover? It shows in calibre and all links refer to the correct place within the file as far as I can see in Sigil. I don't think I've seen covers disappear yet... fingers crossed... sounds like an annoying frustrating weird bug.

Originally Posted by davidfor

I can't help with getting the reading stats back after a factory reset. Some will come back from the Kobo server if syncs happen, but not all. For marking the books as read, just select the books you have read in your calibre library and update the device using the mentioned Kobo Utilities plugin. Of course, that needs an indication in the library. I put all the read books in a reading list. And in the past, I had a Yes/No column that I set to yes when I read the book. Of course, nowadays I let the utilities plugin do all the hard work.

Oooh the possibilities with Calibre and this plugin are just endless! I'm away for a few months and BAM I have no idea what's going on any more. Sounds absolutely delicious, I'm going to try this out. Thanks! Shame about the statistics. I suppose it's understandable that it's not an easy thing to tweak because it's not made or tweaking at all. But with the high risk of factory reset necessity it would be nice if there were some way to put data back. But yes database backup is the way to go. It's the first thing I did after the corruption got fixed and I'll definitely do it more often. It would be so sad to lose all the reading session time... like your 999 hours (I saw in a different thread)! Did you ever find out if that was a weird glitch (like, Kobo can't count higher than 3 digits or something... haha) or your actual reading time?

Thanks!

[koboy]

Originally Posted by davidfor

How many books are you talking about? I did my test artificially by dropping large non-book files on the device to fill the space. If there are a lot of books and no free space, the issue could be that the database gets big and there is a problem writing updates to the database. I'll have to think about how to test that.

the result is the same if i put the books on myself or if i use calibre to send to the device, once i get down to only having 100mb of free space left, any book i put on after that will not show the cover, and that happens on all 3 of my e-readers, and always as done regardless of firmware version.



best wishes koboy

[davidfor]

Originally Posted by Mrs_Often

I don't think the covers ever showed up. Why would the driver not be able to identify the cover? It shows in calibre and all links refer to the correct place within the file as far as I can see in Sigil. I don't think I've seen covers disappear yet... fingers crossed... sounds like an annoying frustrating weird bug.

The driver has to work out which image in the book is the cover. It does this by looking at the file name and some other thins. Sometimes, it cannot work out which is the cover and doesn't mark anything. If calibre has made any changes to the book, a conversion or updating the cover, it will be found, but I have few books that it didn't work for.

Oooh the possibilities with Calibre and this plugin are just endless! I'm away for a few months and BAM I have no idea what's going on any more. Sounds absolutely delicious, I'm going to try this out. Thanks!

I would have said the database backup and reading position store have been in the plugin for at least a year, so maybe you were gone for longer than I thought!

Shame about the statistics. I suppose it's understandable that it's not an easy thing to tweak because it's not made or tweaking at all. But with the high risk of factory reset necessity it would be nice if there were some way to put data back. But yes database backup is the way to go. It's the first thing I did after the corruption got fixed and I'll definitely do it more often. It would be so sad to lose all the reading session time... like your 999 hours (I saw in a different thread)! Did you ever find out if that was a weird glitch (like, Kobo can't count higher than 3 digits or something... haha) or your actual reading time?

I have looked at storing and restoring the stats, but I haven't seen a good way. It's close to being the full backup is the best way. It's definitely the simplest. Some do get restored from the Kobo server, but I haven't sat down and worked out what. I know some of the reading hours does as that 999 hours was a while ago with a factory reset since and I'm at 1001 hours today. I think after the factory reset it was in the low 900's but, I also have the other devices and I'm not sure what they all say.

[davidfor]

Originally Posted by koboy

the result is the same if i put the books on myself or if i use calibre to send to the device, once i get down to only having 100mb of free space left, any book i put on after that will not show the cover, and that happens on all 3 of my e-readers, and always as done regardless of firmware version.

koboy, I have to say this; you have an amazing ability to not answer the question. I understood what you said was happening. I did question it based on my experience. But, I also backed that off because I realised I hadn't done much and I thought of a possible reason that I didn't see it based on what I did. So, I asked a question that could help me to understand what was happening. Your response has no relationship to the question and adds no useful information.

I asked a question because I was curious if the problem was just related to available free space, or to something else. The obvious something else was the size of the database on the device. And that is related to the number of books. It occurred to me that a sensible behaviour was to always reserve enough space on the device to rewrite the database. The device can't stop you copying books to it to fill the space, but it can stop creating files. There were also firmware versions in the past where the cover generation stopped because of the number of books on the device (to be accurate: number of books and length of file names). This could happen with a lot more than 100MB of free space.

But, I was curious enough to do some experiments. As I don't have enough books with me to fill my Glo HD, I used other files to fill it. The database is currently around 78MB, so left 50MB free on the device and deleted all the cover images for sideloaded books. And yes, the covers weren't recreated when I disconnected. So, I increased the free space to bout 85MB (enough for a copy of the database) and the covers weren't regenerated. Then I went to 125MB and the covers started appearing.

So, it appears you are right and having less than 100MB free space does prevent cover generation. There are some other things that stop happening with limited free space. That makes it appear they are trying to make sure there is always some free space to prevent the problems that a lack of space could cause. The obvious one is making sure the database can be updated at any time.

[koboy]

Originally Posted by davidfor

koboy, I have to say this; you have an amazing ability to not answer the question. I understood what you said was happening. I did question it based on my experience. But, I also backed that off because I realised I hadn't done much and I thought of a possible reason that I didn't see it based on what I did. So, I asked a question that could help me to understand what was happening. Your response has no relationship to the question and adds no useful information.

I asked a question because I was curious if the problem was just related to available free space, or to something else. The obvious something else was the size of the database on the device. And that is related to the number of books. It occurred to me that a sensible behaviour was to always reserve enough space on the device to rewrite the database. The device can't stop you copying books to it to fill the space, but it can stop creating files. There were also firmware versions in the past where the cover generation stopped because of the number of books on the device (to be accurate: number of books and length of file names). This could happen with a lot more than 100MB of free space.

But, I was curious enough to do some experiments. As I don't have enough books with me to fill my Glo HD, I used other files to fill it. The database is currently around 78MB, so left 50MB free on the device and deleted all the cover images for sideloaded books. And yes, the covers weren't recreated when I disconnected. So, I increased the free space to bout 85MB (enough for a copy of the database) and the covers weren't regenerated. Then I went to 125MB and the covers started appearing.

So, it appears you are right and having less than 100MB free space does prevent cover generation. There are some other things that stop happening with limited free space. That makes it appear they are trying to make sure there is always some free space to prevent the problems that a lack of space could cause. The obvious one is making sure the database can be updated at any time.

the amount of books on my aura filled to within 100MB of free space, would of course be around double of that on my touch, with the aura having 4GB while the touch only as 2GB.


that is why i didn't answer the question not because i was avoiding it, but because i was quite sure the amount of books on the devices was not causing the problem, but rather it was down to the lack of free space.


i always make sure i leave at least 110MB, and then everything works fine, thanks for looking into this i do appreciate it.


best wishes koboy

[koboy]

Deleted

[davidfor]

Originally Posted by koboy

the amount of books on my aura filled to within 100MB of free space, would of course be around double of that on my touch, with the aura having 4GB while the touch only as 2GB.


that is why i didn't answer the question not because i was avoiding it, but because i was quite sure the amount of books on the devices was not causing the problem, but rather it was down to the lack of free space.


i always make sure i leave at least 110MB, and then everything works fine, thanks for looking into this i do appreciate it.

So, why didn't you answer like that in the first place? That statement is a lot more helpful than your non-answer. If you had stated that, I would have realised I was probably going in the wrong direction.

Of course, what you say assumes you have the same books, or same type of books, on all three devices. I can easily picture someone only putting epubs on a Mini but putting PDF's and comics (CBR/CBZ format) on the Aura. Because of the typical sizes of the formats and how they are handled by the firmware, a lot of epubs will produce a much bigger database than the same number PDF's. Or the same MB's worth of comics.

The thing for me is that the details are important. The details help me or someone else who is trying to solve a problem for you or someone else to work out what is happening or to be able to reproduce it. It's hard to know exactly what little detail will trigger the answer. As far as I'm concerned, you can't give me to many details for a problem. If there is a reason you can't, or don't want to answer a question, then say that. That's much better than ignoring the question.

[koboy]

Originally Posted by davidfor

So, why didn't you answer like that in the first place? That statement is a lot more helpful than your non-answer. If you had stated that, I would have realised I was probably going in the wrong direction.

Of course, what you say assumes you have the same books, or same type of books, on all three devices. I can easily picture someone only putting epubs on a Mini but putting PDF's and comics (CBR/CBZ format) on the Aura. Because of the typical sizes of the formats and how they are handled by the firmware, a lot of epubs will produce a much bigger database than the same number PDF's. Or the same MB's worth of comics.

The thing for me is that the details are important. The details help me or someone else who is trying to solve a problem for you or someone else to work out what is happening or to be able to reproduce it. It's hard to know exactly what little detail will trigger the answer. As far as I'm concerned, you can't give me to many details for a problem. If there is a reason you can't, or don't want to answer a question, then say that. That's much better than ignoring the question.

i didn't ignore the question you asked me intentionally, but please accept my apologies for not answering, to be honest i was concentrating more on explaining what i thought was causing the problem, which you have so kindly confirmed for me, is that leaving less than 110MB of free space is not a wise thing to do for the reasons you explained above.


i only read epub and kepub, and i completely agree the more information someone gives you the easier it will be for you to hopefully find a solution, at the risk of repeating myself, although it never stops me anyway thanks again


best wishes koboy