Update a book (sideload), lose reading position

[Jellby]

Today I updated a book (fixing the formatting, no changes in metadata) in my Aura, running 3.11 (sideloaded epub, with calibre). When I opened the udated book, it returned to page 1, although I was already midway through the book. Fortunately I had no annotations, I'm afraid they could have been lost.

Also, I have the Kobo Utilities plugin change the author and title in the database. This book was already uploaded, an the data was correct (the way I want it). After I updated the book, apparently the reader re-processed the epub file and "updated" the database writing back the author and title contained in the epub file, instead of keeping the info already present.

Has this always been so and I have just been lucky enough not to experience it before? Or has anything changed between 3.5 and 3.11?

[davidfor]

That is the expected behaviour. The file size of the book is recorded in the database. If the books file does not match that, the device assumes it is a new book, removes all details from the database and processes the book as a new book. And that would also mean it gets the metadata from the new copy of the book.

If you sent the book using calibre, the driver updates the file size during the send. That prevents the book from being reprocessed. But, it is only a good idea if you don't change the structure of the book. As long as the internal files and the TOC don't change, it will be OK.

[Jellby]

Originally Posted by davidfor

If you sent the book using calibre, the driver updates the file size during the send. That prevents the book from being reprocessed. But, it is only a good idea if you don't change the structure of the book. As long as the internal files and the TOC don't change, it will be OK.

I did send it with calibre, so according with the above it shouldn't have been reprocessed. I didn't change the book's structure, just some text inside.

I also changed the modification date in the OPF (but only that). Could this have triggered the reprocessing?

[davidfor]

Originally Posted by Jellby

I did send it with calibre, so according with the above it shouldn't have been reprocessed. I didn't change the book's structure, just some text inside.

I do that all the time. Occasionally I find the book has been processed, but usually that means I changed the title or author. These are both used in my save template so a change of these will give the file a different name and the device sees that as a different book.

I also changed the modification date in the OPF (but only that). Could this have triggered the reprocessing?

How did you do that? If you unpacked the epub, changed the file and repacked it, then odds are it would have changed size. If you did that after sending the book to the device, the sizes wouldn't match.

[Jellby]

Ok, let's step back.

1. I had the book in the reader. I had set KU to use a particular plugboard to change the author and title in the database (using *_sort and replacing spaces), but this plugboard is not used for sending books, it is only applied manually, as far a I can tell. At this moment, it was already applied, so the author listed in the Kobo is "LastName, FirstName"

2. In my computer, I modify the book to make some fixes. This I do by modifying the source files and rebuilding the epub. As far as I remember, I only changed some text and the modification date in the OPF.

3. I uploaded the modified file with calibre. I did not change the author, title or send-to-device template, so the file is overwritten. After uploading the file I did not "update metadata" in the database with KU.

4. When I unplugged the device, the book still appeared in its home screen tile (I don't remember what was the "% progress" reported there). I opened the book and instead of returning to where I was reading, it goes to the first page. Then I looked at the library and see it has the metadata from the epub file (author as "FirstName LastName"), instead of what was supposed to be in the database.

So, the device has reprocessed the file. I did not modify the file after sending it with calibre, and the only modifications where in the text files and in some part of the OPF which is probably not read by anyone but myself.

It is of course possible that I did something else that I don't remember.

[davidfor]

OK, I don't know what happened. From what you said, it should have kept the book and not reprocessed it. I know it is working here as I have done the same with at least five books over the last weekend. And as a check, I unpacked a book on the device, updated the modified date in the OPF, repacked and updated the size. The book wasn't reprocessed. The I changed the size in the database to not match and it was reprocessed.

I'll have to wait for it to happen again. Maybe the firmware is checking something else as well as the size. I haven't seen any evidence, but it might be doing it. If I see it happen here, I'll have a look and maybe I will see what it was.

[Jellby]

It has happened again. This is the first time I update something since then, so I'm guessing it will continue happening.

I've updated some books (through calibre), both in the internal memory and SD card, all of them had their status reset. A finished book was marked as unread, the ones I was reading have lost their location and font settings.

I think since I updated to 3.11 the SD card does not appear anymore as a mountable drive in the computer (Kubuntu 14.04), but it is recognized by calibre. I guess this is normal (although inconvenient, since now I cannot backup the card in the device), but I mention it just in case.

[GeoffR]

I edit quite a few books and reload them with Calibre, there has been the occasional one that loses its current reading position as a result, but the vast majority retain the reading position after the edit.

I haven't ever seen a book be reprocessed after editing it, but some of the metadata is cached, so if I edit the database to change the title for example, the old title will still show for a while unless I turn the device off and back on. Same as with replacing covers.

[davidfor]

@Jellby: It shouldn't happen. And doesn't happen to me. It is unusual for me to go more than a couple of days without fixing something in the books I am reading and replace them. I updated five books last night without any issues.

Could you run calibre in debug mode the next time you do this? Then post the log. Maybe that will give me a clue as to what is happening.

[JSWolf]

I don't know if this will apply to Kobo, but it might help.

When I update an eBook on my T1, I don't send it to the T1 via Calibre, I just copy the new version over the existing version so all files that contain the information about this book won't be modified. On;y the copy of the eBook will be changed. Then When I go back into the eBook, the T1 keeps me at the page I was in before I updated.

Try just copying the fixed version over the unfixed version and see what happens.

[davidfor]

Jon: Read the thread. Specifically, my first reply.

[Jellby]

I had to apply further updates to a book, so I uploaded it again. This time I made sure I use the "update metadata in device database" option of the Kobo Utilities plugin after uploading the book and before disconnecting the device. No reprocessing this time, but KU reported that no metadata had been changed, so it may be unrelated.

[davidfor]

It is unrelated. The file size is updated by the driver. None of the changes made using the "Update metadata" function in utilities plugin will affect this. It does have options to reset the reading position, but so long as you don't select them, that won't happen.

[Anak]

Originally Posted by JSWolf

I don't know if this will apply to Kobo, but it might help.

When I update an eBook on my T1, I don't send it to the T1 via Calibre, I just copy the new version over the existing version so all files that contain the information about this book won't be modified. On;y the copy of the eBook will be changed. Then When I go back into the eBook, the T1 keeps me at the page I was in before I updated.

Try just copying the fixed version over the unfixed version and see what happens.

Better don't do that as it doesn't work and avoid an unpleasent surprise that a book seems to be not present on the reader or can't be opened/read.

I do edit normal epubs (not kepubs) to fix problems, mostly OCR problems.
But simply copying the updated version over the old version does not work.
When you do this the book seems to be gone, as it is not present on the ereader. It sure is, but it is removed from de database (or there is no match between the updated book and the info in the database).

Older firmware versions
Or copying a book over an existing book is not guaranteed to work, as I do remember it sometimes did but not always. Note this behaviour occurred with now very old firmware releases. If it didn't work then the book seemed to be disappeared from the reader (that is the same behaviour as I described above). Or, it showed the book on the home page but without the cover image. It only showed the standard 'fall-back' cover for kepubs and epubs, that is some boxed text like "Kobo kepub" or something similar for a regular epub. The cover image from the book was not recreated.
But then it was not possible to open the book and a message was shown that said something like "This book can not be opened".

Why it sometimes did or didn't work I don't really know. I guess, if the modifications are very minimal it might work, otherwise it probably fails.

The solution that does work is.
1. remove existing the book from your reader. This option to remove books is present in the library.
2. Connect your reader to a computer and copy the updated book to your reader. After the database is updated you can read the book.

If you prefer to remove books when the reader is connected to a computer, then you should disconnect it first (to update the database on the reader) before you connect the reader again to copy an updated version of the same book.