Calibre (PRS T1) Errors in SD Files?

enriquep · 10-31-2011, 06:26 AM

Hello everybody.

I think that I am having frequent errors in the books that Calibre (0.8.24) sends to the SD card of my T1. I started a previous thread in https://www.mobileread.com/forums/sho...d.php?t=155286 because I believed that the cause was more T1 related (and at first placed my suspicions in a particular metadata structure), but now I've come to think that it has more to do with the way Calibre sends data to the SD - or in anycase, with the way the T1 "builds up" its database from that data.

The thing is that, quite often, the thumbnails of many books located in the SD are "corrupted":

If you bring up their "info" page it is blank (i.e., no metadata such as title, author... appears).
When you "open" the book ANOTHER book opens. This only happens occassionally, but is not uncommon.
These "doomed books" (that are often quite readable) never make it to the "Continue reading" position, and are ignored in this respect, no matter how much you read it.

This can be fixed, somehow and in the short term, by deleting the SD card's "books.db" file, but I'm afraid that the next time that books are sent to the SD card they are in risk of having the same problems -at least in my experience.

Has anyone else experienced this? - Can anyone think of a "why"?

ldolse · 10-31-2011, 07:30 AM

It's possible that it's a bug with Calibre's handling of the SD card, but it's equally possible that it's a bug with Sony's handling of the SD card/lots of books. Generally speaking I've never heard of a Sony device that behaves well when you load it up with tons of books. The built in memory already holds far more books than the Sony is realistically capable of cataloguing, so few users ever feel the need to expand to the SD card.

So I'm guessing your making yourself doubly special - using the SD card, and (I'm assuming) loading your Sony with far more books than a typical user. Sorting out whether it's Calibre or Sony might be a bit more difficult.. In the other thread you don't mention how many books your trying to load.

enriquep · 10-31-2011, 07:49 AM

I have the habit of placing my downloaded news & English-language books in my cards - and my Spanish-language books in the main memory. I know it is very silly and absolutely against the rules of logic... just a habit, I've been doing it for years and that's the main reason why I use a card.

Currently I have about 100 books in the main memory and about 80 in the SD (and usually I don't have more than that): a lot more of what I can read in a weekend, definitely, and probably more than the typical user (is it??), but hardly a "stress test" for the device, or so I think!

ldolse · 10-31-2011, 08:36 AM

Agree that 180 books is not a stress test - in some ways it's a good thing, as it should make it easier to allow you to create a set of reproduction steps for a dev. Unfortunately that probably means more work for you if you want to pursue the issue. I don't see in the other thread that you specifically mentioned the Sony software works, though someone else seemed to have inferred it. That would probably be the next thing I'd try myself is seeing if the same problem happens with Sony's own software.

If Sony's software works then the step after that would be to try to create a relatively simple set of reproduction steps using Calibre and a small number of books.

enriquep · 10-31-2011, 09:07 AM

Quote:

Originally Posted by ldolse

Agree that 180 books is not a stress test - in some ways it's a good thing, as it should make it easier to allow you to create a set of reproduction steps for a dev. Unfortunately that probably means more work for you if you want to pursue the issue. I don't see in the other thread that you specifically mentioned the Sony software works, though someone else seemed to have inferred it. That would probably be the next thing I'd try myself is seeing if the same problem happens with Sony's own software.

If Sony's software works then the step after that would be to try to create a relatively simple set of reproduction steps using Calibre and a small number of books.

I follow your analytical mind... and you are completely and absolutely right

I haven't tried Sony software at all (I haven't even installed it for the moment), but I guess I should

and see what happens. It can be (... another) Sony bug, after all.

In any case, and FWIW, these problems have been the rule in my case, and have also appeared inmediatly after formatting the SD card and reloading the epub files "ex novo" from my computer. But yes: that doesn't mean that Calibre is guilty of anything.

OK, I'll try to pursue the issue as-it-should-be and let you all know if I find any interesting conclusions. But, for the moment, and for whatever reason, my "Sony PRS-T1+Calibre+SD card" pack is unusable.

enriquep · 10-31-2011, 04:45 PM

Quote:

Originally Posted by enriquep

OK, I'll try to pursue the issue as-it-should-be and let you all know if I find any interesting conclusions.

Here are some preliminary results of my testing, that seem to point out that something is wrong in the Calibre-SD card combination.

I am using Windows 7 x64.

1. I Install Reader for PC, format my T1 memory and card, and sync:
(a) 100 fully standards-compliant ePubs to the main memory and 40 fully standards-compliant ePubs to the SD. Result: The T1 takes a long time to refresh the thumbnails, but everythings works as expected - except a missing cover/thumbnail. All ebooks display their correct metadata in the "info" page, open and behave correctly.
(b) I now add 10 probably "not fully standards-compliant" (but acceptable) ePubs to the SD. Result: They also behave as expected. No problems.

2. I reformat the T1's memory and SD and start testing with Calibre:
(a) I transfer the same 100 fully standards-compliant ePubs to the main memory, in groups: first I transfer 20 and check that everything is OK, then 40, then 60, then 100. Results: The T1 shows most thumbnails almost instantly (compare with above) and everythings works as expected. All ebooks display their correct metadata in the "info" page, open and behave correctly.
(b) In a second stage I transfer the 40 fully standards-compliant ePubs to the blank SD. Result: The thumbnails display, but with the "corruption" symptoms mentioned in the thread: no information displayed in the "info" page, wrong book opened when tapping on the thumbnail, etc.
(c) Given the circumstances, I do not attempt to transfer the "not fully standards-compliant" ePubs to the SD.

I've tried several other scenarios. For example, if a "brute force" loading of the T1 is attempted (trying to transfer, say, these 100 ePubs to the main memory and these 40 to the SD card in the same go with Calibre) the most likely result is that, apart from (of course) getting errors in the SD files, some of the files in the main memory will appear duplicated; that's why I tried a more gradual approach.

Oh, and (don't ask me why, but) when all these problems appear they can usually be solved deleting the "books.db" files and letting the T1 rebuild them

kovidgoyal · 10-31-2011, 11:27 PM

You should post your findings in this thread so that Kolenka is ore likely to see them:

https://www.mobileread.com/forums/sho...d.php?t=151786

ldolse · 11-01-2011, 03:00 AM

Ah, I had no idea the T1 was that different from the previous Sony's, and hadn't seen that dev thread. Apologies to enriquep for suggesting testing the Sony software first, but it seems like you've generated some decent info to pass over to Kolenka.

enriquep · 11-01-2011, 12:34 PM

Quote:

Originally Posted by ldolse

Ah, I had no idea the T1 was that different from the previous Sony's, and hadn't seen that dev thread. Apologies to enriquep for suggesting testing the Sony software first, but it seems like you've generated some decent info to pass over to Kolenka.

No problem, Idolse. Testing that Sony software does what it basically promises to do seemed to me as a bit boring... but not as unnecessary

enriquep · 11-03-2011, 11:02 AM

Quote:

Originally Posted by kovidgoyal

You should post your findings in this thread so that Kolenka is ore likely to see them:

https://www.mobileread.com/forums/sho...d.php?t=151786

Done. My technical knowledge is limited, but If I can be of further help in this issue, I am willing to help.

For the moment I have removed the SD card, and everything works fine with Calibre 0.8.24 and the T1 while using only the main memory.

10-31-2011, 06:26 AM	#1
enriquep Connoisseur Posts: 52 Karma: 8860 Join Date: Jul 2009 Location: Madrid, Spain, EU Device: Sony PRS-505, Sony PRS-T1, Sony PRS-T3	Calibre (PRS T1) Errors in SD Files? Hello everybody. I think that I am having frequent errors in the books that Calibre (0.8.24) sends to the SD card of my T1. I started a previous thread in https://www.mobileread.com/forums/sho...d.php?t=155286 because I believed that the cause was more T1 related (and at first placed my suspicions in a particular metadata structure), but now I've come to think that it has more to do with the way Calibre sends data to the SD - or in anycase, with the way the T1 "builds up" its database from that data. The thing is that, quite often, the thumbnails of many books located in the SD are "corrupted": If you bring up their "info" page it is blank (i.e., no metadata such as title, author... appears). When you "open" the book ANOTHER book opens. This only happens occassionally, but is not uncommon. These "doomed books" (that are often quite readable) never make it to the "Continue reading" position, and are ignored in this respect, no matter how much you read it. This can be fixed, somehow and in the short term, by deleting the SD card's "books.db" file, but I'm afraid that the next time that books are sent to the SD card they are in risk of having the same problems -at least in my experience. Has anyone else experienced this? - Can anyone think of a "why"?

10-31-2011, 07:49 AM	#3
enriquep Connoisseur Posts: 52 Karma: 8860 Join Date: Jul 2009 Location: Madrid, Spain, EU Device: Sony PRS-505, Sony PRS-T1, Sony PRS-T3	I have the habit of placing my downloaded news & English-language books in my cards - and my Spanish-language books in the main memory. I know it is very silly and absolutely against the rules of logic... just a habit, I've been doing it for years and that's the main reason why I use a card. Currently I have about 100 books in the main memory and about 80 in the SD (and usually I don't have more than that): a lot more of what I can read in a weekend, definitely, and probably more than the typical user (is it??), but hardly a "stress test" for the device, or so I think! Last edited by enriquep; 10-31-2011 at 07:59 AM.

10-31-2011, 08:36 AM	#4
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Agree that 180 books is not a stress test - in some ways it's a good thing, as it should make it easier to allow you to create a set of reproduction steps for a dev. Unfortunately that probably means more work for you if you want to pursue the issue. I don't see in the other thread that you specifically mentioned the Sony software works, though someone else seemed to have inferred it. That would probably be the next thing I'd try myself is seeing if the same problem happens with Sony's own software. If Sony's software works then the step after that would be to try to create a relatively simple set of reproduction steps using Calibre and a small number of books. Last edited by ldolse; 10-31-2011 at 08:39 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Issues with my PRS-505 and Calibre, Errors communicating	Devz	Calibre	2	04-04-2010 12:58 PM
Problem transfering files from Calibre to my Sony PRS-300	nnanna01	Calibre	1	01-23-2010 10:38 AM
calibre - prs-505 - sending to sd corrupts files	ykmm	Sony Reader	5	07-31-2009 06:47 PM
Calibre PRS-505 errors on Win7	angryvegan	Calibre	6	05-14-2009 11:13 AM
Errors in .Lit files	shousa	Workshop	17	01-30-2008 08:38 AM

10-31-2011, 07:30 AM	#2
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	It's possible that it's a bug with Calibre's handling of the SD card, but it's equally possible that it's a bug with Sony's handling of the SD card/lots of books. Generally speaking I've never heard of a Sony device that behaves well when you load it up with tons of books. The built in memory already holds far more books than the Sony is realistically capable of cataloguing, so few users ever feel the need to expand to the SD card. So I'm guessing your making yourself doubly special - using the SD card, and (I'm assuming) loading your Sony with far more books than a typical user. Sorting out whether it's Calibre or Sony might be a bit more difficult.. In the other thread you don't mention how many books your trying to load.

10-31-2011, 11:27 PM	#7
kovidgoyal creator of calibre Posts: 43,853 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You should post your findings in this thread so that Kolenka is ore likely to see them: https://www.mobileread.com/forums/sho...d.php?t=151786

11-01-2011, 03:00 AM	#8
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Ah, I had no idea the T1 was that different from the previous Sony's, and hadn't seen that dev thread. Apologies to enriquep for suggesting testing the Sony software first, but it seems like you've generated some decent info to pass over to Kolenka.