Calibre 7.6 question

[Amalthia]

In the past when I'd upload ebooks to my device and disconnected it would only take about 5 minutes to process the new ebooks.

Last night I added 4 new ebooks to my Sony PRS-505 and it took well over 30 minutes to process the additions to my device. (which is something I expect after I reboot my device after it crashes)

I'm still under the 1000 ebook mark on my device and it's only started doing this after the upgrade to 7.0...at least from what I could tell.

I'd appreciate any advice or input on this matter,

thanks!

[chaley]

Do you have standard firmware on the device?

I ask because one reason the delay happens when calibre and the Sony disagree about when a book has been modified. If calibre supplies a date different from what the Sony SW expects, such as the fields being in a different order, then the Sony will rebuild the database from the metadata in the books.

One way to help debug the problem (if you are willing) is to do the following:
1. Ensure calibre is not running.
2. Plug in the device.
3. Assuming the books are in main memory, go to the device's disk, then to the folder 'database/cache'. There will be a file media.xml in that folder.
4. Copy the file media.xml to somewhere on your hard disk. Name it media.xml.before.
5. Start calibre, send a book, stop calibre, but don't eject your device.
6. Copy the current media.xml file to your hard disk. Name it media.xml.after.
7. Unplug the sony. If it takes a long time, then we might have enough information in these files to understand what is happening. If it takes the normal time, then who knows what is going on.

Assuming it took a long time, you now have a choice. The most useful would be for you can send me these two files, but if you do, I will know the contents of your library. That may or may not create a privacy problem (I can understand if it does). If it does not, then send them to me by email to calibre-bugs at charles dot haleys dot name.

If you are concerned about privacy, then you will need to examine the files yourself, looking for differences. Tell me what you find. Be as precise as you can be.

[Amalthia]

Originally Posted by chaley

Do you have standard firmware on the device?

I'm rather sure I have the most up-to-date firmeware. I do still use the Sony Library software and I get a popup box when there are new updates.

One way to help debug the problem (if you are willing) is to do the following:
1. Ensure calibre is not running.
2. Plug in the device.
3. Assuming the books are in main memory, go to the device's disk, then to the folder 'database/cache'. There will be a file media.xml in that folder.
4. Copy the file media.xml to somewhere on your hard disk. Name it media.xml.before.
5. Start calibre, send a book, stop calibre, but don't eject your device.
6. Copy the current media.xml file to your hard disk. Name it media.xml.after.
7. Unplug the sony. If it takes a long time, then we might have enough information in these files to understand what is happening. If it takes the normal time, then who knows what is going on.

Assuming it took a long time, you now have a choice. The most useful would be for you can send me these two files, but if you do, I will know the contents of your library. That may or may not create a privacy problem (I can understand if it does). If it does not, then send them to me by email to calibre-bugs at charles dot haleys dot name.

I'll follow these directions tonight or tomorrow. I will send you the files because you know what to look for and I don't really have a clue.

Most of my ebooks are on a SD card. I'm not sure if that'll mess up your directions?

Thank you for your help. This is something that's been driving me a bit nuts lately.

[chaley]

Originally Posted by Amalthia

Most of my ebooks are on a SD card. I'm not sure if that'll mess up your directions?

It does change the name of the .xml file. On cards, it is named Sony Reader/database/cache.xml. The rest of the instructions stay the same.

Thank you for your help. This is something that's been driving me a bit nuts lately.

You are welcome. I hope that I can find whatever the problem is.

[Amalthia]

Originally Posted by chaley

It does change the name of the .xml file. On cards, it is named Sony Reader/database/cache.xml. The rest of the instructions stay the same.
You are welcome. I hope that I can find whatever the problem is.

Here are the two xml files.

Edit: messed up on the attachments.

[chaley]

Thanks for the files. The problem is clear. The solution is not.

The Sony SW is adjusting the modification time of the file back 8 hours, which is consistent with an adjustment from AKDT to GMT. My problem is that I have no idea how the Sony knows what zone it is in, so I can't compensate for the adjustment. My experiments with my 300 indicate that it does not know the timezone. Timezone information is not stored in the times of the files (FAT and FAT32 file systems do not store timezone). It seems that your 505 does know where it is. The question is "how?"

Some questions:

1) When you set the clock on your 505, do you tell it what timezone you are in?

2) Does the same problem (slow after unplug) happen if you send to main memory? Or is it a problem only on the card?

3) Do you use Sony's Reader SW? It could be that it 'secretly' provides timezone information to the 505. I strongly suspect that this is the channel by which the 505 knows.

4) If you look at the books on your reader with a file explorer, do you see times mostly in the afternoon & evening, or mostly in the early AM? One specific file to check is 'Ontogenesis/Desynchronization - Ontogenesis.lrf'. Do you see either of the dates "Mon, 05 Apr 2010 03:39:32 GMT" or "Sun, 04 Apr 2010 19:39:32 GMT"?

[Lady Fitzgerald]

oopsie

[Amalthia]

Originally Posted by chaley

Some questions:

1) When you set the clock on your 505, do you tell it what timezone you are in?

I don't recall ever setting the time zone for the 505? I checked and it seems to be matching up with the time on my computer.

2) Does the same problem (slow after unplug) happen if you send to main memory? Or is it a problem only on the card?

This is something I'd have to test.

3) Do you use Sony's Reader SW? It could be that it 'secretly' provides timezone information to the 505. I strongly suspect that this is the channel by which the 505 knows.

I use Calibre to transfer everything to my device, however I do have the Sony Reader software installed on my computer and I use it to check my converted LRF and epub files.

4) If you look at the books on your reader with a file explorer, do you see times mostly in the afternoon & evening, or mostly in the early AM? One specific file to check is 'Ontogenesis/Desynchronization - Ontogenesis.lrf'. Do you see either of the dates "Mon, 05 Apr 2010 03:39:32 GMT" or "Sun, 04 Apr 2010 19:39:32 GMT"?

Everything is pretty much in the PM from what I can see.

What would happen if I just deleted everything off of my device? Would that clear the cache??? or fix anything?

[tponzo]

I am having the same problem on my 900. After upgrading to 0.7.6 and now 0.7.7 it takes forever for my reader to finish loading new info. I put new books in calibre and sent them to my device. After calibre finished loading the new books I ejected the reader and unplugged the USB cable. My reader was still showing "loading" almost a half hour later. I thought something was wrong. Also since 0.7.7 calibre crashed when I was sending books to my device. (3 times)

thanks
Tonip.

[kovidgoyal]

@tonyp: When you say calibre crashed you mean the whole program? And were you sending PDF files?

[kovidgoyal]

@Amalthia: Go to add/remove programs look for an entry titled sony drivers and uninstall it. Thta should prevent the sony library from talking to your device, but will still allow you to use it for viewing books.

[Amalthia]

Originally Posted by kovidgoyal

@Amalthia: Go to add/remove programs look for an entry titled sony drivers and uninstall it. Thta should prevent the sony library from talking to your device, but will still allow you to use it for viewing books.

I have Windows 7 and I looked for Sony Drivers in my control panel add/remove programs area and it's not there.

[tponzo]

Originally Posted by kovidgoyal

@tonyp: When you say calibre crashed you mean the whole program? And were you sending PDF files?

Files were microsoft reader files. When I sent to device, program asked if I wanted to auto convert for device. I answered yes. Transfer began. After a minute or so program kicked. "unexpected error caused program to close..."

[chaley]

Originally Posted by Amalthia

I have Windows 7 and I looked for Sony Drivers in my control panel add/remove programs area and it's not there.

I also have windows 7, and I don't see the drivers. I suspect that they are built-in.

[chaley]

@Amalthia & tponzo:

If either of you are willing to try an experimental driver that I think fixes the problem, let me know. I know of no reason why this driver would be worse than the one currently in calibre, and it could be a lot better.

To try it, you must be running calibre 0.7.7. If you are, then download the attached zip file and unzip it to get the file sony_cache.py. Copy this file into your calibre installation. If you are using windows, then unless you have changed something, you will copy the file to
C:\Program Files\Calibre2\Lib\site-packages\calibre\devices\prs505\sony_cache.py
There should be a file sony_cache.pyo already in the directory.
If you are using linux or a mac, you will need to do a bit of pattern matching to find where the file goes. If the file sony_cache.py is already there, please save a copy before overwriting it.

Next, run calibre in debug mode so that we can be sure that the new driver is running and to give me some extra information. To do this, open a command box/terminal and type
calibre-debug -g
There will be stuff printed in the command box/terminal. Connect your sony (more stuff is printed), then add one book (yet more stuff is printed). If the new driver is running, buried in the last 'stuff' printed will be two lines of interest that I want you to post back to me. Samples are shown below. The number after DEBUG: will be different, because it is a time. 'localtime' will probably be GMT. I don't need the name of the book.

[ stuff ...]
DEBUG:    7.2 Using localtime TZ for new book database/media/books/...
[ stuff ...]
DEBUG:    7.2 Timezone votes: 0 GMT, 165 LTZ
[ stuff ...]

Unplug the device and see if the time to recover has improved.

Please be sure to delete the added file sony_cache.py when you next upgrade calibre (or right away, if it doesn't help). If you don't, strange things could happen.

Edit: changed the driver. There was a problem in the GMT code (that my 300 doesn't use)