Troubleshooting Huge & random delay when taking notes

[Algiedi]

Hello everyone,

I've been having an issue with my relatively recent Kindle 3, and I'm wondering whether it's a normal limitation or some kind of defective-ness.

Lately I've been proofreading a translator's text, and taking a lot of notes.
What happens is that my Kindle sometimes freezes for up to one minute when saving a note.
This seems to happen somewhat randomly. It can happen on note 1 or note 97, just once or 15 times in a row, on small or large texts... There's no discernable pattern to this lag - it even seems to happen less when the battery is close to being empty!

Has anyone had this issue before, and is it something that I should take up with Amazon's customer service?

[Sir Alex]

@Algiedi,
Some users reported this as problem with Content Indexing. Try to disable indexing.

[DreamWriter]

I have had this problem occur quite often, and I've read that others have as well. In my case, I have 3G/wireless turned off and there is no indexing taking place when it happens. It seems to occur randomly, but it may happen more often when I make more than one note in a short period of time. I don't know that there's any solution. In general, I have noticed that my Kindle is sometimes slower than it is at other times.

I have restarted the Kindle in the past, hoping that would help, but the slow notes problem returns. If you want to try a restart, one way to do that is (from the Home screen) press Menu, select "Settings", press Menu again, and then select "Restart". It will take some time, and then the Kindle will reboot. You can also restart by holding the on/off switch to the right for about 30 seconds (sometimes longer).

I have over 500 books on my Kindle. Do you have a lot of books on yours?

I'd be interested to know if anyone's found a solution to this problem.

[tomsem]

How big is your "My Clippings" file? the bigger it gets, the slower it will be to add notes. There are two reasons: 1) Kindle does not keep the file open, each time you add a note, it has to start at the beginning and seek for the end of file before it can append the new note. The seek will take slightly longer and longer as the file grows. 2) whenever you add a note, Kindle has to index the 'my clippings' file again. Kindle cannot modify 'my clippings' until indexing is finished, as this requires exclusive, write access. Again, the larger the 'my clippings' file, the longer it takes to index the file after each note is added, and the longer the wait for write access.

The 'randomness' is due to the fact that sometimes it has finished indexing by the time you add the next note, and sometimes it has not.

The solution is to periodically backup your existing clippings file (whenever it starts feeling slow) and start a new one. You could even just rename 'My Clippings.txt' (e.g to 'My Clippings 20110517.txt') and still be able to read it on Kindle as a text file. Kindle will create a new 'My Clippings.txt' to hold all subsequent notes you make.

I can think of several ways Amazon could address this issue, but they all have drawbacks. For example, they could automatically 'archive' My Clippings when it got to a certain size (again, renaming the existing file and starting a new one). But then when one wanted to extract something to use on the computer, you'd have several files to look through instead of just the one.

Or (my favorite) they could get rid of My Clippings (and thus its 'overhead' in terms of processor and storage) altogether and add an 'Export notes and highlights' option to each item. However they'd also need to figure out how to index your notes and highlights (as stored in the binary sidecar files) so they could still show up in search results. (hmm and will they ever bother to index PDF files so they can show up in search results?)

Amazon usually seems to favor brain-dead simplicity whenever the alternative is to add additional UI and 'options'. The only thing that might influence them is the cost of support calls generated by people calling with this particular performance issue. Since most Kindle users are not copious note-takers (I assume) this is probably not an issue for them, however.

[hrynkiw]

Yup, I've noticed the same problem on my K3.

Tomsem said:

1) Kindle does not keep the file open, each time you add a note, it has to start at the beginning and seek for the end of file before it can append the new note. The seek will take slightly longer and longer as the file grows.

Nope, I don't think that's it. The delays don't get progressively longer. First it'll take a long time to save, then another note a few minutes later will save immediately. And a third note a few minutes after that will take a long time (or short).

Tomsem again:

2) whenever you add a note, Kindle has to index the 'my clippings' file again. Kindle cannot modify 'my clippings' until indexing is finished,

Okay, now this sounds like a reasonable explanation for what might be happening. Although then I would expect the first note I make to save quickly and a subsequent note made shortly after to take a long time -- and this isn't always the case.

And Tomsem again:

The solution is to periodically backup your existing clippings file (whenever it starts feeling slow) and start a new one. You could even just rename 'My Clippings.txt' (e.g to 'My Clippings 20110517.txt') and still be able to read it on Kindle as a text file. Kindle will create a new 'My Clippings.txt' to hold all subsequent notes you make.

Is this something I can do directly on the K, or only when it's connected to my computer? Is there only one 'My Clippings' file for all the books I've taken notes in? Will I be able to access all the notes from a given book even if I've started a new Clippings file in the meantime?

Hm. I obviously need to learn more about the Clippings file. Is there a reference page/documentation somewhere?

(And while you're answering my questions, what should I make for dinner tonight? )

[sirmaru]

I type all my notes for my Kindle WiFi inside my Kindle 4 PC application. Its easier and encounters no delays at all. I only place the highlights on my Kindle WiFi.

[DreamWriter]

Originally Posted by hrynkiw

Hm. I obviously need to learn more about the Clippings file. Is there a reference page/documentation somewhere?

You can learn about "My Clippings" in Section 3.5 (Annotations and Clippings) in the "Kindle User's Guide," which is loaded on new Kindles. If you need to download a copy of the User's Guide, you can find that here: http://www.amazon.com/gp/help/custom...deId=200317150. In the PDF version of the User's Guide, the "Using Clippings" section is on page 83.

Originally Posted by tomsem

How big is your "My Clippings" file? the bigger it gets, the slower it will be to add notes.

I often delete the "My Clippings" file, so it doesn't usually get very large, but I will pay more attention to that.

[Snowman]

Originally Posted by DreamWriter

I have had this problem occur quite often, and I've read that others have as well. In my case, I have 3G/wireless turned off and there is no indexing taking place when it happens.

From my own observations, it looks as if the *current* open book suddenly starts to re-index just as you hit "enter" on an annotation, and this hangs the k3 just after the save completes. If you wait long enough - about 60 - 120 secs, your k3 will eventually start responding.

I turned off indexing (via the debug option) a couple of months ago, and have made a considerable number of annotations without any repeat of the problem. Debug options do not persist across reboots, so after I have added another batch (shelf?) of books, I reboot to enable the revised collections, then leave overnight to complete the latest indexing run. I then disable indexing again.

Originally Posted by DreamWriter

I have over 500 books on my Kindle. Do you have a lot of books on yours?

Home screen shows 1263 items. Allowing for collections, this represents about 1240 books, the vast majority of which are "legacy" from my old cybook, and generated using mobi-creator. None are drm'd.

[Algiedi]

Oh wow I hadn't seen that so many people had answered. I expected to find this thread on page 10.

I've taken up this issue to the amazon customer service chat, where a delightful fellow with an unpronouncable name explained the whole indexing business to me. I didn't even know about indexing. Eh.

So to test this, I've deleted every book on my Kindle except for the one I was currently reading, and I haven't had that problem yet - so I guess book indexing was indeed to blame.
Which kinda makes sense since the worst lag happened right after I re-imported 300+ books (most of which got duplicated because of title formatting changes) due to a mis-click in Calibre. DERP.

I do however also happen to have a HUMONGOUS clippings file, and I was wondering how to flush it, just in case (pretty much all my notes are proofreading-related, therefore disposable). So thanks a lot to tomsem for that extra bit of very useful trivia.

I would also love to disable indexing, since I never ever ever use search, but my quick research on this revealed that you have to go through jailbreaking, telnet-ing into your Kindle and such terribly complicated things.

EDIT: how do you know that indexing is taking place btw? Is there some kind of icon somewhere that indicates the indexing process explicitely?

[kkinser]

I had the same problem before and posted on it on this forum. Ultimately it is an indexing problem. You can tell Kindle is indexing by search for a random set of characters from the home screen. If indexing is continuing, you'll get a message that says the kindle is waiting for a file. By the way, this also eats a lot of battery, because indexing takes much more power than simply flipping a page. So if you take a lot f notes, you'll notice it.

Fixing it was rather complicated. As previous posters have noted, Kindle indexes the document/book after each note is made, but that is not the same thing as the my clippings file. You can delete that file, but the indexing still remains. A hard reset sometimes corrects the problem temporarily, but I found it to be unreliable. My solution was to delete the indexing file itself, which resides as a hidden file on your kindle. Then I let the kindle re-index everything (make sure you are connected to power). That corrected the problem and it hasn't reappeared in over two months.

[Snowman]

Originally Posted by Algiedi

I would also love to disable indexing, since I never ever ever use search, but my quick research on this revealed that you have to go through jailbreaking, telnet-ing into your Kindle and such terribly complicated things.

You do not have to jailbreak in order to disable indexing. Use the (undocumented) debug option:

• At the home screen, press del to get a text box. You need to do this for each command.
• Type ;debugOn then press enter [Capital O; the text-box will disappear, nothing will appear to have happened]
• Check debug status by typing ~help. This will list the debug commands.
• Enter the command ~disableIndexing [capital I] and enter
• Finally, enter ;debugOff and enter. Check by typing ~help again. If you get a list of commands, then debug is still on, otherwise your k3 will search for this string.
• Debug and disableIndexing do not persist across reboots. If you make a mistake or just get lost, simply do a reboot to get back to normal.

[Algiedi]

Originally Posted by Snowman

You do not have to jailbreak in order to disable indexing. Use the (undocumented) debug option:

• At the home screen, press del to get a text box. You need to do this for each command.
• Type ;debugOn then press enter [Capital O; the text-box will disappear, nothing will appear to have happened]
• Check debug status by typing ~help. This will list the debug commands.
• Enter the command ~disableIndexing [capital I] and enter
• Finally, enter ;debugOff and enter. Check by typing ~help again. If you get a list of commands, then debug is still on, otherwise your k3 will search for this string.
• Debug and disableIndexing do not persist across reboots. If you make a mistake or just get lost, simply do a reboot to get back to normal.

I wish I could bookmark or favorite posts on MR.
Thanks so much, I'm definitely gonna try this!