How to safely stop Kindle Touch logging

[Poetcop]

The Kindle Touch logs information about your second-by-second actions while using it, and it has a script on it that has the capability of sending those logs to Amazon. Some people aren't bothered by that, and if that's the case for you, you don't need to read any further. It bothers me, because though I'm a little resigned to my activities being monitored and analyzed when I use the web, I hate the feeling that a company is looking over my shoulder while I'm turning the pages of a favourite book. If you store your highlights and last page viewed in the cloud, then you face the same privacy tradeoff as all cloud computing - essentially, it's fair game for data mining and profiling. But if you have opted out, then you shouldn't have to worry about your private reading time being observed.

So I have done a close examination of logging on the Kindle Touch, and I have answers to the two questions that first motivated me to start learning on this forum: first, how serious is the privacy threat of these logs? And second, how can the logging be stopped? And I mean stopped, since although sabotaging the showlog script, as some past approaches have, would probably stop most automated attempts to transmit the logs, as long as it is accumulating on your device it is still a privacy threat. My method prevents any log information more than 15 minutes old from being stored, and I believe it to be safe. But first, some information about these logs.

WHAT IS LOGGED, AND WHAT IS NOT LOGGED?

There are two continuously updated log files on the Kindle Touch 5.1.0, stored in /var/log, one called "messages" and one called "odotlite". In 5.0.4 there are two others in the same directory, "netlog" and "wpa_supplicant", but those seem to have gone away with the update. Of the four, only "messages" seems to hold information that could be desirable to keep private - of course depending on your personal comfort level. To help you decide, here are things that are logged that are potentially a concern:

• The time, down to the millisecond, of every touch on the device's screen, as well as the type of touch (e.g. tap, swipe).
• Time of opening a book, down to the second, including the book's ASIN (catalogue identifier), if Amazon book, or unique identifier (probably assigned by the device) if not.
• When closing the book, the location you were reading in the book, and the text display settings.
• Activation times of many other little features, such as text-to-speech, turning on wireless, going to sleep etc.

Things that I can confirm are not logged, at least not in /var/log files:

• Not that much metadata about the book that is opened, compared to K3s apparently - not the title, the publisher, or publication date.
• Typing (i.e. passwords) - coordinates of touches are not stored.
• Search queries.
• Text that is highlighted or notes that are added.
• Web browsing (although this is of course stored as application data for the browser, and on Amazon's servers if the web browser goes through a proxy - not sure about that)

My Kindle is not the 3G model, so I was not able to test whether cellphone tower information is logged, as was suggested in an older post. If it is, this is quite a serious privacy threat (since it could be used to determine location, as in the iPhone debacle), and it would be very valuable if someone can confirm this either way. My method should however provide protection against that.

HOW LONG ARE LOG MESSAGES KEPT?

The oldest messages on the device are probably no more than 6 days old. I arrived at this figure by tracking the growth of the log over 24 hours of typical activity, which took up about 41 Kb compressed. Given that the maximum log archive size is 256 Kb (as I'll discuss later), this gives us about 6 days. I got the same estimate a different way, by looking at the snapshot of log archive I took before I started messing around with the logging, and it had almost exactly 6 days worth on it. Yet another way is the serial numbers on the log archives, which also serve as a kind of clock and confirms the 6 days figure. This applies if you use your Kindle like I do, about 15 minutes a day (reading that is, I seem to spend more time lately hacking it!) If you use it more, the logs will cover an even shorter period of time.

Are the logs routinely sent to Amazon? Two reasons to think not: first, people who have captured the packets sent from the Kindle to Amazon have not found any evidence of a file big enough to be a log being transmitted, as in for example this 10 hour capture by PoP:
https://www.mobileread.com/forums/sho...6&postcount=37
Second, a recursive grep did not turn up any scripts that call showlog, which is the script that assembles and transmits the log, except for dm.sh, which corresponds to the dump messages shortcut (which was also yifanlu's finding from the decompiled source). But we can't be 100% sure. One thing is certain: if wireless is kept turned off most of the time, even if all the logs are grabbed as soon as it is turned on there will be no more than 256 Kb (compressed), that is, 6 days worth, that could be transmitted.

In light of these facts, it seems to me that the logging is not designed the way it would be if it was intended to collect information on reading behaviour. My best guess would be that it is mostly there as a developer tool for debugging, and possibly for extreme tech support. Of course, this could easily change with the next update, if Amazon decides that stream of data is worth collecting.

If the 6 days of logging that type of information is more exposure than you want, here's:

HOW TO STOP LOGGING

1. Edit /etc/tinyrot-files.conf, for instance using vi. You may need to run mntroot rw first to be able to save changes.
2. Find the lines that say

/var/log/messages	256	-
/var/log/odotlite	2048	25

and change the first number in the line to be 0:

/var/log/messages	0	-
/var/log/odotlite	0	25

If you are still using 5.0.4 then change the other similar lines too (be careful not to change the type of whitespace, and the returns must be Linux returns, not mac or pc returns. Best to only change the number). Save and quit.

3. If you want to see the results immediately, run the script /usr/sbin/tinyrot. Otherwise tinyrot will run automatically within 15 minutes. Now when you type ;dm into the search box, the oldest log messages should be no more than 15 minutes old.

What this does is to change the maximum archive size (in Kb) of the log files, so that the tinyrot process, which runs every 15 minutes and deletes the logs in /var/log after first archiving them in /var/local/logs, does the work for you. My hope is that the fact that my method only changes the configuration file makes it somewhat more robust both to future updates and to other hacks. More details of how it works, if you're interested, in the appendix, "the life cycle of a log message".


The word "safe" in the post title is of course provisional, since it will take other people's scrutiny and testing to be positive that it's safe, but I feel some confidence, based on two things: first, my careful mapping of the log archiving process, as I describe at the end, and second, that I have had it installed on my own Kindle Touch for 4 weeks now, including 1 week after updating to 5.1.0 (so I can confirm that it survives the update). During that time I haven't experienced any problems, and partition 3 (/var/local) has not grown.

Please let me know if you have any corrections or new information, especially about potential issues it could cause, and I'll keep this updated. The safety is something I'm taking very seriously, with the eventual goal of a "parent-and-partner-safe" fix - privacy shouldn't be a privilege only for the people with technical skills (and our girlfriends and boyfriends don't deserve to have their Kindles bricked because of our well-meaning mods!). I hope that we'll soon have an equivalent of geekmaster's awesome data.tar.gz/RUNME.sh combo for 5.1.0, so that I can make a one-time magic bullet that will even work on unjailbroken KTs. But a little of my motivation is gone now that I think there isn't that much sensitive information being stored, let alone transmitted - for now.

THE LIFECYCLE OF A LOG MESSAGE

1. Log messages are appended to files in /var/log, only "messages" and "odotlite" as of 5.1.0.
2. Every 15 minutes, tinyrot runs, which clears all the log files.
3. However, first the log contents are appended to the end of a gzip file in /var/local/logs, with a filename like messages_00000005_20120319043003.gz. The last part of the name is updated to the current system time.
4. A new log archive file is started after a while, which for most of the log types is measured by the number of times tinyrot has run since starting the last archive file, AGGR_COUNT in tinyrot.conf, which is set to 20. So each log archive file contains log messages collected at 20 15 minute intervals, covering 5 hours if the Kindle was on the whole time.
5. If the total contents of the archives for a log type exceed a certain max size, in the case of "messages" 256 Kb, then the oldest archive file is deleted, repeatedly, until we're back under the max size.

So when the max log archive size is 0, tinyrot will go through and delete all the archive files for that type of log every time it runs, as well as emptying the current log file. It will alternate between the log message "Cleaning up any old logs matching X" and "Removing X to save storage space", but this is harmless, as is the incrementing of the counter files in /var/local/logs at the normal pace.

Thanks to yifanlu, knc1, ebs, and PoP among others! Special thanks to geekmaster, without whom I wouldn't have a Kindle to hack!

[geekmaster]

The time, down to the millisecond, of every touch on the device's screen, as well as the type of touch (e.g. tap, swipe).

I tail that log file to read touch gestures in my scripts. But I only need stuff from the past few seconds.

My Kindle is not the 3G model, so I was not able to test whether cellphone tower information is logged, as was suggested in an older post.

If it is, this is quite a serious privacy threat (since it could be used to determine location, as in the iPhone debacle), and it would be very valuable if someone can confirm this either way. My method should however provide protection against that.

It is almost certainly logged at the cellphone tower anyway, which amazon has access to, so this is really a non-issue with the only solution to leave wireless always turned off (including wifi), or physically remove the 3g modem. Removing modem drivers could be defeated by a firmware update.

I hope that we'll soon have an equivalent of geekmaster's awesome data.tar.gz/RUNME.sh combo for 5.1.0, so that I can make a one-time magic bullet that will even work on unjailbroken KTs.

It works on *MY* 5.1.0 (which is why my "secret" jailbreak method is still secret). Other developers have reported that my data.tar.gz works on 5.1.0 for them too, so what's the problem here?

Special thanks to geekmaster, without whom I wouldn't have a Kindle to hack!

You are welcome!

[Poetcop]

Hey geekmaster, I'm not be following the threads as closely as I should be so forgive me: is it the case that as of 5.1.0, for your data.tar.gz to work it must be placed into the mounted USB drive while the Kindle is in diags mode? I'm hoping to put together a solution that doesn't require any technical steps, such as the use of MfgTool, but only dropping something into the normal USB drive and restarting. Am I mistaken, and we're already there?

[Poetcop]

And I didn't realize that about the cellphone towers - I thought it would just be the cellphone companies that would have access, not Amazon.com. That must mean that's the case for Apple and Google too in their smartphones... ugh...

[knc1]

Originally Posted by Poetcop

And I didn't realize that about the cellphone towers - I thought it would just be the cellphone companies that would have access, not Amazon.com. That must mean that's the case for Apple and Google too in their smartphones... ugh...

Disclaimer: "In the USA"

The "cellphone company" is a service provider, the information available is the property of the client (Amazon, Apple, Google, ...) to whom the service is provided.
In the simplest possible terms:
"He who owns the service contract, owns the service information".

Other countries telecommunications carrier laws might differ from the USA.

[knc1]

Note on the "maintain existing white space" comment in the O.P:

Quote from the /etc/tinyrot-files.conf file:

# NOTE: Separate the filename and maxsize by ONE (1, one, unos)
# tab. No fancy formatting please

Unmentioned, but also true - maintain the correct line endings of the file (use *nix, not Win or Mac line endings).

[geekmaster]

Originally Posted by Poetcop

Hey geekmaster, I'm not be following the threads as closely as I should be so forgive me: is it the case that as of 5.1.0, for your data.tar.gz to work it must be placed into the mounted USB drive while the Kindle is in diags mode? I'm hoping to put together a solution that doesn't require any technical steps, such as the use of MfgTool, but only dropping something into the normal USB drive and restarting. Am I mistaken, and we're already there?

It still works when booting from main (which is 5.1.0), as well as diags (which did not change). Immediately after the 5.1.0 update was released, I and other developers tested that because data.tar.gz still works when booting from main, I should keep my new jailbreak method secret until we need it.

It still calls RUNME.sh if it sees one, so you just need a custom RUNME.sh that calls your installer, or rename your installer to RUNME.sh.

If anybody thinks my "jailbreak and repair" method does not work, they are probably NOT following the instructions provided with it. The instructions say you need to reboot TWICE. The first reboot INSTALLS the payload inside data.tar.gz, and the second reboot RUNS the payload. Also, a RUNME.done file will prevent the payload from running so you need to delete or rename that file to run the payload one time.

"Your information about the demise of my method was premature", to misquote Mark Twain.

[knc1]

The O.P. deals with changing the configuration of the log rotation utility.
I.E: What, when, how to deal with the log files as they accumulate on the machine.

The configuration file for the system logger can be found in the: /etc/syslog-ng/syslog-ng.conf file.

No mysteries there, on a Linux machine (or ask Google this) man syslog-ng.conf will describe the configureation file contents for your study.
as in: http://linux.die.net/man/5/syslog-ng.conf

Quoting snippets of the K3 /etc/syslog-ng/syslog-ng.conf (an open source file) that apply to the O.P:
Note: the formating of these quotes got messed up because I used "quote" rather than "code" tags - so allow for that when reading these please.

The system logger has these sources of messages defined:

source src { file("/proc/kmsg"); unix-stream("/dev/log"); internal(); };
source net { udp(); };

That is how the messages to be sorted/filtered/passed can get to the system logger.

The destinations of the O.P. are defined as:

destination messages {
file("/var/log/messages"
template("$YEAR$MONTH$DAY:$HOUR$MIN$SEC $MSG\n")
template_escape(no)
);
};
destination odot {
file("/var/log/odotlite"
template("$YEAR$MONTH$DAY:$HOUR$MIN$SEC $MSG\n")
template_escape(no)
);
};

As seen in the first quote above, syslog-ng can be on the receiving end of network logging, also it can be on the sending end of network logging:

#destination loghost { udp("loghost" port(999)); };

Which in the copy of the K3 syslog-ng.conf file I am quoting from is "deactivated" by the first character (making this a comment).

The system logger is responsible for filtering the messages, here are the filter quotes for the O.P. logging:

filter f_odot { facility(local6) and match("DeviceMessaging"); };
filter f_msgs { not facility(local6); };

Finally, with all of those things defined, the "what to do with the messages" configuration lines:

log { source(src); filter(f_odot); destination(odot); };
log { source(src); filter(f_msgs); destination(messages); };

If you now read this post from the bottom-up, you can see how messages are being handled in the stock K3 configuration.

This configuration file is 99% the file that is used in Debian/Linux - with a few changes by Amazon.

For your reading pleasure, they left the "stock" message filter in the file:

filter f_messages { level(info..warn)
and not facility(auth, authpriv, mail, news); };

It is just not referenced, Amazon supplied the f_msgs filter instead.
But you will note that "sensitive" information is "normally" excluded from the /var/log/messages file in the "Debian stock" configuration file.

If you really don't want anything to be recorded, changing the destination description to: file("/dev/null" ... would be the "big hammer" way of doing that.

Of course, you can be much more selective than that about how log messages are handled (RTFM).

BIG NOTE:
This only controls messages that are handled by the system logger. This does not prevent an application from sending information "off-Kindle" by some other (more direct) means.

Blocking those paths using the firewall of either the Kindle or your own Router has been already covered in other threads here.

EDIT
The O.P. subject of the "log rotate" utility is more general than just the system logging files. The "log rotate" utility can be configured to handle any file that accumulates over time, not just the system logging file(s).

[Poetcop]

Thanks for the correction geekmaster, glad to hear that the data.tar.gz method still works! I'll put together basic packages to install and uninstall that won't require any technical skills.

And that is super interesting knc1, and not something I would have figured out on my own! It sounds like it is possible to suppress certain kinds of messages in a very granular way. And I see what you're saying: that changes to tinyrot-files.conf could be used to wipe out *any* steadily accumulating file, e.g. that might contain privacy-comprimising info! As an aside, I avoided using the term "rotate" in my explanation, despite the script being called tinyrot, because that has always struck me as a poor, misleading metaphor - it's more like a conveyor belt, whereas "rotate" to me implies that the old logs are going to rotate back in! And thanks for the note on returns in the file, very important! I have added it to the original post.

[knc1]

I don't know where the traditional name for this utility came from.
Probably buried in *nix trivia somewhere.

But your point is correct, it is much more a "log shift" than a "log rotate".

[knc1]

The use of a eMMC device for the storage media in the Kindles brings another level of considerations into this subject.

Under a "spinning rust" storage media, just "deleting" the file name does not disturb the information stored (which is what makes "un-delete" programs possible).
The general solution in this case is a "secure delete" function, which is actually a "secure overwrite" of the data followed by a delete of the name.

Now consider that this is a eMMC device.

It can not "delete" information it can only "erase" information, and even then, only in "erase block" increments.

The visibile "delete" function is provided by the eMMC on-chip controller, which "fakes it".
How it "fakes it" depends on the programming of the micro-controller - which is often a vendor "trade secret".

But to give an example (used in some older, software controlled devices)...

To delete, say 64 bytes -
Look up the erase block that contains those 64 bytes -
Look up a "free" (and already erased) erase block -
Copy (write) all of the first block - except those 64 bytes - to the second block -
Now assign the second block as the storage area for the file with the "deleted" 64 bytes -
Then assign the first block as a "free" (and not yet erased) erase block.

See the problem?
Those 64 deleted bytes still exist in the erase block that is pending being erased (at some later time, because the erase function is very time and power consuming).

When using this type of storage media, trying to use a "secure overwrite" does nothing to the actual data - just moves that erase block (with data intact) to the "pending erase" list.

This makes it possible for someone who gains access to your eMMC device to possibly learn that "erased" information.
Note "access to your eMMC device" because they might have to pull it off the board and use special hardware to defeat the on-chip controller.

This situation is a major concern in the case of SSD devices, and some manufacturers are beginning to provide SSD devices with a command set that allows the system to be sure that stored data is really, truely "gone".

Translation: Don't write anything to an eMMC device that is of a sensitive nature (I.E: write it to /dev/null instead).

[geekmaster]

Originally Posted by knc1

This situation is a major concern in the case of SSD devices, and some manufacturers are beginning to provide SSD devices with a command set that allows the system to be sure that stored data is really, truely "gone".

I have several older SSD drives that do not have Trim, so I have to periodically do a secure erase when they start writing very slowly (write wear-leveling fragmentation) to restore them back to full factory-fresh write speeds. The newer SSD drives with trim support can do that automagically in erase block increments if supported by the OS (such as Win7).

SSD drives that were formatted using the old XP default "spinning rust" parameters have this little problem where all 4K clusters span across erase block boundaries, compounding the problem. It is a good idea to partition and format them on a Win7 box (which uses 1MB alignment) before installing in a WinXP box. Just FYI.

Of course eMMC devices may also have this partition alignment problem if not done correctly. Has anybody verified that lab126 got the FAT partition alignment right for optimal performance and longevity of the eMMC device?

P.S. Many modern "spinning chrome" drives do some amount of write wear leveling to prevent mechanical wear. In addition, bad blocks can get copied and replaced. In both cases, a copy of sensitive data might get left behind. So the classical old way of overwriting with a series of different data patterns for "NSA software secure erase" is no longer reliable, and for SSD drives also contributes unnecessary and useless additional write wear.

[knc1]

Originally Posted by geekmaster

I have several older SSD drives that do not have Trim, so I have to periodically do a secure erase when they start writing very slowly (write wear-leveling fragmentation) to restore them back to full factory-fresh write speeds. The newer SSD drives with trim support can do that automagically in erase block increments if supported by the OS (such as Win7).

Thanks for the reminder.

In a thread about a month ago, we determined that the Kindle eMMC does support the "trim" command (at least according to the documentation).

I don't recall any post reporting if the Kindle kernel driver is using the command.

Anyone want to take bets on if tinyrot is using the command?

Nor have I seen any reports of Win7 having been ported to the Kindle.

[mergen3107]

Has anyone tried this method on Paperwhite 3?
Is it any better than, for example, this or that?