Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Kobo Reader > Kobo Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 12-27-2011, 09:23 AM   #1
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
Arrow KT's Google Analytics integration, and how to disable it.

Scroll down for instructions to disable tracking.

Almost definitely true, non-sensational facts about Kobo's use of Google Analytics. tl;dr:
  • KT Usage data is being sent to Google-Analytics (If/when your KT is online and Google Analytics is accessible.)
  • This is not a new part of .16. It's probably been there since launch.
  • Usage data includes stuff such as 'home screen loaded @time', 'book view opened for X minutes @time,' 'device put to sleep @time'
  • If you keep your KT in airplane mode your data is not sent (if you disable it to update over WiFi, some stats probably are sent.)
  • A session lasts until power-off/reboot (or 500 GA actions, no idea if the Kobo then makes a new session or pauses logging.)
  • Unsent data is cleared at each reboot.
  • Randomly assigned user ID's are not kept across sessions. They are consistent within sessions.
  • This is not sending your credit card number, pet's name, side-loaded books, or email address to Google. (You're probably doing that already.)
  • Kobo cannot see your IP or practically associate your actions with your KoboBooks account.
    Google can see, and is undoubtedly logging IP's.
  • Traffic is not 'redirected through Google Analytics'
  • This has nothing to do with the built-in browser.
  • devices.kobo.com is an illusion.

The GAQueue variable in .kobo/Kobo/Kobo eReader.conf stores the Google-Analytics callbacks until the next time you connect the KT to the Internet. The GAQueue is cleared on each reboot.

If you want to decode the entries in the GAQueue variable, go ahead.
The documentation is right here: Most values, GA's cookies, the utms parameter.

Here's a dissection for nerds, of what data is sent/stored:
http://pastie.org/3079390

The KT reports URL's like 'http://devices.kobo.com/ReadingView/Book' to GA. 'devices.kobo.com' doesn't exist. It's not a real subdomain, and not because of some sinister 'hide the evidence' motive either. (If that were the case, the configuration file would be obfuscated and Kobo's CEO would have way more money than he does now.)

Google Analytics provides a handy, free of charge, platform and lots of pre-built tools for monitoring websites. All Kobo has done is made KT send stats to GA as if the device is a website you're browsing. They're using the GA for mobile devices approach described here.

As Lunchtray said, a device tracking usage metrics is not surprising. But I disagree with them about whether stats of the detail they described using for their games (tracking individual option usage, etc.) are necessary, or even that useful.

I'd known about the KT's GA integration for a while, but after just now looking into the implementation, I am concerned by the detail of the stats. I'm happy to see that Kobo has made an effort to avoid much of the highly intrusive, and useless, tracking the a tool like GA affords. They don't tie metrics to accounts and clear UID's and unsent callbacks across sessions (meaning they don't track devices across reboots.)

But aiming to report each time you load the home-screen, or read a book for X minutes nonetheless seems like overkill to me. Their metrics are not super-invasive, and an extremely far cry from stuff like CarrierIQ, or individual menu button tracking.

That said, for me at least, they still do cross the line into 'creepy,' especially since the callbacks are being sent to Google, where those stats will be be saved and analyzed as a part of Google's hoard of personal data. (If it were Kobo infrastructure, I'd still be concerned, but much less so. Kobo isn't a business built on harvesting personal data.)

Marco Arment describes the creepy threshold in mobile analytics better than I could. (49mins in, past all the coffee talk)

How to disable Google Analytics on the Kobo Touch

What this does: it edits the hosts file of the Kobo Touch so www.google-analytics.com is translated to the loopback IP address of your device: 127.0.0.1. This means the device can't talk to Google Analytics, thus breaking stats tracking.

The exact line I'm adding to the /etc/hosts file is '127.0.0.1 www.google-analytics.com ssl.google-analytics.com google-analytics.com' I encourage you to open up the KoboRoot.tgz file and verify this yourself.

Directions:
Save this file in your KT's '.kobo' directory.
Eject & disconnect.
Watch 0.5 seconds of the upgrade animation.
Be happy.


(KoboRoot.tgz is deleted after the upgrade. Don't worry.)

The tweak should persist across upgrades, unless the devs ship a hosts file in an update, but that's very unlikely.

I was able to whip this up thanks to the Kobo's accessible system, research done by Stef70 and the Hacking into the KT & Manually upgrading KT firmware threads. Much love.

Last edited by novou; 01-06-2012 at 07:59 PM. Reason: added 'scroll down' messages
novou is offline   Reply With Quote
Old 12-27-2011, 09:46 AM   #2
GoingDown
Zealot
GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.GoingDown is no ebook tyro.
 
Posts: 114
Karma: 1274
Join Date: Dec 2010
Location: Finland
Device: Kobo Touch
Wow. Thanks for the detailed report. I always adblock those statistics pages on my web browser, so it will be nice to do the same on KT
GoingDown is offline   Reply With Quote
Old 12-27-2011, 10:06 AM   #3
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
I thought about making ad/tracker blocking hosts file for the KT, it wouldn't be hard.
I wanted to ship something simple first, but I might put one together for the KT, unless someone else beats me to it.

The vanilla hosts file is just "127.0.0.1 host localhost.localdomain localhost localhost localhost.localdomain" and I added my line right below.

If you know your way around a *nix/OSX command line:
Code:
# mkdir etc
# editorofchoice etc/hosts
# tar czf KoboRoot.tgz etc/hosts
Then copy KoboRoot.tgz to .kobo. Keeping proper permissions would be ideal, but don't sweat it.
novou is offline   Reply With Quote
Old 12-27-2011, 10:08 AM   #4
CJJ
Zealot
CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.CJJ can program the VCR without an owner's manual.
 
CJJ's Avatar
 
Posts: 121
Karma: 189826
Join Date: Dec 2011
Location: Wisconsin, USA
Device: laptop, iriver Story HD, Galaxy Tab2 10.1
Quote:
"That said, for me at least, they still do cross the line into 'creepy,' especially since the callbacks are being sent to Google, where those stats will be be saved and analyzed as a part of Google's hoard of personal data. (If it were Kobo infrastructure, I'd still be concerned, but much less so. Kobo isn't a business built on harvesting personal data.)"
Pretty sure Kobo is a business built on harvesting personal data. And it seems to be getting worse. Google is no angel but then what corporation isn't mining information to get an edge? Good piece though. Thanks for info.
CJJ is offline   Reply With Quote
Old 12-27-2011, 11:28 AM   #5
Koboyashi
Zealot
Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.
 
Posts: 135
Karma: 207434
Join Date: Dec 2011
Device: Kobo Touch & Glo
Cool! Thank you for clearing up that matter and also for the option to turn off that stuff. The hosts file - of course. My notebook (Windows) already has a 'hardened' hosts file.
Koboyashi is offline   Reply With Quote
Old 12-27-2011, 12:19 PM   #6
Lunchtray
Connoisseur
Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.
 
Posts: 78
Karma: 107708
Join Date: Jul 2011
Device: Kobo Glo, Kobo Touch, Kobo Gen 1
Quote:
Originally Posted by novou View Post
As Lunchtray said, a device tracking usage metrics is not surprising. But I disagree with them about whether stats of the detail they described using for their games (counting button presses!) are necessary, or even that useful.
It's not exactly counting button presses but sometimes yes. For instance if you have an option in your game that allows the player to change languages to any EFIGS and you notice it's never used that might indicate to not translate some games in the future or that you need to highlight it elsewhere in your marketing assets for other countries. There may be a menu button that you want players to use but they don't... what can be done to fix it?

Notice how in the new firmware it's not "Library|Store|Reading Life"? Odds are they noticed that having that configuration wasn't working for them so they put it all under "Home" and are now trying the recommendation engine at the bottom to get people into the store. So, In Kobo's case the answer to the possibly less used store button was to implement "Featured/Recommendations" bar at the bottom to hopefully drive customers to buy more books and get them into the store.

Usually the analytics just grabs everything you can possibly do for a program and sends back the data and the company has to clean it up to understand how the customers are using the products and how to improve it.
Lunchtray is offline   Reply With Quote
Old 12-27-2011, 09:49 PM   #7
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
Quote:
Originally Posted by CJJ View Post
Pretty sure Kobo is a business built on harvesting personal data. And it seems to be getting worse. Google is no angel but then what corporation isn't mining information to get an edge? Good piece though. Thanks for info.
You're correct, they are harvesting data, and to an extent, most corporations are somewhat 'built on harvested data.' I'll reword that part so it's more accurate.

What I meant to say, was that Kobo's business is book sales. They do harvest this data and book buying patterns to optimize their book sales. But it is in service of that aim. Google on the other hand, makes 99% of their money from highly targeted advertising. Personal profiling & data mining is their business. Kobo isn't using the devices' GA metrics to personally profile users, but Google is using them to target and profile individual users. (That's why Google provides GA)

Quote:
Originally Posted by Lunchtray View Post
It's not exactly counting button presses but sometimes yes. For instance if you have an option in your game that allows the player to change languages to any EFIGS and you notice it's never used that might indicate to not translate some games in the future or that you need to highlight it elsewhere in your marketing assets for other countries. There may be a menu button that you want players to use but they don't... what can be done to fix it?

[SNIP]

Usually the analytics just grabs everything you can possibly do for a program and sends back the data and the company has to clean it up to understand how the customers are using the products and how to improve it.
I agree that detailed metrics can be informative, as in the examples you gave, but it's very possible to make a good product without them. Additionally, even if intensive tracking might provide greater insight, there's a limit to what is reasonable, no matter how 'anonymized' or aggregated the statistics are. (I think this is something we both agree on, we just draw the line differently.)

I think Kobo adding an opt-in toggle for usage statistics, with a statement of what they're tracking and the data's lifecycle, would allow them to still get good device data, while giving users a choice.

PS: I updated my post above so it describes your metric use better.

Last edited by novou; 12-27-2011 at 09:55 PM.
novou is offline   Reply With Quote
Old 12-27-2011, 10:35 PM   #8
JD Gumby
Cynical Old Curmudgeon
JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.
 
Posts: 1,085
Karma: 8495224
Join Date: Jul 2011
Location: Halifax, Canada
Device: Kobo Mini, Kobo Arc, HTC Desire C
Quote:
Originally Posted by novou View Post
Directions:
Save this file in your KT's '.kobo' directory.
Eject & disconnect.
Watch 0.5 seconds of the upgrade animation.
Be happy.

The tweak should persist across upgrades, unless the devs ship a hosts file in an update, but that's very unlikely.
Sadly, the file got deleted when I (safely) unplugged my Touch and it went through the upgrade display and restarted itself. Good try, though.
JD Gumby is offline   Reply With Quote
Old 12-27-2011, 10:52 PM   #9
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
JD, it sounds like the procedure worked just as it should. The KoboRoot.tgz file is supposed to disappear. The tweak to the hosts file happens in the KT's underlying system by using the upgrade method.

I've make that more clear in the post.

Last edited by novou; 12-27-2011 at 10:57 PM.
novou is offline   Reply With Quote
Old 12-27-2011, 10:55 PM   #10
JD Gumby
Cynical Old Curmudgeon
JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.JD Gumby ought to be getting tired of karma fortunes by now.
 
Posts: 1,085
Karma: 8495224
Join Date: Jul 2011
Location: Halifax, Canada
Device: Kobo Mini, Kobo Arc, HTC Desire C
Guess there's a hidden partition there that Windows can't spot, eh? (without outside tools, of course)
JD Gumby is offline   Reply With Quote
Old 12-27-2011, 11:02 PM   #11
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
Yeah, read the threads in the bottom of my first post if you're curious.
novou is offline   Reply With Quote
Old 12-28-2011, 09:06 AM   #12
geormes
Zealot
geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.geormes ought to be getting tired of karma fortunes by now.
 
geormes's Avatar
 
Posts: 148
Karma: 216188
Join Date: Jul 2010
Device: Nook Classic, Kobo Touch, Nook Simple Touch, Kobo Aura
Thanks!! Excellent work and another way my Kobo keeps getting "better" long after the initial purchase!!
geormes is offline   Reply With Quote
Old 12-28-2011, 12:25 PM   #13
Lunchtray
Connoisseur
Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.
 
Posts: 78
Karma: 107708
Join Date: Jul 2011
Device: Kobo Glo, Kobo Touch, Kobo Gen 1
Quote:
Originally Posted by novou View Post
I think Kobo adding an opt-in toggle for usage statistics, with a statement of what they're tracking and the data's lifecycle, would allow them to still get good device data, while giving users a choice.
I can't exactly recall but this opt-in is likely embedded withing the End Usage License Agreement for the product. I have a friend who won't use STEAM for this reason.
Lunchtray is offline   Reply With Quote
Old 01-04-2012, 04:27 PM   #14
novou
Pointed Pontification.
novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!novou rocks like Gibraltar!
 
Posts: 64
Karma: 100000
Join Date: Sep 2011
Device: Kobo Touch (Black)
There wasn't a click-wrap EULA on the KT when you load it up. I think all you agree to are the KoboBooks.com terms. I didn't see any specific message of statistics collection when I looked through them, but that's not the whole point.

They should add a clearly labeled & explained toggle in the settings which is disabled by default. As it is, the Kobo tracking defaults to opt-in, with no official, easy to locate means to opt-out. Personally, I believe they should have informed consent for this tracking, which is a standard clearly not met currently.
novou is offline   Reply With Quote
Old 01-04-2012, 06:02 PM   #15
Lunchtray
Connoisseur
Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.Lunchtray is my name, but call me Ishmael.
 
Posts: 78
Karma: 107708
Join Date: Jul 2011
Device: Kobo Glo, Kobo Touch, Kobo Gen 1
Quote:
Originally Posted by novou View Post
There wasn't a click-wrap EULA on the KT when you load it up. I think all you agree to are the KoboBooks.com terms. I didn't see any specific message of statistics collection when I looked through them, but that's not the whole point.

They should add a clearly labeled & explained toggle in the settings which is disabled by default. As it is, the Kobo tracking defaults to opt-in, with no official, easy to locate means to opt-out. Personally, I believe they should have informed consent for this tracking, which is a standard clearly not met currently.
You can find their Terms of use and Privacy Policy on the kobobooks website. http://www.kobobooks.com/termsofuse

By using any of their services on any device (this I believe includes syncing to update firmware or using their firmware files) means you've agreed to and have read their policy. Linked out from the terms of use I found their Privacy Policy that says they do analytics. http://www.kobobooks.com/privacypolicy

Quote:
Originally Posted by Kobo Privacy Policy
1. ACCOUNTABILITY
(a) User Accounts and Personal Information
We may record information about Your usage, such as when and how often You use the Kobo Service as well as information You display or click on within the Kobo Service (including UI elements, settings, and other information), and may share this information to trusted third parties, including Our publishers or other suppliers of content.
I'm sure there's more info there that goes over what and why they're doing it and how everyone has "Techincally" agreed to it.

They're far more transparent that pretty much every single mobile app developer on Android, iPhone, Blackberry, etc. Those all have Google Analytics.

I don't agree with your want for a button to turn off and opt-out by default. They need their data to run their storefront/business as best they can and to improve their UI and products.

If you don't like the terms for a product or service don't use it and choose someone else. In their business most (if not all) companies will be doing the same thing so pick your lesser evil or whichever company you feel more fuzzy about.

Last edited by Lunchtray; 01-04-2012 at 06:13 PM.
Lunchtray is offline   Reply With Quote
Reply

Tags
kobo touch, privacy

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Google Analytics URLs won't validate ccraig13 Sigil 8 07-18-2012 05:17 PM
mouse click tracking or analytics? PChuprina ePub 3 10-30-2010 12:44 AM
Google analytics slowing things down pilotbob Feedback 17 02-11-2009 12:41 PM
Google Books allows integration of book clips into websites TadW News 2 09-07-2007 01:46 PM


All times are GMT -4. The time now is 06:46 PM.


MobileRead.com is a privately owned, operated and funded community.