The intersection of DRM, fair use, and ereader device spying

[JSWolf]

Originally Posted by WillysJeepMan

It IS that simple.

. . . . . . . . .

In a so-called free market, supply and demand help shape the products being sold. If people buy products in spite of the fact that those products don't suit their needs (ie. non-DRM ebooks) then there is no incentive for companies to do anything different.

By purchasing a DRM'ed ebook what you are saying is that the DRM issue has a lower priority than the ebook format. In these instances, the presence of DRM was not enough to prevent you from spending your money.

I dislike DRM'ed media. I'm not a criminal and I don't like being treated like one, and I'm certainly not going to pay for the "privilege" of being treated like one. So I don't buy DRM'ed media.

But unfortunately, many of the people who feel the same way about DRM, buy DRM'ed media anyways and then strip it away... which undermines the power of the consumer.

Having said that, I know that my opinion is in the minority.

They do suit my needs. And stripping the DRM is not a bother. I just do what I would do even if they were DRM free and that is to put them into Calibre. The DRM is gone and I've done nothing different. I then run the Modify ePub script on them and I am then good to begin any fixing up I want to do. It is that simple and it works that well. The DRM does not get in the way. So there or not, the process to deal with my eBooks is exactly the same. The only difference is the directory I pull the eBook out of to put into Calibre.

[spindlegirl]

........................

[bigtext]

Originally Posted by Sydney's Mom

So - although personally I think the odds of Amazon looking at my account for books with DRM removed are low, if the law and/or political mood would change, I don't want to count of the removal of the books from the Amazon cloud as being effective, since that "stuff" remains forever.

Excellent point. Let's pretend this technology was developed in the 1910s instead of today. Senator Joseph McCarthy would certainly be interested in getting a hold of these electronic databases years later. "So you read the communist manifesto back in 1915 and highlighted such and such passages and made these notes." Do we think that our advanced technology makes us immune from the fear-mongering of politicians? Some of the things after 9/11 and leading up to and shortly after the Iraq war got kind of scary. We know that the European countries undergoing fiscal crisis and strain right now have an electorate moving to more extreme political parties including neo-nazis. Imagine the election of a extreme far right religious candidate in a country economically devastated or hit by with a big terrorist attack again. Better hope you don't have Richard Dawkins on your reading list either or books from any other religion.

[WillysJeepMan]

Originally Posted by JSWolf

They do suit my needs.

If you want to support the continued use of DRM on ebooks that's your choice, why complain that ebooks have DRM on them?

[bigtext]

Originally Posted by Elfwreck

It's entirely uncertain how much access Amazon has to sideloaded & personal document reading info. If it wasn't purchased through Amazon, they may not have a way to track it--and more, they have much less right to collect or sell that information; the Kindle TOS gives them rights to certain data related to digital purchases, but its rights to your other data are a lot blurrier.

It would actually be more an effort for Amazon to set up their software to not track sideloaded content. After all think about how sideloaded content behaves. Does it not show up in the library where you select a book on the device? Does it not open when you tap on it? If its showing up in the library that means the device found the file and parsed it's metadata so it could show you the Title and Author of the book. If you open the book to read then it clearly can read and see the file.

As a programmer, why would you write instructions that says "IF sideloaded content THEN don't store metadata into database that syncs to cloud ELSE store metadata into database that syncs to cloud". No, its easier to write a program that just stores everything in the device database regardless of how the book was loaded. If side loaded content displayed in a different library view and reader than items purchased from Amazon then there would be a different story.

Now is there some legal restriction on whether they can track side loaded content? I don't know and I didn't read the TOS? I'm a programmer, not a lawyer.

Originally Posted by Elfwreck

Much of that depends on whether it's illegal to strip DRM for personal use, an issue that has yet to hit the courts. If that's legal, they can easily all be legit.

It can easily "all be legit" and it can easily all be illegal too. Nothing is really decided as far as I know.

Originally Posted by Elfwreck

I suspect that checking the details of non-DRM'd/sideloaded content, even if it's allowed by the TOS, is far too much of a bandwidth drain

No, it's really not a bandwidth drain. Books in themselves are very small in the scheme of a network that now supports transmission of video. And they don't need to upload the entire book, they just need to grab the meta data parts. This metadata will be much smaller than what it takes for them to serve up a single web page when you go to their site.

Originally Posted by Elfwreck

Amazon (nor anyone else) doesn't have the time to put into sorting out what personal content has similar filenames to works in copyright that are currently only distributed with DRM. I don't think any retailer is going to start tracking use of personal documents, just because there's so *much*, and the potential return to them is so little, even if they do have the legal right to read those documents, which I'm not at all sure they do.

Most purchased books (not interested in personal PDF documents that you made yourself) will have a ISBN that looks like this in the OPF: <dc:identifier id="ISBN" opf:scheme="ISBN-HERE">ISBNHERE</dc:identifier>. That's a unique identifier for the book. You also have Title <dc:title> and Author <dc:creator> that in many cases end up being a unique identifier when combined. And for good measure most books have <dc:publisher> also. You talk about filenames, but if you have used Calibre and loaded books into Calibre then you must realize that filenames are irrelevant. You can name an epub whatever you want and Calibre is going to add the book to the library with the appropriate title, author, etc because its reading the OPF not the file name.

And it isn't a matter of having the time. I don't believe anyone actively monitors people's accounts. What happens is that all this information is stored in a database. In general, these databases are permanent and information can be sorted, search, manipulated in any way imaginable at any time in the future. It's very hard for someone who hasn't worked with some of the tools out there to realize how powerful the technology for understanding and manipulating data is.

I don't think Amazon is interested in turning people in for DRM violations. I don't know if they track whether the files are encrypted or not. I think its an easy thing to do if they wanted from a technical perspective. And I don't think tracking such things are a serious resource drain (network usage or storing the information). I think there is more of a concern with a company likes Barnes and Noble that is financially unstable and could be bought by anyone and sold off in pieces. There is also a concern that the book industry equivalent of the RIAA could come in and try to obtain this data if it exists.

[jgaiser]

Originally Posted by WillysJeepMan

If you want to support the continued use of DRM on ebooks that's your choice, why complain that ebooks have DRM on them?

I'm not Jon, but I'll answer anyway.

I don't complain about DRM. DRM is a no-op. It's a waste of time and money for publishers who seem to think that they can make me believe I'm buying a licenses instead of a product. It takes seconds to remove and it's not preventing the pirating of books they firmly believe they're preventing.

If you want to oppose DRM for philosophical reasons or you somehow believe that you're only purchasing a license, be my guest. Get off your high horse about other people's decisions.

[JSWolf]

Originally Posted by WillysJeepMan

If you want to support the continued use of DRM on ebooks that's your choice, why complain that ebooks have DRM on them?

It's not a matter of supporting DRM or not. It's a matter of do I want to read what I want to read or do I have to force myself to read what I may not want to read just because it has no DRM?

Tor has released it's first eBook without DRM. That book is Redshirts by John Scalzi. I've already read it. Have you bought your copy to help support DRM free eBooks? Once Tor fully goes DRM free, are you going to buy more from Tor to help ensure that they keep the DRM off their eBooks?

[AndrewH]

To me, DRM is the digital equivalent of that line in paper books that says, "If you purchased this book without a cover you should be aware that this book is stolen property" or like those tags on mattresses that say, "It is unlawful to remove this tag".

It's actually ok to remove the tag from your mattress, after it's been bought and delivered. It's also ok to remove the cover from your own paper books (though why you'd want to, I've no idea). If it ever comes up in the courts, I'm sure it'll turn out to be ok to remove DRM from your own ebooks... just don't distribute them afterwords.

[Elfwreck]

Originally Posted by bigtext

As a programmer, why would you write instructions that says "IF sideloaded content THEN don't store metadata into database that syncs to cloud ELSE store metadata into database that syncs to cloud".

No, I suspect it's "IF content is not in Amazon's database THEN ignore." I don't think sideloaded content is viewable online nor synced across other devices on the same account; it's only present on the machine it was loaded to. (Please, someone correct me if I'm mistaken. How do Kindles deal with personal documents, either those mailed to its service or those loaded through USB?)

No, its easier to write a program that just stores everything in the device database regardless of how the book was loaded.

Store in device database: yes.
Upload and sync across other devices: no; Amazon is not paying to upload your personal files across their servers.

If side loaded content displayed in a different library view and reader than items purchased from Amazon then there would be a different story.

I believe they are (or can be) stored in different places--a "documents" folder instead of the "books" folder.

And they don't need to upload the entire book, they just need to grab the meta data parts. This metadata will be much smaller than what it takes for them to serve up a single web page when you go to their site.

Metadata from well-formatted ebooks is easy to grab. Metadata from poorly-made ebooks and personal documents is a mess.

Most purchased books (not interested in personal PDF documents that you made yourself)

If they're tracking sideloaded books, they need to deal with those... or they can just assume they're all legit, but that's an awfully big assumption.

will have a ISBN that looks like this in the OPF: <dc:identifier id="ISBN" opf:scheme="ISBN-HERE">ISBNHERE</dc:identifier>. That's a unique identifier for the book. You also have Title <dc:title> and Author <dc:creator> that in many cases end up being a unique identifier when combined.

But those will not tell you if the book was legitimately acquired as a non-DRM freebie (for example, when Tor.com was setting up, it distributed a number of free ebooks as promotional gimmicks), or a review copy, or was acquired while in a nation in which that book is in the public domain. (Not so likely with SOIAF... but there are a couple of nations that don't have copyright laws.)

And for good measure most books have <dcublisher> also. You talk about filenames, but if you have used Calibre and loaded books into Calibre then you must realize that filenames are irrelevant.

So is metadata, in a personally-created ebook. I can set the title, author, ISBN and publisher to anything I want it to say. While this isn't likely to happen through direct Calibre conversions, it *can* happen with Sigil, where a person grabs one ebook to use as a template for others, and doesn't bother to change the metadata.

I don't believe anyone actively monitors people's accounts. What happens is that all this information is stored in a database. In general, these databases are permanent and information can be sorted, search, manipulated in any way imaginable at any time in the future.

Hm. New Discordian project: distribute a swarm of free ebooks with bestseller ISBNs. Declare them to be parodies recognizable by silicon-based life-forms.

It's very hard for someone who hasn't worked with some of the tools out there to realize how powerful the technology for understanding and manipulating data is.

I have worked with enough to know how hard it is to restrict data to *useful* content, unless starting with strict limits like "only track books purchased at Amazon."

I don't think Amazon is interested in turning people in for DRM violations. I don't know if they track whether the files are encrypted or not. I think its an easy thing to do if they wanted from a technical perspective.

I think they could spot DRMd-vs-not-DRM'd content on a Kindle. I'm not sure they could easily track info (like title, author, etc.) about non-DRM'd, sideloaded content, because the formatting options are too great--they'd have to decide what to do with all the documents they can't get info from. I'm very unsure they'd be willing to open themselves up to the liability issues of attempting to tackle those isues.

And that's not a small, side issue to the technical side of things. "We don't track that info so we can't be held liable for getting it wrong, and can't be forced to hand it over" is a BIG reason to avoid it.

I think there is more of a concern with a company likes Barnes and Noble that is financially unstable and could be bought by anyone and sold off in pieces.

B&N has a *sharp* separation between books bought through it, and sideloaded content. They definitely don't track or sync the sideloaded content.

[bigtext]

Originally Posted by Elfwreck

B&N has a *sharp* separation between books bought through it, and sideloaded content. They definitely don't track or sync the sideloaded content.

See MovieBird's comment on the first page of this thread. He claims that B&N is giving him recommmended readings based off his side-loaded content. And I have to agree with him that if B&N is doing, then the much sharper Amazon who pioneered some of these recommendation engines is probably doing it too.

I think we might be getting our wires crossed in a few different areas:

1) I'm really talking about books that you have purchased in another store that came with DRM. Stripping the DRM and converting it to another format will preserve the metadata. You would have to actively make an effort to corrupt the metadata in order to remove the unique identifiers.

2) I don't believe that Amazon is uploading the entire side-loaded book to the cloud. This is what I think happens as it makes the most sense from a design standpoint. The device has a database on it that I will called device_db. What format (sqlite, xml, etc) device_db is in is irrelevant. All books purchased from the store online, emailed to the device, or side-loaded have their meta-data read and stored in device_db. device_db is always synced to the cloud_db whenever you had a change (like a new file side-loaded) and connect via Wifi. device_db is probably the same database the library\reader on the device uses to display your selections.

3) Amazon isn't the biggest problem here in terms of targeting the customer for DRM violations. It's the fact that someone could sue them to acquire the data. A publisher would be in a position to cross reference de-DRMed copies vs their official list of who received them.

* Edit - add part in italics

[jgaiser]

Originally Posted by bigtext

See [URL="https://www.mobileread.com/forums/showpost.php?p=2139746&postcount=14"]2) I don't believe that Amazon is uploading the entire side-loaded book to the cloud. This is what I think happens as it makes the most sense from a design standpoint. The device has a database on it that I will called device_db. What format (sqlite, xml, etc) device_db is in is irrelevant. All books purchased from the store online, emailed to the device, or side-loaded have their meta-data read and stored in device_db. device_db is always synced to the cloud_db whenever you had a change (like a new file side-loaded) and connect via Wifi. device_db is probably the same database the library\reader on the device uses to display your selections.

The paranoia is strong with this one.

Your proof of the above statement is..?? A lot of "I think* in it. I'm more worried about Google's keeping a record of my searches (which by the way, they are) than the *maybe's* of what Amazon is doing.

As Elfwreck has tried to make you understand, the effort to monitor the *millions* of kindles out there is so great that Amazon is not reasonably going expend the time or bandwidth

[Elfwreck]

Originally Posted by bigtext

3) Amazon isn't the biggest problem here in terms of targeting the customer for DRM violations. It's the fact that someone could sue them to acquire the data. A publisher would be in a position to cross reference de-DRMed copies vs their official list of who received them.

There is no list of "people who have content without DRM but don't have the right to it." There is no list of "unauthorized non-DRM copies." There is *maybe* the *possibility* of a list of DRM-stripped copies... but no way to verify if those are from a legitimate purchase on another site, not linked to this account. (Amazon has no way of knowing what I've bought at Fictionwise.) For obscure-fringe situations, there's no way to identify who has a disability that qualifies them to strip the DRM to get access to the TTS functions of a Kindle. And since non-DRM'd copies are user-editable, there's no way to tell which have accurate metadata and which are spoofed.

The collection of this data serves no legal or practical purpose. It's on par with recording the sounds coming out of people's houses and saying, "I suspect they have bootleg DVDs because we heard The Avengers playing, and none of their credit cards show a record of buying it."

While some publishers would *like* to believe that only DRM'd versions of ebooks are legit, and therefore any non-DRM'd copy of a recent bestseller is a sign of piracy, that claim involves (1) DRM removal to be entirely illegal, which it's not, and (2) the ability to identify a non-DRM'd copy of a specific title, which, because of editing abilities, they don't have.

If Amazon or B&N were required under point of lawsuit to investigate a specific user, they could probably turn over a list of the contents of that user's ereader, to the best of their knowledge, which might include data that could be used to affix charges of piracy or copyright infringement. However, to collect and manage that data from everyone would require a *purpose*... and spending the effort to aggregate millions (billions?) of ebook listings from all sorts of sources, while leaving themselves open to liability for the accuracy and maintenance of the data, would be beyond stupid.

"Close enough for marketing purposes" is not the same as "admissible in court." If they guess wrong on marketing, they show a few titles the customer wasn't interested in; shrug. If they submit inaccurate statements to court, they're guilty of perjury or contempt, and possible libel. The data can't be collected at an accurate enough level for lawsuits--not even really ridiculous lawsuits like the RIAA is so fond of.

[Yapyap]

I wonder what would happen if someone was sued for having only DRM-stripped titles (which can, somehow, through metadata for example be proven to be content that is sold only DRM-protected) on their Kindle, and then the person provides proof that these are all books she has bought from other shops e.g. as epubs and then stripped DRM and converted for personal purposes in order to read on one's preferred reader.

It's still a violation of something, yes, but would any court actually find the person guilty of something and punish the person? That would be an interesting case, really.

As for the question of what happens when sideloading stuff ... as far as I can tell, books I've sideloaded to my Kindle via the USB cord, with the Kindle's wifi switched off, only appear on the Kindle, not in my Amazon cloud. If I've either emailed the books to my Kindle or used the Send to Kindle PC application (the latter allows me to specify whether I'll want to save it in the cloud or not), the books will show up in my Personal Documents list under Manage my Kindle and I can also download them from the cloud/archives on for example my phone or iPad, in the Kindle app there. I'd be highly surprised if Amazon didn't have access to the metadata of those "personal documents".

[Lynx-lynx]

I've found this thread to be extremely interesting and informative; and as usual on this Forum, I've learned heaps!

Thank you to all contributors.

[Greg Anos]

I think this whole chain has missed the main point.

Why do you have a reader that has WiFi/3G/4G/ect. ?

Why do you store backup on the Cloud?

You have as much privacy as you are willing to pay for. Sometimes in money, sometimes in extra bother.

I run a Hanlin V3 reader. Old, not particularly popular. It doesn't have Wifi, everything has to be sideloaded. I backup to my sterile (non-internet computer), and keep a SD chip copy of my library off-site.

No worrys about who is watching my metadata, whether from a Big Brother sense, or just marketing 'bots trying to sell me things. Nobody knows what I read, they can only know that I bought it. Useful, but limiting for the marketing 'bots.

If you want privacy, give up your reader connection and the Cloud. It's that simple...

(I live by the rule - "If it can't happen, it won't happen." Call me paranoid if you wish....)