Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Workshop

Notices

Reply
 
Thread Tools Search this Thread
Old 10-15-2015, 03:09 PM   #1
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
Scrambling copyright ebooks to help troubleshoot problems ???

I have a question regarding trying to help users who post on these Forums about a problematic copyright ebook on their ereader of choice.

Does anyone think it would it be useful to have a simple utility which scrambles the contents of a de-DRM'd copyright ebook sufficiently that the Moderators would be happy for that scrambled copy to be attached on MR? The aim being to scramble the text content whilst leaving the ebook structure and styling intact to make troubleshooting the user's actual problem more straightforward for all who try to help?

Any thoughts? Especially regarding what would constitute "enough scrambling" for the Moderators to be happy.

If it's thought to be worth pursuing I could attach a first attempt at a Scrambler utility - for epub, kepub, azw3.

Just FYI ... for my own private purposes I needed to practice using some of the Calibre ebook manipulation coding tools so I chose this as a little project. If it's not worth pursuing, that's OK, I needed the practice anyway In its current state it's not a Calibre plugin (although it easily could be) it's a drag-and-drop the ebook onto a Windows .bat file and a scrambled copy of the book is created on the PC. For the time being I decided to keep scrambled copies well away from the Calibre library.

Some past observations:
When a user posts asking for help with a perceived problem with a copyrighted book there often follows a frustrating sequence of events: e.g.
  • due to ignorance rather than malicious intent, user attaches a copy of the copyrighted material in the Forum. This results in a threatening intervention from a Moderator which probably is quite upsetting for the user, especially if they are new to MR and ebooks.
  • a more experienced user tries to help and asks to see small HTML/CSS extracts to try to diagnose the problem. This can be a lengthy to-and-fro with many misunderstandings, after all, if the user was knowledgeable about HTML/CSS they probably wouldn't be posting for help in the first place!
  • user tries to construct a small sample of the ebook to demonstrate the problem and creates more (and different) errors than exist in the original. This is not helpful to either party.
  • swapping of PMs, email addresses or public links to Cloud storage ensues. Putting copyright material in a public area is not really desirable because there's a risk of having your Cloud account closed.

Last edited by jackie_w; 10-15-2015 at 03:15 PM.
jackie_w is offline   Reply With Quote
Old 10-15-2015, 03:11 PM   #2
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
ScrambleEbook commandline

Purpose:
  • Read a copyrighted EPUB/KEPUB/AZW3 un-DRM'd ebook file and create a copy with scrambled text contents.
  • The aim is to create a book sufficiently mangled to allow a full upload to MobileRead without breach of copyright.
  • This should make it easier for MR members to request and offer help with problematic ebooks.

N.B:
The upgrade to calibre 4.0 required some technical changes to ScrambleEbook. As a result it is no longer available as a standalone python script and so the old attachment to this post has been removed.

Instead, a commandline option has been added to the ScrambleEbook calibre plugin. Those who wish to scramble a book which is present on their OS disk but not in any of their calibre libraries can do the following:
  • Open calibre and install the ScrambleEbook plugin exactly like you would for any other calibre plugin. This is a one-off task and you do not need to have calibre open to do the rest.
  • From the commandline of your OS (this example assumes Windows) use the following one-liner:

    Code:
    "c:\program files\calibre2\calibre-debug" --run-plugin ScrambleEbook path\to\your\book
  • Alternatively, users who prefer to use a Windows .bat file can create one containing these 3 lines:
    Code:
    echo off
    "c:\program files\calibre2\calibre-debug" --run-plugin ScrambleEbook %1
    PAUSE
    Then drag-drop the book to be scrambled onto it.
In both cases the red bit is the absolute path to wherever you have installed calibre on your machine.

Last edited by jackie_w; 12-21-2020 at 07:19 PM. Reason: New instructons for standalone users
jackie_w is offline   Reply With Quote
Old 10-15-2015, 03:24 PM   #3
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 12,119
Karma: 73448614
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
Sounds interesting. If possible leave items like ToC entries / Headings un-scrambled.

Also I presume that rather than scrambling, the text would be replaced by random text (possibly Lorem Ipsum based).


Might also be an idea for a Sigil plugin?
PeterT is offline   Reply With Quote
Old 10-15-2015, 04:51 PM   #4
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
Quote:
Originally Posted by PeterT View Post
Sounds interesting. If possible leave items like ToC entries / Headings un-scrambled.
Thanks for replying, Peter.

Re: the NCX TOC - to scramble or not to scramble is trivial from a technical POV. The question would be what the Moderators think is acceptable. At the moment I've played safe and scrambled.

Re: an inline TOC - this is more difficult as there is no 100% reliable programmatic way to determine whether the page with the most hyperlinks is an inline TOC or a page of Endnotes (or something else entirely). The Eyeball Method is very reliable but I'm not currently considering an interactive utility with endless user settings. I'd rather err on the safe side of scrambling an inline TOC than revealing the Endnotes.

Re: Headings. That might be possible to some extent if only text from <h1>, <h2> etc tags were revealed but, as we all know, many books are not created with logical tags. Once again the Moderators would need to have their say. Until then, when in doubt ... scramble.

Quote:
Originally Posted by PeterT View Post
Also I presume that rather than scrambling, the text would be replaced by random text (possibly Lorem Ipsum based).
No I did mean scrambling. I want the book to have the look&feel of the original as far as possible without being able to read/comprehend any of it. Rather than describe it I've added a sample before/after page in post #2.

Quote:
Originally Posted by PeterT View Post
Might also be an idea for a Sigil plugin?
Perhaps, but only by someone other than me I haven't used Sigil since the Calibre Editor was released.
jackie_w is offline   Reply With Quote
Old 10-15-2015, 05:31 PM   #5
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 12,119
Karma: 73448614
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
So images are mangled too it would seem?

What are you thinking about for embedded fonts?
PeterT is offline   Reply With Quote
Old 10-15-2015, 05:52 PM   #6
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
Quote:
Originally Posted by PeterT View Post
So images are mangled too it would seem?

What are you thinking about for embedded fonts?
I think the images would have to be mangled, or in this case replaced with a dummy image of the correct dimensions/format, as when a book has DRM the image files are often (always???) encrypted, even when there are only 2 included images (a useless generic cover and the Publisher's logo ).

Re: embedded fonts I haven't done anything with this so far - mainly because I'm not clear what the best do-able solution would be. More thinking and Calibre code-digging required.

I usually strip all embedded fonts myself so don't have a lot of experience. Is it possible to de-obfuscate an obfuscated font? If not then I'm not sure why they can't be left as-is.

Non-obfuscated fonts are another matter. Some are OK to leave as-is (e.g. Charis), some are not. The safest legal thing to do would be to remove them all from the scrambled version. I'm sure this is do-able because Calibre Polish and Modify Epub already do it. Although I don't think there's an option to remove some fonts and not others.

Last edited by jackie_w; 10-15-2015 at 07:16 PM. Reason: incorrect info
jackie_w is offline   Reply With Quote
Old 10-15-2015, 11:34 PM   #7
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
I like the idea and that is a very good start. The things I can think of:

- I think I'd like to see the ToC left alone. As this might be where the problem is, regenerating or playing with it might fix the issue. Or maybe not touch it of it is just "Chapter x".

- The fonts are a problem and I haven't a clue on handling them.

- I think it would be safe if the only images are the cover and a publishers logo. Both of these are easily obtainable and "fair use" probably applies. But, a book with a lot of images becomes a problem.
davidfor is offline   Reply With Quote
Old 10-15-2015, 11:54 PM   #8
compurandom
Guru
compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.compurandom ought to be getting tired of karma fortunes by now.
 
Posts: 917
Karma: 417282
Join Date: Jun 2015
Device: kobo aura h2o, kobo forma
If you're going to scramble a book, you should change its title too just to not be confusing. Maybe prepend "scrambled" to it.
compurandom is offline   Reply With Quote
Old 10-16-2015, 12:02 AM   #9
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
ToC is probably fair use.
As is the semantic cover image (easy to pinpoint with a plugin).




Sounds like an interesting and useful plugin.


Though I'd be happy too if moderators took the time to write a more descriptive warning.



My form response for oversized images:
Quote:
Please take a few moments to read our Posting Guidelines, in particular the part about the maximum image size allowed (600x600 pixels). Images that violate those guidelines will be deleted.

Feel free to update your original post with an appropriately-resized image, or use our attachments feature (in the Go Advanced reply mode).

Thanks.
eschwartz is offline   Reply With Quote
Old 10-16-2015, 12:24 AM   #10
MGlitch
Wizard
MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.MGlitch ought to be getting tired of karma fortunes by now.
 
Posts: 2,839
Karma: 22003124
Join Date: Aug 2014
Device: Kobo Forma, Kobo Sage, Kobo Libra 2
Only issue I can see with scrambling is someone reversing it and thus getting whatever books users post for free.

I don't know if it would really be worth their time and energy given the general ease of pirating digital content, but it's an objection I can see the moderators raising. Where as replacing the text of a book with a static block would be impossible to reverse, if you're worried about it not representing the text of the book maybe just generate word lists so there are a few words of each size to throw in the place of the text of the book. Even if the end user knows the entire word list, they'd have a monumental time trying to reconstruct the book from this since 'read' and 'said' could, in theory, both be replaced by 'boat'.
MGlitch is offline   Reply With Quote
Old 10-16-2015, 01:13 AM   #11
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 12,119
Karma: 73448614
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
From looking at the sample, I don't think the scrambling is based on the same letters being transposed; rather it looks like a random character is generated, with the only proviso being that words stay the same length.
PeterT is offline   Reply With Quote
Old 10-16-2015, 04:51 AM   #12
patrik
Guru
patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.patrik ought to be getting tired of karma fortunes by now.
 
Posts: 647
Karma: 4566069
Join Date: Jan 2010
Location: Sweden
Device: Kobo Forma
Does borkify meet the criterias?

https://www.mobileread.com/forums/sho...d.php?t=263156
patrik is offline   Reply With Quote
Old 10-16-2015, 08:25 AM   #13
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
Quote:
Originally Posted by MGlitch View Post
Only issue I can see with scrambling is someone reversing it and thus getting whatever books users post for free.
Quote:
Originally Posted by PeterT View Post
From looking at the sample, I don't think the scrambling is based on the same letters being transposed; rather it looks like a random character is generated, with the only proviso being that words stay the same length.
Just to confirm, there is no "scrambling key", hence no "de-scrambling key" either. Every alpha-character is scrambled independently of every other alpha-character. So unless anyone strongly suspects that the Python random module isn't really random then I don't think that should be a problem, but I don't have any cryptography skills

At the moment punctuation, spaces and digits 0-9 are left unscrambled.

In addition, every alpha-character retains its upper/lower case, so that paragraphs and word-wrapping look "realistic" ... but possibly Klingon-esque.
jackie_w is offline   Reply With Quote
Old 10-16-2015, 08:38 AM   #14
HarryT
eBook Enthusiast
HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.
 
HarryT's Avatar
 
Posts: 85,544
Karma: 93383043
Join Date: Nov 2006
Location: UK
Device: Kindle Oasis 2, iPad Pro 10.5", iPhone 6
Technically, I'm pretty sure that this would still be copyright infringement. You'd be creating what copyright law calls a "derived work" - ie you're creating a new work from the copyrighted original - and you require the permission of the copyright holder to do that. The fact that it's not readable doesn't change that.
HarryT is offline   Reply With Quote
Old 10-16-2015, 08:47 AM   #15
jackie_w
Grand Sorcerer
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 6,171
Karma: 16228536
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
Quote:
Originally Posted by patrik View Post
I wasn't aware of this Sigil plugin, however it's a long time since I used it. Has it been extended to open AZW3 and KEPUB books these days?

IIRC Sigil used to make some fairly significant structural auto-changes during opening. Is this still the case? IMO this is not necessarily helpful when the aim is troubleshooting. Does anyone else have an opinion?
jackie_w is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sony ereader troubleshoot chrisms Sony Reader 3 10-02-2013 02:08 PM
Out of copyright ebooks/writers you'd most like to see digitized? pstjmack Reading Recommendations 18 09-14-2012 08:46 PM
whispersync not working: can anyone help me troubleshoot? rheostaticsfan Amazon Kindle 4 10-27-2011 08:09 AM
troubleshoot battery life kkinser Amazon Kindle 2 04-18-2011 09:05 PM
The copyright issues of fan fiction eBooks Kirok Lounge 33 12-08-2008 06:54 PM


All times are GMT -4. The time now is 03:38 AM.


MobileRead.com is a privately owned, operated and funded community.