[GUI Plugin] ScrambleEbook: Getting help with copyrighted books

[jackie_w]

*** May 15, 2022
Unfortunately, due to issues at home, I will no longer be providing ongoing support for this plugin. Hopefully the latest version will work OK in calibre v5.xx and the upcoming calibre v6.xx
*** Feb 2024: Still working OK in calibre v7.xx
***

Purpose:
This plugin will read an un-DRM'd ebook file (EPUB/KEPUB/AZW3) and create an external copy with scrambled content.

The aim is to create a book sufficiently obscured to allow a full upload to the MobileRead (MR) forums without breach of copyright. It should make it easier for MR members to request and offer help with problematic ebooks.

The book can be either:

• in your calibre library
• on an ereader device whose contents are visible to calibre

N.B: This plugin has been sanctioned by the MR Moderator Team who were instrumental in setting the rules about what must be scrambled and what can remain intact for upload to be permissible. The rules are subject to change if circumstances change.


Background:
No-one likes to see an MR member, especially a brand-new member, being issued with a severe warning from a Moderator because they uploaded a copyrighted book to the forum due to ignorance and desperation for help rather than malicious intent.

In addition, I have seen the frustration which can occur on both sides, when a knowledgeable user tries to talk a novice user through the non-trivial process of creating a short extract of their problematic book which won't fall foul of MR copyright considerations.

Anyone interested can read more in this thread Scrambling copyright ebooks to help troubleshoot problems ??? .


How to install: Preferences - Get plugins to enhance calibre Select Scramble Ebook then press Install

Configuration: None

How to use:

Spoiler:

1. Select a single non-DRM'd book in calibre. Either from:
   • the Library view
   • the Device view of a USB-attached device
2. Select ScrambleEbook from whichever calibre toolbar/menu you added it to
3. Press the Scramble now button. You will see a brief description of what was done in the large box, top-left.
   (See attached image, ScrambleEbook.jpg, below)
4. When scrambling has finished, press the Preview content button to view Original and Scrambled content side-by-side.
   (See attached image, Preview.jpg, below)
   This step is optional, but advisable, just to double-check that the text is well-and-truly obscured.
5. Press Save scrambled book & Exit. You will be prompted to choose the directory/folder name where the scrambled copy of the book will be saved.
   Please note that AZW3 books take a little longer to process than EPUBs because of the necessary AZW3 repacking.
6. Create your MobileRead post explaining what problems you are seeing with the original book and attach the newly-created scrambled ebook file.
7. Hope for a speedy response from a willing and more-knowledgeable helper

Limitations:

1. The selected ebook file must have a filetype of one of the following:
   • .epub
   • .azw3
   • .kepub
   • .kepub.epub (USB-attached Kobo only)
2. There are some languages for which the scrambling algorithm is not effective.
   This currently includes: Chinese, Japanese, Korean, Arabic, but there will be others.
   
   The rule-of-thumb about which languages are 'safe' is that if both of the following are true then it's safe:
   • ALL alpha chars of ALL alphabets used in the book have 2 different unicode versions, an uppercase and a lowercase
   • All numeric chars used in the book are standard 0-9
   You are encouraged to use the included 'Preview content' button to double-check that copyrighted content has been properly obscured before posting to MobileRead.

Acknowledgements:
Thanks to RbnJrg, BetterRed and eschwartz for beta-testing and, as always, to kovidgoyal for help with all things calibre-code-related.

How is the book scrambled?
MR default settings are:

Spoiler:

• Book content:
  - alpha chars - replaced with random choice a-z
  - digits 0-9 - scrambled to another digit
  - UPPER/lower case retained. Paragraphs look "realistic" but meaningless.
• Hyperlink anchors: Any which do not contain any alpha chars are unchanged
• TOC ncx: alpha chars replaced with random choice a-z, digits 0-9 unchanged
• Punctuation, spaces: unchanged
• Images:
  - .jpg .gif .png (including cover image) : all replaced with dummy image of same dimensions/filetype
  - .svg : all replaced with dummy .svg image
• Embedded fonts:
  - non-obfuscated fonts removed
  - obfuscated fonts unchanged
• OPF file: some descriptive metadata text removed (description, calibre-specific items)
• CSS files: unchanged

Version History:

Spoiler:

• Version 0.6.0: 15 May 2022
  • Maintenance upgrade for PyQt6/calibre6 compatibility.
• Version 0.4.5: 02 May 2020
  • Maintenance upgrade for python3 compatibility.
  • Plugin icon updated to be more visible when using calibre's Dark mode.
• Version 0.4.1: 23 Oct 2019 - Preview feature working again for books located on a connected device.
• Version 0.4.0: 4 Oct 2019
  • Updated to work with calibre 4.0
    N.B: If trying to scramble a book physically located on a connected device, rather than in your calibre library, then the Preview feature has been disabled (hopefully temporarily). However, you can still create the scrambled book.
  • Added a command-line option.
• Version 0.0.5: 19 Jan 2017 - Remove extra calibre-specific metadata from EPUB3 books
• Version 0.0.4: 27 Nov 2015 - Initial release

[jackie_w]

For those who would prefer to use a commandline version of ScrambleEbook, please see this post.

[eschwartz]

Excellent work!

I've included this in my quick-fire template for responding to inadvertent copyright-infringing uploaders.

[jackie_w]

Originally Posted by eschwartz

Excellent work!

I've included this in my quick-fire template for responding to inadvertent copyright-infringing uploaders.

Thank you That should help.

[jackie_w]

• Update v0.0.5 - remove extra calibre-specific metadata from EPUB3 books

The update should show up via the calibre Plugin Updater later today.

[JSWolf]

I have a request.

Would it be possible to allow ScrambleeBook to save the scrambled eBook back to the directory the eBook came from and have Calibre create an Original_ePublike it does with Modify ePub? Thanks.

[jackie_w]

Originally Posted by JSWolf

I have a request.

Would it be possible to allow ScrambleeBook to save the scrambled eBook back to the directory the eBook came from and have Calibre create an Original_ePublike it does with Modify ePub? Thanks.

No, I don't think so. I don't want to facilitate storing a scrambled book anywhere near a user's real library in case they end up accidentally destroying their original.

[JSWolf]

Originally Posted by jackie_w

No, I don't think so. I don't want to facilitate storing a scrambled book anywhere near a user's real library in case they end up accidentally destroying their original.

Or it could be done like the ePub Merge plugin and make another database entry. That would work as well.

Either way can be done as an option with the option defaulting to off.

[theducks]

I'll bite. Why do you need a 'file copy' of a book that is unreadable?
AFAIK the only purpose is to allow developers to debug code using one of YOUR books without violating copyright
I don't even think it matters that you sent 2 different scramble runs on the book to others. The tagging should be the same

[jackie_w]

@JSWolf,

This plugin doesn't currently have any config options and I don't see the point of adding config for the purpose you suggest. It's meant to be very easy to use for it's target audience, i.e. those who know very little about ebook internals and need help diagnosing a problem. Why confuse them with an option which, at best, is an accident waiting to happen?

If you really want to store a scrambled book in your calibre library just drag-drop it in there as you would with any other new book.

In a nutshell, I don't consider this request a good use of my free time.

[Marco77]

Hello,
So I've run into this case when the plugin crashes on clicking scramble, the input file not being compliant (missing metadata element in opf)

for child in [e for e in self.eb.opf_xpath('//opf:metadata')[0]]:

(IndexError: list index out of range)

dunno if you're willing to fix this since it's an edge case, but it might be better to add a try except handling just in case things go south unpredictably.

[JSWolf]

Originally Posted by Marco77

Hello,
So I've run into this case when the plugin crashes on clicking scramble, the input file not being compliant (missing metadata element in opf)

for child in [e for e in self.eb.opf_xpath('//opf:metadata')[0]]:

(IndexError: list index out of range)

dunno if you're willing to fix this since it's an edge case, but it might be better to add a try except handling just in case things go south unpredictably.

I would say not going to happen. The plugin works with compliant ePub and if your ePub has errors, then you need to fix them before running the plugin.

[jackie_w]

Originally Posted by Marco77

Hello,
So I've run into this case when the plugin crashes on clicking scramble, the input file not being compliant (missing metadata element in opf)

for child in [e for e in self.eb.opf_xpath('//opf:metadata')[0]]:

(IndexError: list index out of range)

dunno if you're willing to fix this since it's an edge case, but it might be better to add a try except handling just in case things go south unpredictably.

Hmm. I think you might have a good point. My notes say that this particular bit of code was added to remove metadata cruft from Amazon books. If there's none to remove then the plugin really shouldn't fall in a heap. I've made an assumption that an OPF file always has a <metadata> section and perhaps I shouldn't have. Does calibre's CheckBook and/or epubcheck flag errors with this book?

I'm due to release an update soon to prepare for calibre's python2 to python3 update later this year so I'll have a look at this before release.

If you're willing I'd really like to have a look at this book myself in case it triggers other unexpected errors. Would it be possible to PM me a link?

[Marco77]

Originally Posted by jackie_w

If you're willing I'd really like to have a look at this book myself in case it triggers other unexpected errors. Would it be possible to PM me a link?

Nay, it was just a book whose opf was manually edited through the ebook editor. I wasn't aware <metadata> is a mandatory section. BTW it seems calibre itself embeds metadata prior to editing the book and it also expects the element to exist as well (also raises an exception when it doesn't)

The suggested advice was rather not abruptly closing the window/dialog when an error occurs, but rather report it so the end user can either seek help or fix it themselves.

[jackie_w]

Originally Posted by Marco77

Nay, it was just a book whose opf was manually edited through the ebook editor. I wasn't aware <metadata> is a mandatory section. BTW it seems calibre itself embeds metadata prior to editing the book and it also expects the element to exist as well (also raises an exception when it doesn't)

The suggested advice was rather not abruptly closing the window/dialog when an error occurs, but rather report it so the end user can either seek help or fix it themselves.

I don't think it's going to be possible to fix this satisfactorily. I can stop the plugin crashing at the point you highlighted but parts of the underlying base calibre code expect an OPF <metadata> section to be present when saving the scrambled book. I think when a book is this badly broken it will be necessary to do some initial troubleshooting in the Editor with Check Book which will show error details and provide optional auto-fixes.

BTW, in case it helps, the automatic metadata embedding done by the Editor can be enabled/disabled as you choose in the Editor's own preferences.