ePub & PDF decryption on Mac OSX

[epstewart]

Originally Posted by pthwaite

I know exactly how you feel. Been there, been baffled by that & eventually got it figured out.

Mac comes with Python installed so you may not have needed to install the Python, but I don't suppose it will make much difference.

However, what you MUST do is ensure you have the optional development tools installed that come on the setup disks.
Just shove the setup disk in and you should see an option to install other bits and pieces.

Open it up and choose to install the development tools. I installed X11 just to make sure I got everything.

Once you have the tools installed, try re-installing the pycrypto package. It sounds like you've got your head around terminal to get to the correct directory. The pycrypto package also ha a test programme for you to see if it's installed again it's listed in the readme file and that should give you some confidence that you are winning.

Try the above first and see how you get on, I'm by no means the best to give much help on Terminal and unix as I have no experience of unix and associated commands and just do a best guess with the old DOS commands

Howard

Thanks, Howard. I probably should have mentioned in my original post that (based on posts in this very thread) I already installed the Developer tools. At first I hadn't done that, but after I did I reinstalled PyCrypto in the way indicated in my OP -- i.e., to my Desktop.

I'll answer you further in the process of responding to adullday below. Thanks again.

[edembowski]

Originally Posted by epstewart

I would like to get someone's expert help on this. I'm trying to use the ineptepub.py script on my Mac running Leopard 10.5.8, and I have no clue what I'm doing when it comes to Python and Crypto.

pthwaite and adullday are right, you probably had Python installed already, and this confused things.

I obtained the Python 2.6.2 for the Mac package (it seems to be called MacPython) and installed it by double clicking its .mpkg installer, resulting in a Python 2.6 folder in my Applications folder. This folder has a Python Launcher app, an IDLE app, a Build Applet app, an Update Shell Profile command file, and a folder of Extras. I am not sure if the actual Python interpreter is in there???

I just got Python from python.org (it's 2.6.3). Yes, the interpreter is there. Here's a sample of what happens after installation:

lizard:build ed$ which python
/usr/bin/python
lizard:build ed$ which python2.5
/usr/bin/python2.5
lizard:build ed$ which python2.6
/usr/local/bin/python2.6
lizard:build ed$ ls -l /usr/bin/python
lrwxr-xr-x  1 root  wheel  72 Mar 11  2009 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python
lizard:build ed$

That's from a command prompt. What it's showing you is that python 2.5 and 2.6 are both installed. If you navigate to a python script in Finder and hit `get info`, you'll probably see that the default is to open with IDLE. That's Python's default development environment.

...Huh? Does that mean to run those commands in Terminal?

Yes, exactly.

In doing so, I figured I had to use the cd command in Terminal to change my current directory to the pycrypto-2.0.1 folder on my desktop. Then I entered python setup.py build and saw a raft of messages rolling by in my Terminal window. Though there were several "warning" messages, there didn't seem to be any errors. So I entered python setup.py install and got a handful of running ... messages, a lot of copying ... messages, and then:

running install_egg_info
Removing /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Writing /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info

I gather that means Crypto installed properly???

Yes, and pay close attention to the output. It's telling you that it's installing to the 2.6 installation, not the 2.5 (we're OK so far)

But now I try to run my ineptepub.py script -- it's on my desktop
...

I had to use Get Info in the Mac Finder to direct that double clicking on .py files should open Python Launcher rather than IDLE, which seems to be the Python development environment.

That's the default behavior for 2.6.

After doing that, when I double click on ineptepub.py, I see an app called Python try to run the script but instead fail on an error message:

This script requires PyCrypto, which must be installed separately. Read the top-of-script comment for details.

I also saw a new window open up in Terminal, with the contents:

Last login: Mon Oct 19 12:23:16 on ttys002
[iMac:~] eric% cd '/Users/eric/Desktop/' && '/usr/local/bin/pythonw' '/Users/eric/Desktop/ineptepub.py' && echo Exit status: $? && exit 1

OK, so at this point I'm totally lost. I thought I installed PyCrypto, but now I'm being told I need to install PyCrypto.

Can someone please help me get this working? Thanks in advance ...

This is where I think your problem is. The laucher is probably pointing to 2.5, and you have pycrypto in your 2.6 install. Try running it from the command line and see what happens.

edit: from the command line, try running python2.6 not python.

- Ed

[epstewart]

Originally Posted by adullday

Without going through your exact steps, I'm taking a wild guess here, but it looks like the pycrypto library got attached to the built-in python (which is good), but when you double click on the script it's running itself using the python you installed (/usr/local/bin/pythonw).

Try running the script from a fresh Terminal window.
e.g.

/usr/bin/python /Users/eric/Desktop/ineptepub.py

Thanks, adullday. When I ran

/usr/bin/python /Users/eric/Desktop/ineptepub.py

in Terminal I again got the error I mentioned in my first post. When I followed the advice I got from Howard in his post and ran

/usr/bin/python /Users/eric/Desktop/pycrypto-2.0.1/test.py

I got

Traceback (most recent call last):
File "/Users/eric/Desktop/pycrypto-2.0.1/test.py", line 18, in <module>
from Crypto.Util import test
ImportError: No module named Crypto.Util

It seems I have too many Pythons. I used

defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder

in Terminal to make all my files show up in Finder. Then I went into /usr/bin/ and found various items whose names begin with python. Most are aliases. The original of the python alias is

/System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5

Meanwhile, I've been able to discover that Crypto installed itself as

/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/Crypto

So I'm somehow using Python 2.5 instead of 2.6, which is where Crypto is!

I note that 2.5 is in the /System/Library/ hierarchy while 2.6 is in the /Library/ hierarchy.

Can one of you experts tell me how to correct this? Many thanks in advance ...


Eric

[edembowski]

Originally Posted by epstewart

...
So I'm somehow using Python 2.5 instead of 2.6, which is where Crypto is!

I note that 2.5 is in the /System/Library/ hierarchy while 2.6 is in the /Library/ hierarchy.

Can one of you experts tell me how to correct this? Many thanks in advance ...


Eric

run python2.6 instead of python and you'll have it! read my post above!!

- Ed

[pthwaite]

Yes, as edembowski says, if you read his last 2 posts, I think it should work then.
cheers, H

[epstewart]

Originally Posted by edembowski

run python2.6 instead of python and you'll have it! read my post above!!

- Ed

Ed (and the others who have so kindly offered help in this thread),

Thanks, but I am not totally sure what you mean by running python2.6 from the command line. Could you give me the exact syntax I would need to run ineptepub.py from the command line using python2.6 instead of python? Because when I try

[iMac:~] eric% python2.6 /Users/eric/Desktop/ineptepub.py

from the command line, I still get my original error message:

This script requires PyCrypto, which must be installed separately ...

I can tell by inspecting the file usage of the Python process in Activity Monitor while the message is still being displayed that

/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python

is being used, meaning I'm really using the 2.6 version I installed. Entering the where python2.6 command, I see

[iMac:~] eric% where python2.6
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6/usr/local/bin/python2.6

To prove Crypto is installed for Python 2.6:

[iMac:~] eric% ls -l /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
total 16
drwxr-xr-x 11 eric admin 374 Oct 13 16:02 Crypto
-rw-rw-r-- 1 root admin 119 Oct 2 01:58 README
-rw-r--r-- 1 eric admin 629 Oct 19 12:10 pycrypto-2.0.1-py2.6.egg-info

I must be missing something in my understanding, but the best I can tell, my Python 2.6 has the PyCrypto package duly installed in its hierarchy, but still it is unable to find it and use it when I run the ineptepub.py script using python2.6 in Terminal.

Furthermore, when I simply double click on ineptepub.py on my desktop, I get the same error message, Activity Monitor again shows that the Python 2.6 hierarchy is being invoked (not 2.5), and I see

Last login: Tue Oct 20 11:27:32 on ttys001
cd '/Users/eric/Desktop/' && '/usr/local/bin/pythonw' '/Users/eric/Desktop/ineptepub.py' && echo Exit status: $? && exit 1
[iMac:~] eric% cd '/Users/eric/Desktop/' && '/usr/local/bin/pythonw' '/Users/eric/Desktop/ineptepub.py' && echo Exit status: $? && exit 1

appearing in a new Terminal window that opens in response to my double click, but there is no new prompt in that window. Python's icon magically appears in my Dock. When I click on it and then click OK in the Python error-message window, the withheld prompt reappears in the Terminal window that has just opened.

Summarizing: I seem to have Python 2.6 properly installed, and PyCrypto duly installed within its folder hierarchy. When I invoke python2.6 /Users/eric/Desktop/ineptepub.py from the Terminal command line interface or when I double click the ineptepub.py icon on my Desktop, it is the 2.6 version of Python that starts up ... but that version is unable to satisfy the script's desire to use PyCrypto!

Ed and all you other experts: do you have any further things for me to try? Thanks incredibly much in advance ...

[pthwaite]

It does look like your pycrypto install hasn't worked.
I'm afraid I have 10.6 and the structure is slightly different than 10.5, I have

Computers Hard Drive, Library, Python, 2.6, site-packages, Crypto

I did have a similar problem when I tried installing initially, I unzipped the pycrypto package but didn't use all the directories. eg:
When I unzipped it I ended up with:

pycrypto2.0.1, pycrypto2.0.1 and then the files.

I then moved all the files up a level before trying to install pycrypto and it didn't work. Re-created the structure as it unzipped and re-ran the installer and it then worked.

If you can't get the test file to run, it hasn't been installed correctly. Which isn't much help I'm afraid.

Your:
[iMac:~] eric% python2.6 /Users/eric/Desktop/ineptepub.py

is correct in its usage (at least it works for me) but not having 10.5 I cannot remember what I used to have.

[epstewart]

Ed, Howard, et al.:

I went back to square one and deleted /Library/Frameworks/Python.framework/Versions/2.6/ on my system and then reinstalled Python 2.6 and then PyCrypto from their respective downloaded installers ... and everything is working fine now!

I can't be sure of this, but the problem could have had to do with my having renamed one copy of my installed pycrypto-2.0.1 folder, calling it Crypto. I did that because my examining the ineptepub.py script seemed to indicate that it wanted to find PyCrypto in a folder having the name Crypto. I was floundering around trying to see why the ineptepub.py script wasn't working originally, and thought I'd try that. But maybe it wasn't working for some other reason, which I managed to fix, but then it wouldn't work because I had renamed the folder. Before I did the re-installs, I deleted that folder, so the re-installs may have been superfluous.

Anyway, thanks for all your help.

Now that I'm up and running, I'd like to ask for some clarifications about how all this works. First, I note that the original post said to rename my .pyw files and change their extensions to .py. What was that for? What is the significance of the "w" which I had to strip away?

Two, am I correct in saying that I can't run on a Mac the script that creates the adeptkey.der file, that it has to be run under Windows? Why is that? It would seem to be a major drawback, since a lot of Mac users don't have access to Windows. (I happen to be able to run Windows under Parallels Desktop, so I was able to get around this problem.)

Three, in installing PyCrypto, the associated README file suggests there is a need to run python setup.py build and python setup.py install. I find (1) only the latter is needed; (2) after a successful install of Python 2.6, invoking python from the command line in Terminal in fact uses Python 2.6, which is good; (3) I had to include the full path to setup.py, i.e., /Users/eric/Desktop/INEPT/pycrypto-2.0.1/setup.py, by dragging the Finder icon of that file to the Terminal window. Does that square with your understanding?

Also, I'd like to mention my reason for using ineptepub: I read eBooks on my iPhone in the Stanza app, and I'd like to be able to borrow eBooks from the Maryland Digital Consortium, which includes all of Maryland's public libraries. These eBooks come in ADEPT/EPUB format (also ADEPT/PDF, but I'm not interested in that) and accordingly can't be used in Stanza or any other reader app on the iPhone — unless the DRM is stripped off!

Thanks again ...


Eric

[pthwaite]

You can't run the script as it stands to get the adeptkey as it was written to extract it from windows and it is stored differently. See my PM
Howard

[edembowski]

Originally Posted by epstewart

Ed, Howard, et al.:

I went back to square one and deleted /Library/Frameworks/Python.framework/Versions/2.6/ on my system and then reinstalled Python 2.6 and then PyCrypto from their respective downloaded installers ... and everything is working fine now!
...
Anyway, thanks for all your help.

I'm very glad to hear that you're working now!

Now that I'm up and running, I'd like to ask for some clarifications about how all this works. First, I note that the original post said to rename my .pyw files and change their extensions to .py. What was that for? What is the significance of the "w" which I had to strip away?

The .pyw extension stands for 'Python for Windows' and it goes back to Microsoft's old limitation of 3 letter file extensions. The limit is no longer there, but it's kept that way for historical reasons. Macs (and Unix in general) uses .py

Two, am I correct in saying that I can't run on a Mac the script that creates the adeptkey.der file, that it has to be run under Windows? Why is that? It would seem to be a major drawback, since a lot of Mac users don't have access to Windows. (I happen to be able to run Windows under Parallels Desktop, so I was able to get around this problem.)

The key is stored in a plain text file in clear text. Howard's PM should sort you out.

Three, in installing PyCrypto, the associated README file suggests there is a need to run python setup.py build and python setup.py install. I find (1) only the latter is needed;

A dependency for installation is the file to install. The installer knows how to build it, so if you try to install without building, it will build automagically.

(2) after a successful install of Python 2.6, invoking python from the command line in Terminal in fact uses Python 2.6, which is good;

Your mileage may vary. I've done the install and it keeps 2.5 as the default python. Maybe there's a preference pane that controls it, I haven't checked.

(3) I had to include the full path to setup.py, i.e., /Users/eric/Desktop/INEPT/pycrypto-2.0.1/setup.py, by dragging the Finder icon of that file to the Terminal window. Does that square with your understanding?

I've never tried dragging things to the terminal window, it seems fishy to me. From the shell, I just `cd` to the right directory and take it from there.

- Ed

[epstewart]

Ed,

Originally Posted by edembowski

I'm very glad to hear that you're working now!

Thanks.

Originally Posted by edembowski

The .pyw extension stands for 'Python for Windows' and it goes back to Microsoft's old limitation of 3 letter file extensions. The limit is no longer there, but it's kept that way for historical reasons. Macs (and Unix in general) uses .py

Makes sense!

Originally Posted by edembowski

The key is stored in a plain text file in clear text. Howard's PM should sort you out.

It did. I have been PM'ing back and forth with him in an effort to get the necessary commands working in my Terminal shell, and I have also asked adullday for his permission to incorporate the commands in an AppleScript script that I would make available to one and all through a blog post. If all that works out, then the script would be capable of putting an adept_key.der file for the user in whatever folder the Mac user designates.

Originally Posted by edembowski

A dependency for installation is the file to install. The installer knows how to build it, so if you try to install without building, it will build automagically.

So that's why I found doing the build before the install was unnecessary!

Originally Posted by edembowski

Your mileage may vary. I've done the install and it keeps 2.5 as the default python. Maybe there's a preference pane that controls it, I haven't checked.

It may be desirable at some point for someone to figure this out, since it is a possible pitfall for the unwary and uninitiated. I also found that I had to manually change what gets opened for a double-click in Finder on a .py script file: Python Launcher, not IDLE. (For lurkers: this is done by doing Get Info for a .py file and selecting Open with: Python Launcher, and optionally clicking the Change All ... button if you want the behavior to apply to all .py files.) Also, the Preferences pane for Python Launcher needed to be changed to uncheck Run in a terminal window. Otherwise, every time a .py script is double-clicked, Python Launcher is invoked and in turn fires up Terminal needlessly. Of course, if you are using the command-line interface in Terminal and not Finder double-clicks, you can circumvent all such concerns.

Originally Posted by edembowski

I've never tried dragging things to the terminal window, it seems fishy to me. From the shell, I just `cd` to the right directory and take it from there.

Actually, one way to 'cd' to the right directory is to enter 'cd ' and drag the folder for the directory from Finder to the Terminal window. It is a matter of personal preference whether a user wants to pollute the purity of the CLI by doing so, admittedly.

Thanks for all your help, Ed.


Eric

[kjk]

Originally Posted by epstewart

It did. I have been PM'ing back and forth with him in an effort to get the necessary commands working in my Terminal shell, and I have also asked adullday for his permission to incorporate the commands in an AppleScript script that I would make available to one and all through a blog post. If all that works out, then the script would be capable of putting an adept_key.der file for the user in whatever folder the Mac user designates.

This would be awesome

[MrKrueger]

I'm having a problem that I haven't seen mentioned yet. It's probably something on my end not set up right, but..

I've updated Python to 2.6 and installed PyCrypto, and all of that seems to be running fine. But whenever I run the ineptepub script, it goes fine until I press the decrypt button after choosing all of the appropriate files. When I press decrypt, it pops a message saying...

Error: File name in directory "

Exactly like that. I can't seem to figure out where my problem is on this. Any ideas?

[tlrowley]

Wow, I came here to post that exact problem. I can decrypt in my Windows VM. I've verified the .der file is the same whether created on Windows or Mac. I've re-downloaded the files on the Mac (instead of copying from the VM). But nothing works. If I use the epub file that I downloaded in the VM and the .der file I created on the Mac, but do the decrypt on the Mac, it works, but that kind of defeats the purpose in getting this running on the Mac. I'm running python 2.6.1 on Mac OSX 10.6.1.

It seems that files that are downloaded on the Mac never work for me, only the files downloaded from Windows. Looking at the files in a hex editor (one epub downloaded in the VM, one downloaded on the Mac), they differ near the very end and that's where the problem lies. ADE on the Mac will read the file, but the decrypt fails.

Suggestions?

[adullday]

This may or may not be your problem, but I've run into similar issues with several epubs. Since I usually download my files on my Mac, I don't know whether or not this is an issue particular to the files distributed to the Mac. But what the heck... try this and see if it works....

1. Make a copy of your epub and rename the epub extention to zip (you might have to do a cmd-I and change it from the info window).
2. Double click the renamed file so that the MacOS unzips it. Do not use the unzip command in the Terminal application.
   Use pdurrant's script from https://www.mobileread.com/forums/showthread.php?t=55681 (or the zip command from the same link) to recreate the epub.
3. Try the inetepub.py script again.

It appears that there is an issue related to the zlib library, although it may be a problem in the zip file itself since the built-in de-zippifier on Windows seems to have the same issue. If you unzip the epub (which you may know is a zip file with the epub extension) with the MacOS built-in and re-zip it should (hopefully) work fine.