A free Calibre Windows development environment using Visual Studio

[ShellShock]

In this post I give a brief overview of how to use Visual Studio to create a freeware Windows development and debugging environment for Calibre.

1. First set up the Calibre Windows development environment as normal: http://manual.calibre-ebook.com/develop.html.
2. Install the freeware Microsoft Visual Studio Community 2013 edition: http://www.visualstudio.com/products...oper-offers-vs. Click the "Visual Studio" Download button to install.
3. Install the Python Tools for Visual Studio: http://pytools.codeplex.com/. Pay attention to the installation instructions: http://pytools.codeplex.com/wikipage...20Installation. Under step 2, install the CPython interpreter by installing the Python 2.7.8 32 bit Windows Installer.
4. Run Visual Studio 2013.
5. On the menu, chose File, New, Project.... Drill down into Installed, Templates, Other Languages, Python. Select the "From Existing Python code" template. Enter the Name as Calibre, and the Location as your GitHub root, e.g., "C:\Users\user\Documents\GitHub". Solution should be set to "Create new solution", and Solution name to "Calibre". Tick "Create directory for solution". Click OK.
6. In the "Create New Project from Existing Python Code" wizard, set the folder containing your Python code to your Calibre source directory, e.g., C:\Users\user\Documents\GitHub\calibre\src. This must be the same location as your CALIBRE_DEVELOP_FROM environment variable. Click Next.
7. Set the Python interpreter to use as 2.7. Click Next.
8. Click Finish.

You will now have a project hierarchy that contains all the Calibre source code, making it easy to navigate the various folders and python files, edit the code etc.

Visual Studio 2013 has GitHub integration for source control. Yay! It is explained in detail here http://msdn.microsoft.com/en-us/library/hh850445.aspx, especially the section "Put an existing solution under local Git version control".

You cannot build the Calibre project you have just created, but that is not necessary because you can install the latest Calibre release, and run that from source, which will use the Python files in your Calibre project, as defined by the CALIBRE_DEVELOP_FROM environment variable.

So to debug Calibre:

1. Make sure the CALIBRE_DEVELOP_FROM environment variable is set correctly.
2. Run Calibre as normal.
3. With the Calibre project open in Visual Studio, click the DEBUG menu, Attach to process, and select the calibre.exe. Make sure "Attach to" is set to "Python code" (use the Select button to change if necessary). Click Attach.
4. Visual Studio will attach to the calibre process, using the Python debugger.
5. You can now set breakpoints, step through code, inspect/watch variables, etc.

An optional extra: a simpler alternative way to attach the debugger to Calibre. Normally we use F5 in Visual Studio to debug. This builds the solution and launches it in the debugger. This is not possible with the Calibre project we created...unless:

1. Download the attached CalibreDebug.zip, and unzip to your GitHub root, to create a CalibreDebug sub-directory.
2. Add the CalibreDebug project to your existing Calibre solution: in Solution Explorer, right click "Solution Calibre", choose Add, Existing project, and select the CalibreDebug.csproj.
3. In Solution Explorer, Right click on CalibreDebug, and click "Set as StartUp Project".
4. In Solution Explorer, Right click on CalibreDebug, and select Properties.
5. Select the Debug tab. Set the Command line arguments to
   
   

   "C:\Program Files\Calibre\calibre.exe" "CALIBRE_DEVELOP_FROM=C:\Users\user\Documents\GitHub\calibre\src"

   Change the paths to the correct values for your environment.

Now when you hit F5, the CalibreDebug project builds and launches in the debugger. CalibreDebug is a small C# project that does two things:

1. Launches the calibre.exe you specified in the command line arguments, with the correct CALIBRE_DEVELOP_FROM environment variable.
2. Attaches the Python debugger in the current Visual Studio session to the calibre process.

In theory it should be possible to mix Python debugging with C/C++, so we could step from Python code into the C/C++ libraries that Calibre uses, but I have not got this working yet.

I hope you find this post useful.

[kovidgoyal]

That should come in handy for people that like to use Visual Studio, I have added a link to this thread in the User Manual's development section.

[davidfor]

Very, very nice. I have so missed being able to step through the code.

[ShellShock]

It is also possible to debug plugins using the Visual Studio environment I described in the first post.

1. In your GitHub root, create a plugins sub-directory.
2. Put the plugin source code into a sub-directory of plugins, e.g., C:\Users\user\Documents\GitHub\plugins\iOS reader applications.
3. In Visual Studio, with the Calibre solution open, right click the Calibre solution in Solution Explorer, and select Add, New Project.
4. Add new Python project, From Existing Python Code. Enter the location as e.g., C:\Users\user\Documents\GitHub\plugins\iOS reader applications.
5. "Enter or browse to the folder..." should be set to the plugin source code folder, e.g., C:\Users\user\Documents\GitHub\plugins\iOS reader applications. Click finish.

This will create a new solution with the new Python project in it. This seems to be a bug in the wizard - I was expecting it to add the new project to the existing Calibre solution. Not to worry.

1. Close the new solution and open the Calibre solution.
2. Click the Calibre solution in Solution Explorer, and choose Add, Existing Project.
3. Navigate to the new pyproj file, e.g., iosra.pyproj and select it.
4. You can delete the redundant *.sln and v12.suo solution files that were generated for the new project, unless you want to debug the plugin without the rest of Calibre.

[dny238]

Not sure why I'm having such trouble, I did find this article helpful with the following additional hacks..

I modified my Microsoft Visual Studio install to include:

*Python Tools
*Universal Windows App Development Tools
*Common Tools for Visual C++ 2015

Since I'm running 32bit, I had to run (there's a 64bit version)
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\vcvars32.bat

I then edited the:
C:\Python27\Lib\distutils\msvc9compiler.py to fix a 'can't find the vcvarsall.bat' error.

def query_vcvarsall(version, arch="x86"):
"""Launch vcvarsall.bat and read the settings from its environment
"""
#vcvarsall = find_vcvarsall(version)
vcvarsall = r'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat'

I added these to my PATH variable in the 'Advanced System Properties'

C:\Python27\;c:\Qt\5.6\msvc2015\bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin

I had to install PyQT5 from source, since the installers online didn't work and were for newer python 3.3:

Read this
http://blog.abstractfactory.io/pyqt5...or-python-2-7/

I had to disable the build of GtNfc as it was crashing nmake. no ill effects so far.

nmake configure.py --disable QtNfc

I then added a PYTHONPATH environmental variable with:

C:\Python27;C:\Python27\Lib\site-packages\PyQt5

Hope this helps someone else.
Rob


References:
http://blog.abstractfactory.io/pyqt5...or-python-2-7/
https://bugreports.qt.io/browse/QTBUG-50191
http://stackoverflow.com/questions/3...tudio-2015-v-1
https://www.riverbankcomputing.com/p...ch/037061.html

[harmtemolder]

Unfortunately I haven’t gotten this to work with “Attach using Process ID” in VSCodium on macOS Catalina (10.15.7). This is the error I keep getting from VS:

Timed out waiting for debug server to connect

If anyone encounters the same on macOS, see if my question on Stack Overflow has been answered by then.

[lastcent]

Hello, I wonder if debugging this way would be possible to see the way the different supported devices (ereaders) store their ebook files. For example, my kindle stores their mobi files inside "documents" main folder. I guess not, but in case, many thanks.