Because sometimes python is not complex enough ….

Recently I had a task to connect to an existing server to get some data via python so I had two options. Handle all connections myself, communicate with the database in python, cross fingers and hope I would not break anything.

ORRR find a way to call C++ from python and use the existing libraries.

I opted to look into the second solution as it gave me a nice opportunity to research something new. The main thing for me was:

1) Easy to understand. (I am not a hardcore c++ coder)
2) No external dependencies. Just standard python and standard C++.
3) My brain should not explode while trying to get this done !!( Very important and very underestimated requirement)

Seeing as there were only a few beginner resources about this online I thought I’d make a step by step tutorial about it now that I kind of figured it out :). I can’t show how I implemented it with existing code but hopefully the example should make you understand how to do the same.

*side note:
seeing as I’m not a C++ expert I’d like to note that my C++ code might not be perfect but hopefully it will help you understand.

If you want to start learning more on C++ I recommend this resource (most corny website name ever but their basic tutorials on c++ are great):
http://www.gametutorials.com/

Their opening line usually is something like: “Talk to me like I am a 3 year old” which is why I’m finally starting to understand C++ 😛

Tools I used:

Python 2.7.3 x64
Visual Studio 2012 (any other version should do as far as I know)
Python tools for visual studio. (Ptvs)

What we will do:

1. Create a c++ and pythonproject.
2. Setup debugging
3. Add some code that can later be called from python.
4. Call the c++ dll with the ctypes module in python

Stage 1 – setup: (Skip it if you know how to setup a visual studio project for a DLL)
1.1)    Open visual studio.
1.2)    Create a new c++ project. Select Empty Project. (I like starting from a clean project)
1.3)    Give it a name and location and click ok.

1.4)    In the Solution Explorer right click on your project and go to “properties”

1.5)    in the Configuration Properties -> General -> Configuration Type set it to “Dynamic Library (.dll)”
1.6)    in the target extension change it to “.dll” instead of “.exe”
*Note that in some tutorials / online resources they refer to this as a .so (shared object / shared library).

Also one thing to recommend.
If you build release and debug the folder locations change. This makes it annoying for python as you have to swap between folder paths.
But you can also just change the build location for the debug and release to be the same folder path.

So change the output directory to:
$(SolutionDir)\dll\

For debug and release.

1.7)    To be consistent with good coding practices lets make a header and cpp file.

Right click the “Header Files” Folder in the solution Explorer and click
“Add -> new Item”
Select the “header file (.h)” and give it a logical name for example:
RingRingPythonCalling.h
And if you want to be tidy create a proper “headers” folder in your project windows folder. Click ok and now you should have a .h file.

Lets do the same in the Source Files but now for a .cpp file
Please make sure that whatever you called your .h file that you call the .cpp file the same.
This will make it easier to manage our files and code in c++.

1.8)    Creating your python project.

1)    Right mouse button click on your solution and goto
Add -> New Project -> select a Python application.

2)    Give it a name and put it in a logical location.
(preferably in the same folder as your c++ project  )
Mine is:  
3)    Now your solution should look something like this.

4)    Now right click the project and click.
Set as StartUp project.

This makes sure that when you press F5 on your keyboard this python file is run. Else it will try to run the C++ file which is not possible as it’s a dll and not an executable 

1.9)    Settings for debugging.

Now in order to be easily able to debug our code in python and c++ at the same time do the following.
1)    Right mouse click on your python project and click properties
2)    Now goto debug and tick”Enable native code debugging”
3)    This now allows the c++ code to be debug able.
4)    Now goto
https://github.com/Microsoft/PTVS/wiki/Symbols-for-Python-mixed-mode-debugging
And download your corresponding pdb files for your python version.
5)    Create a folder called symbols in your python install folder and put the pdb files in there.  

6)    Now goto the menu bar

Tools -> Options
Debugging -> Symbols. .
Click on the folder icon and add the folderpath to the symbols there.

7)    You might also need to click “Load All Symbols” Once for it to work.
8)    Do note that enabling this makes running your python file slower.

So sometimes its recommended to just disable it. Especially once the c++ work is done or you are not making modifications in the c++ file.

Stage 2:

1.0) Creating some example code:
Open your header file by double clicking on it.

Now I wanted to find a use case for this tutorial instead of just doing a hello world.
So the code I have written here would be the basis of doing a c++ matrix or array class but focusing on the communication between c++ and python.

For example. Imagine you are taking skin cluster data from maya and you want to do some special smoothing operations on the skin cluster data.
The data that comes out of maya is usually a list of lists. So per vertex per influence
So on every row you have a vertex. And every column corresponds to every influence.

In python iterating over all these numbers can be quite slow. So if you send the vertex weight information over to c++ and do your math magic there it could be a lot faster.
So keeping this in mind I made this tutorial

In your header file write the following: *note I tried to comment the c++ to the best of my knowledge however I can not guarantee it’s a 100% correct 😀

Now for the cpp file

And last but not least the python file

Now the code should speak for itself and its comments.
As for running it.

Step3: Building the code.

1)    Build the code in Debug or Release

If you press F7 it will build the project into a DLL.
alternatively you can go to Build –> Build solution.
Because a python file doesn’t need to be precompiled visual studio will only compile the DLL.
Scroll down to the Problems and solutions if you run into any problems or errors.

2)    Now press F5.

Seeing as you set the python file to be the startup file it will run the script.
Here are the project files that you can download and test yourself.

PythonCalling

Just running CallingCPP.py if you have python installed should work out of the box. Or you can compile it the dll yourself 🙂

Feel free to post any comments if you run into any problems 

Problems and solutions:

1)    When compiling you get this error:

fatal error C1083: Cannot open include file: ‘RingRingPythonCalling.h’: No such file or directory
a: Check that you wrote the name correctly of the header file.
b: Right mouse button click on your project and go to properties
Goto configuration properties -> c/c++ -> general
At Additional Include directories add the folder where your header file is located.
That should do the trick.

2)    Debugging pops up the “Python Symbols Required” window.

You need to add the python symbols of the correct version.

Click download symbols it will take you to this link.
①    https://github.com/Microsoft/PTVS/wiki/Symbols-for-Python-mixed-mode-debugging
②    MAKE SURE YOU GET THE RIGHT VERSION.
③    To find your version do the following:

Run your python executable.
1.    For me that path is “C:\Python27\python.exe”
2.    Then you will see the python window appear with the version in the header.
(note your window will most likely look black)
Now my version is 2.7.3 so make sure you also download the pdb for your corresponding version. Else the “Python Symbols required” dialog will keep popping up.

3)  Python command window pops up and closes instantly.

This sometimes happens with the symbols messing with the python tools for visual studio.
The easiest workaround is to run the script without debugging.
So instead of pressing F5 press Ctrl-F5 untill you fix your bug.

4)  WindowsError: exception: access violation reading 0x000007FE004148D0

There are a million reasons why you could get this error. However the one I ran into the most is:
①    If you get this error the object is most likely being garbage collected by python.
②    So Make sure you store the object in a variable returning from a c++ function. If your c++ function does not need to return anything just give it a int as a return value.
Additionally. I noticed that there is a more severe underlying bug that I could not find the source of.
①    However I did find a solution. (Though very hacky)
②    Just add a Sleep(0.0001) at the end of the function. As seen in the smoothArray function.
③    My guess is python is doing its own memory management after the function is called and moves the objects around. But the function returns to soon so python cant call the function again. This mainly happens in release and not in debug so it’s a nightmare to find.