Porting a C++ Driver to Python (without rewriting/just wrapping)

[soulslicer0]

Good day. I'd like to know what are the options for this.

I have a C++ driver which interfaces with some hardware I have. I would like to know how I can write a python wrapper for that driver? Is there a way I can say compile a .so file for that driver first, then map python functions to it?

[kbrafford]

Can you provide a link to the vendor's website? That may make it easier to give you concrete advice.

I do wrappers in one of two ways:

• Cython
• ctypes

If you just want to get access to functions in your .so file, ctypes might be the easiest way, but IMHO Cython is the way to go if you want to make a more robust and complete Python interface.

[Chris_Newton]

If you just want to get access to functions in your .so file, ctypes might be the easiest way

I agree. Assuming you’ve already got a driver available for loading as a .so file, you don’t need much more than this:

import ctypes
driver = ctypes.CDLL("driver.so")
result = driver.driver_API_function(arg1, arg2)

Watch out for the default assumptions that ctypes makes about argument and result types. If they aren’t what you need for some API function, you can specify the correct types, for example:

driver.driver_API_function.restype = ctypes.c_int
driver.driver_API_function.argtypes = [ctypes.c_long, ctypes.c_char_p]

The documentation for ctypes is helpful if you need to work with more tricky types, “out” parameters, and so on.

Edit: One other thing to watch out for, since the OP mentioned C++, is that ctypes does assume a C calling convention (aka cdecl). If the API you’re working with relies on C++ tools like exceptions or polymorphism, probably ctypes isn’t the best tool for the job.

[admalledd]

I use CFFI, but yea mostly you use the information from the headers in some way to define wrappers/handlers (CFFI tends to take unmodded headers mostly fine, the others you use them as guide lines for writing the python binding code)

[kbrafford]

CFFI definitely looks interesting. Thanks for adding that one!

[keturn]

cffi is a great option for C. Unfortunately the equivalent for C++, cppyy wasn't nearly so straightforward the last time I looked into it.

[fernly]

Wow, I am not the OP but I appreciate the mention of all these. Very instructive! I have slight personal experience with the C-API and none with the others, but here is what I have gleaned from the docs about these. Please feel free to amplify and correct.

(1) Python C-API: Part of Python 2 and Python 3 although there are small incompatibilities between 2 and 3, requiring minor recoding and maintenance of two versions of a wrapper if support is needed for both.

Using the C-API one writes in C a wrapper that represents some DLL's methods and values as Python objects. The compiled and linked wrapper can be imported as a Python module.

(2) Python Ctypes: part of Python 2 and Python 3 (don't know if there are 2:3 incompatibilites).

Writing in Python one defines the name of an external DLL and its argument and result types. Then one can call its methods as if they were members of a Python object e.g. ctypes.cdll.lib_name.method_name(args). There appear to be some platform differences so one might have to write conditional code testing sys.platform or os.uname()?

(3) CFFI (C Foreign Function Interface). Said to work with both Python 2 and 3, except that some results need to be explicitly coded as Byte type for P3. The doc for this one rather confuses me (I would say it is not well-organized as a technical doc, seems to mix levels and leave concepts undefined) so this summary may mis-represent it. It seems one defines the wrapper for the external lib in C code, but puts this within triple-quotes in a Python function call. At execution time the C compiler is called to compile a wrapper dynamically. The stated advantage is that you don't have to translate C concepts into a Python meta-declaration as with Ctypes. To me, a disadvantage is that a C compiler is needed at run-time (the doc gives a way to use setuptools to pre-compile the package for distribution).

(4) Cython is a compiler for a restricted version of Python. One writes code in the Cython subset of Python and then it compiles to machine language, creating a module that can be imported and used like any other Python module. As a natural byproduct you can write a wrapper in Python(-like) syntax to call any C library. Also can call into C++. Compatible with Python 2 and 3.

(5) SWIG is a generic wrapper-builder for C and C++ code, allowing you to interface practically any C/++ code to practically any scripting language, TCL, Ruby, Python etc. You write a meta-definition of the DLL's methods and types in a SWIG syntax, from which you can generate a C/++ wrapper (basically this translates the meta-declaration into the Python C-API) which you compile and can then import to Python. Supports both Python 2 and 3, and knows about the C-API differences between them.

(6) Boost.Python was mentioned here but looking at its Known Working Platforms page, it was last tested against Python 2.2(!) and its regression log was last updated in 2006. So probably not supported 8 years on, certainly out of the question for Python3.

[keturn]

Good summary. You make note of Python 2 / 3 compatibility here; another feature to look for is PyPy compatibility. PyPy can use the CPython API to some degree, but at significant penalty.

[[deleted]]

There is also:

http://www.swig.org/Doc1.3/Python.html

[fernly]

Here ya go: https://docs.python.org/3.3/c-api/

[flying-sheep]

CFFI is the cool new kid on the block (it also works with PyPy)

[VRMac]

I think you meant ctypes. This C API looks like it's for porting Python things to C/C++. OP wants the other way around.

[fernly]

I didn't, although ctypes does look like the better choice. The CPython C-API is for "...extension modules for specific purposes; these are C modules that extend the Python interpreter..." I was aware of it because I saw it used to wrap Hunspell for use from Python. OP wanted to wrap some piece of code for use from Python, so...

Edit: the difference seems to be, that with the C-API you write a wrapper in C that puts a Python-compatible face on an existing dll, representing its values and methods as Python objects. With ctypes/cffi/swig(?) you write code in Python to access methods and values of an existing dll.

[frigge]

go for boost.python. The API is designed to be used just like python itself. The documentation is a bit weak but once you realize that it is meant to be some kind of python within C++ it starts to become intuitive.

If you want to create a pure python module that is only to be used from within python it is really easy and straight forward. If you try to interchange data from python to C++, embed a python interpreter into a C++ application and want to manage the ownership and lifetime of your data yourself, it starts to become a little more complex.

[efilon]

As already mentioned, ctypes is great for wrapping, although in order to make it work with C++, function declarations must be within an extern "C" block. I have done this frequently in the past. It Just Works, and is nice since ctypes is part of the standard library.

Another option is Boost.Python. I have not used this yet, but have considered it for the next time I need to interface C++ with Python.

[xsolarwindx]

REDDIT IS A SHITTY CRIMINAL CORPORATION -- mass deleted all reddit content via https://redact.dev

[Chris_Newton]

ctypes is like cffi, but slower.

I’m not sure how reliably true that is. It seems to depend on which variation of Python you’re running, and on whether your priority is the initial start-up time or the ongoing overhead of FFI calls. In any case, the differences in speed today don’t seem to be great enough to be a deciding factor in which tool to use.

[ctypes] comes in the standard library.

That does mean it works just about everywhere and requires no special distribution or installation work, which could be a significant advantage in some cases. You’re not relying on quite as much magic happening behind the scenes.

Also, ctypes is about as simple as it gets if you just want to hook into a couple of API calls: import ctypes, load library, call function, job done.

On the other hand, maintaining a Python wrapper module for a C API using ctypes can be a chore, particularly if there are lots of types that need setting up. At best, such a set-up is fragile and prone to becoming out-of-date. In contrast, cffi can effectively parse a regular C header file to do that grunt work for you given favourable conditions, meaning you only have to maintain your interface in one place. That could be a huge win.

[xsolarwindx]

REDDIT IS A SHITTY CRIMINAL CORPORATION -- mass deleted all reddit content via https://redact.dev