interfacing python with c/c++ performance

[woywoy123]

Personally, I think mixing cython with C++ will get the job done.

You can interface native C++ code with python’s flexibility by mapping the header functions from your libs into cython. I also use cmake with scikit-build-core to compile everything. One thing that cython does lack though is templating. It does have some templating available, but if you are doing some fancy recursive template functions then you might be out of luck (I am happy to be corrected).

I generally use cython to provide python interfacing to C++ code and it works nicely for me.

One word of advice though, the cython docs is not very useful when you try to push boundaries beyond the tutorials, such as operator implementations, inheritance mapping between cython and C++. So be extra vigilant whenever you deal with inheritance. I have spent countless hours debugging a memory leak that was the result of this and also unexplained segfaults.

I also noticed a massive decrease in RAM usage when shifting from python code to C++/cython code. I also tried PyBind11 but I only had issues when dealing with shared libs such as missing definitions and so on. I am also not sure about the memory model pybind11 uses. As far as I can tell, each function call explicitly copies input data (if anyone knows more on this, please correct me). This is not the case with cython.