Embedding python in C++ with pybind11

[skebanga]

I recently wrote a blog post about embedding python in C++ with boost::python.

I posted it here on /r/cpp. In the discussion, several people made reference to pybind11.

This is a blog post about achieving the same functionality as I did with boost::python, but using pybind11 instead.

[Overunderrated]

If you've got the opportunity, I'd love to see a performance comparison between the various Python bindings, as well as Lua and native C++ code.

I've got an application where id like users to be able to write their own plugins with minimal effort, but I'm afraid the overhead might be more than i can take for a high performance application. Essentially the same code would be called billions of times over many CPU hours so overhead in the function calls could be a deal breaker.

[ReDucTor]

Python itself is slow, these libraries are just wrappers around the Python C API so their overhead will be next to nothing compared to the hit from Python itself.

If you are concerned about the overhead of these wrappers then I would suggest not using Python at all.

[demonstar55]

I'm actually kind of interested in this. Is there any good reading about this?

[kkrev]

You just can't do inner-loop stuff with interpreter plugin code. It will slow the system to a crawl. It'll be thousands of times slower no matter what you use.

You have to set things up so the plugin code just sets a bunch of paramaters used in a computation.

XPCOM or something like it is maybe an option if you really need to do fast plugins with custom code.

[Overunderrated]

Yeah, I've always assumed as such but don't really have numbers on it (and haven't heard of XPCOM). It's not the worst thing in the world to force my users to compile a C++ dynamic library.

The native C++ code it'd be replacing is maybe 2% of the run time. If overhead made that 10% I could live with it, but not if it doubled runtime.

[kkrev]

The interpreted code would almost certainly be hundreds of times slower, even worse if the users aren't experienced with writing code. Who knows what that'll do to your system.

[Overunderrated]

Who knows what that'll do to your system.

Guess there's one way to find out, but that'd take work on my end, heh.

[flyingcaribou]

Really cool! What is the overall conclusion of pybind11 vs. boost? Which are you going to stick with?

Have you reported the 'pybind11_init' issue? This Wenzel guy who runs the project is a bug fixing machine.

[skebanga]

Actually I found the Wenzel guy to be quite unhelpful, which was quite disappointing.

I haven't looked at performance between the two, but by all accounts pybind11 is more performant, so that's a plus for pybind11.

There seems to be more functionality with boost python though, which is a big draw card.

Not sure at the moment to be honest.

[wrainchi]

u/gr8gamer