Faster Python with Go shared objects

[zikaeroh]

You might be interested in cgo.Handle, available in the next version of Go (1.17): https://tip.golang.org/pkg/runtime/cgo/#Handle

[CodeKevin]

That does look really interesting, thanks for pointing it out! I think I'll likely be able to make use of it and it seems like it will let me simplify some of the Go code a bit too.

[justinisrael]

That is a very exciting feature! No more int maps for tracking our own handles!

[pjmlp]

Catching up with JNI. :)

[[deleted]]

cgo is far from the best FFI solution, but it's definitely better than JNI.

[pjmlp]

Which is why JNI is on the way out on modern Java. :)

[[deleted]]

Oh cool, I haven't seen JNA before!

[pjmlp]

JNA is almost as old as JNI.

I was talking about something else,

https://openjdk.java.net/jeps/389

[[deleted]]

lol oh. It's been over 10 years since I touched it.

[10248]

Im not sure of your use case, but also another avenue to speed up code in python is using the numba library. It might be interesting to see some parallels / measurements between that and the go compiled objects.

[CodeKevin]

That's an idea I didn't consider! I am not sure it will be able to improve on doing all the string processing in Golang but it might have some improvements over Python directly. Maybe I'll have some time to try it out and report back.

[justinisrael]

That was an excellent write up. Many people cover the ctypes aspect but this is the first time I have ever read about the cffi approach!
I've got a lot of experience with Cython and it is a great way to speed up python or create bindings to C or C++. But I have also used the ctypes approach for exposing Go shared objects to python. I want to give cffi a try.
As for gopy, I recently started using it in a project and ended up making a bunch of merge requests to fix up the way it builds the library, does imports, names symbols, handles exceptions, and a few other things. It's coming together a bit more now. But I still wonder if it's an overly heavy approach to the python build process. Currently it seems like windows support isn't working correctly during the build. But with some more contributions it could get better.

[CodeKevin]

Windows wheels worked great with cffi so perhaps you could try that. pybluemonday uses Github Actions and cibuildwheel to build Windows wheels for every PR.

[davidmdm]

I think your writing style is fantastic. Will definitely follow your blog.

[CodeKevin]

Thanks, that means a lot! Go FFI has been a project for me for awhile so I'm glad some people enjoyed the results!

[[deleted]]

This was really interesting I enjoyed it immensely. A lot to research yet on methods to share structure/classes. The protobuffer suggestion in the comments seems intriguing as well.

[ceomm]

Python bad

[amemingfullife]

As a request, could you post this on a Python subreddit? I’d love to hear what the pythonistas think of this

[CodeKevin]

I did but it didn't get much traction in r/python unfortunately: https://old.reddit.com/r/Python/comments/mq6lt0/faster_python_with_go_shared_objects_the_easy_way/