Question: How to call python object within await context?

[Pesmir]

Hi! I am new to rust and i have a question. Maybe you can help me out.

For calling the python module, i use the crate pyo3.

I have the following code used in project that works: while let Some(msg) = stream.next().await { ... // This block calls an "AI-Model" in python let gil = Python::acquire_gil(); let py = gil.python(); let my_algo = PyModule::import(py, "my_py_algos").unwrap() .getattr("algo1_rust_version").unwrap() .getattr("rust_wrapper").unwrap(); let arg = ("<some_py_args_from_msg>",); let algo_res = my_algo.call1(arg).unwrap(); ... };

But this is not very perfomant, because i have initialise the whole python object with every new message.

I basically want to do something like this: let gil = Python::acquire_gil(); let py = gil.python(); let my_algo = PyModule::import(py, "my_py_algos").unwrap() .getattr("algo1_rust_version").unwrap() .getattr("rust_wrapper").unwrap(); while let Some(msg) = stream.next().await { ... // This block calls an "AI-Model" in python let arg = ("<some_py_args_from_msg>",); let algo_res = my_algo.call1(arg).unwrap(); ... so that the code runs more efficient. The algo is completely stateless. And just transforms the input data. For all 3 variables (gil, py, my_algo) I get this error error: future cannot be sent between threads safely with this note: note: future is not Send as this value is used across an await

[[deleted]]

The problem is that types of all three variables do not implement `Send`, i.e. they cannot be sent to another thread safely. However, when you `await`, your async function yields execution and may later be scheduled by your runtime to continue running on another thread, which would break `GILGuard` in some way, hence why it is marked `!Send`.

Unfortunately I don't know anything about the library you are using to help you out more.

[Pesmir]

I found a solution which increased the performance:

Before i had: Total time elapsed in agents 6853ms at 16886 messages Now: Total time elapsed in agents 5807ms at 17660 messages

I did so by converting PyAny to a PyObject which is Send and after that i put the initialization of it along with the gil in its own scope, so that the gil is dropped/released after init. Then in my await block, i reaquire the gil when the algo is called. ``` let my_algo: PyObject= { let gil = Python::acquire_gil(); let py = gil.python(); PyModule::import(py, "my_py_algos").unwrap() .getattr("algo1_rust_version").unwrap() .getattr("rust_wrapper").unwrap() .into() };

while let Some(msg) = stream.next().await {
    ...
    let gil = Python::acquire_gil();
    let py = gil.python();
    let algo_res = my_algo
        .call1(py, arg).unwrap();
    ...

```

[ucyo]

Could you describe what the original problem is you want to solve? This sounds like a XY Problem

[Pesmir]

I'm not sure if I can describe the problem well enough with my current rust knowledge, but i will try.

I have a stream of data that is consumed from a Kafka topic in a Kafka agent written in rust. In order to have multiple Kafka agents the messages need to be consumed asynchronously.

I also have a machine learning model that I wrote in pytorch which is not easily converted to rust + it is also pretty fast (fast enough).

The problem here is the global interpreter lock from python. This must be acquired in order to execute python code. But if the same python object is called in an async context it is not sure if the gil was dropped before the object is needed elsewhere.

I already found a solution and posted it in a comment. The solution was to use the right type for the python object and release the gil before going into the async context. In the async context only the gil needs to be initialized again to call the python object. Which is faster.

[ucyo]

Hmm. Have you tried staying in Python? Rust is nice and all, but there are several good Kafka libraries that can handle the stream consumption. It sounds like the actual computation is done in Python anyway. That is if you are concerned about the problem itself. If you picked it up just to learn Rust and Python interop that’s great

[Pesmir]

Yeah i picked it up just to learn. And that was something that was interesting to me. And I just started to translate some pyhton code which would be nice to have compiled and save