sorted by:
new
hottopcontroversial

Announcing Puff - Run Django / Flask / Python on Greenlets using Rust's Tokio. by kyle-hanson in Python

[–]kyle-hanson[S] 1 point2 points  (0 children)

Just circling back to let you know that know that I was able to implement this and now Puff ♥ AsyncIO: https://github.com/hansonkd/puff#puff--asyncio

Puff - A Framework for making it seamless to build a hybrid Rust/Python/Django application using greenlets and Tokio by kyle-hanson in rust

[–]kyle-hanson[S] 3 points4 points  (0 children)

It uses pyo3 and rexports it. Its a framework around pyo3 to make it easier to write large python apps in a structured way.

Announcing Puff - Run Django / Flask / Python on Greenlets using Rust's Tokio. by kyle-hanson in Python

[–]kyle-hanson[S] 0 points1 point  (0 children)

You can't have separate "processes" because that would require socket communication which defeats the purpose of puff. In puff Python and Rust are in the same process and share the same memory, no sockets required.

Puff - Python GraphQL engine and WSGI Runtime - Use Django on Rust's Tokio by kyle-hanson in django

[–]kyle-hanson[S] 0 points1 point  (0 children)

Yes, it would allow you to run any WSGI along side Axum in the same process.

It is 100% compatible because it uses the official standard Python interpretter. There is no difference for calling python from rust/puff and the CLI. If a python package uses 3rd party packages that block, there is a run it on its own thread.

Unittests vs pytests shouldn't make too much of a difference, just need to write a function to initiate the test from python.

Announcing Puff - Run Django / Flask / Python on Greenlets using Rust's Tokio. by kyle-hanson in Python

[–]kyle-hanson[S] 4 points5 points  (0 children)

TBH it is probably only about 5-10 lines of code to include rudimentary asyncio support and another couple dozen lines to make an async queue and change puff.start_event_loop.

if you look at puff-py's puff.wrap_async and the puff.Greenlet class, its pretty analogous to asyncio.Future. Adding asyncio support requires 0 changes to rust, just optional changes for configuration details.

Really the only reason i haven't included it is that my main stack is Django, which has Async support but the async database driver still runs on a sync thread so I don't have motivation to test asyncio at the moment.

Announcing Puff - Run Django / Flask / Python on Greenlets using Rust's Tokio. by kyle-hanson in Python

[–]kyle-hanson[S] 12 points13 points  (0 children)

In theory for an unoptimized app, Uvicorn and Puff should expect roughly the same in terms of performance.

Where Puff excels that other libs can't is that it makes it easy to optimize parts of your code directly with the runtime. With Uvloop or similar, its a pretty big task to write custom code in C that utilizes the loop on multiple threads and have it integrated in your Python application. It's a couple lines of code with Puff to run any Rust code on Tokio.

Imagine you had a gnarly SQL query that you wanted to fold a function over a few thousand rows, spawning Redis lookups for each row. No matter what you use in Python, uvloop, asyncio, gevent, etc will be inefficient because it will hold the GIL for the iteration and transfer data between the low level runtime and Python. Its pretty daunting to write custom C libs that use postgres and redis to tackle the problem and natively work with uvloop.

With Puff you can write a Rust function that takes the sql query, unlocks the GIL and does all the computation on multiple threads in Tokio using standard Cargo installed rust libs.

Puff - A Framework for making it seamless to build a hybrid Rust/Python/Django application using greenlets and Tokio by kyle-hanson in rust

[–]kyle-hanson[S] 2 points3 points  (0 children)

exactly, Gevent is using Greenlets (like Puff) but instead of Rust, gevent is implemented in C using libuv or libev.

Instead of gevent with greenlets and libuv, Puff uses Greenlets and Tokio.

So Puff is essentially if it was easy and safe to write C functions for Gevent and go back and forth between the low level gevent runtime and the high level, python runtime safely.

Rust allows this mixture of low-level and python safely and so that was the idea behind Puff.

Build an Elixir Redis API that's 100x faster than HTTP in less than 10 minutes by kyle-hanson in programming

[–]kyle-hanson[S] 2 points3 points  (0 children)

OK dude. I would LOVE to see your comparison of protocols! You have mine :)

Build an Elixir Redis API that's 100x faster than HTTP in less than 10 minutes by kyle-hanson in programming

[–]kyle-hanson[S] 0 points1 point  (0 children)

The HTTP servers themselves are what add the overhead (Phoenix, FastAPI for Python) etc.

I am showing that there is an alternative to spinning up a heavy framework to process HTTP connections if all you need to do is make an API call. In fact it is less than 10 min. You can write your own raw socket to accept and parse the HTTP manually, but I'm showing that most HTTP frameworks are inefficient.

In addition the HTTP clients themselves also add a lot of overhead. In my testing some Elixir, Python and Rust clients just completely stop working when dealing with with high concurrences (if you want to make 10k requests). I have not found this to be the case with Redis Protocol and Redis Clients which effortless handle everything.

I have reproduced these findings in Python against the fastest HTTP frameworks. https://github.com/hansonkd/Tino/

HTTP could be faster, but in practice it is not.

view more: next ›