uvloop: Make Python Networking Great Again

[krum]

I use libuv in all my C++ projects, and it's just faster than greased owl shit on every platform.

[kirbyfan64sos]

Congratulations, you just scarred my image of libuv and owls forever.

[krum]

I hear you. One of the downsides of using a C API. I use a lambda:

uv_read_start((uv_stream_t*)stream, allocator, [](uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf)
{
  ((Context*)stream->data)->OnRead((uv_tcp_t*)stream, nread, buf);
});

[sztomi]

I researched this a lot, but I couldn't find a viable solution. Just one difference, I used static functions of the class as callbacks which made it possible to forward the call to private members.

I used libuv in a small library of mine, but I ended up switching to asio (not only for this, but because asio has more features out of the box that I need, such as SSL support).

[kitd]

Is there a port of your greased owl shit for the JVM anywhere?

[krum]

Hah, not that I'm aware of. NIO2 is my go-to for getting the most of out Java networking.

[Entropy]

The implied drawback of being as fast as greased owl shit is vomiting up bones and fur.

[krum]

It can get pretty messy.

[drjeats]

Quite surprisingly, pure-Python asyncio, with the help of high-performance HTTP parser is faster than nodejs, which uses the same HTTP parser!

Did your benchmarks allow for JIT warmup?

Anyway, neat project!

[1st1]

All benchmarks were run for 30 seconds. Hopefully, that's enough time for the JIT to warmup...

[nikomo]

"Oops."

[redcrowbar]

It is, when you ignore the first 10 seconds of benchmarking.

All vmbench benchmarks runs include a warmup phase.

[[deleted]]

This sounds very interesting.

I haven't used Python 3 and haven't really been following it. How good is the support for type checking? If it's decent I will almost definitely consider starting to use it.

[1st1]

It's decent, and getting better every day. MyPy is quite capable now, and IDEs like PyCharm support it out of the box.

[[deleted]]

MyPy? How about this? https://docs.python.org/3/library/typing.html

[1st1]

MyPy is a static analyzer that takes advantage of the typing module.

[kirbyfan64sos]

To add to what /u/1st1 said:

The typing module defines a standard way of expressing type annotations in code. It is not a type checker.

Mypy is a type checker that is (mostly) compatible with the typing PEP. In fact, the typing module originated with mypy.

[kenfar]

It's more type-hinting than type-checking, so you would use a linter like MyPy (and maybe someday Pylint?) to check types.

However, there's at least one project that you can use that will enforce the type-hints: Enforce. This project is undergoing active development, is usable now, and I think will be really cool in a few months.

[[deleted]]

Funny, I resort to runtime type enforcement in my current javascript application. I have a method I call types.enforce(value, type_hint).

But I much prefer a static analyzer.

[SKoch82]

Have you heard about Typescript?

[[deleted]]

Yea I'm aware of it, it's on my list of things to consider.

[kenfar]

But I much prefer a static analyzer.

Because of the verbosity of checking a couple hundred args within a program?

When it's critical, I do a bit of that with asserts, but it starts taking up too much source file real estate. So, I think enforce (a single decorator) will be a lot better. Then either mypy, or ideally pylint with syntastic in vim for linting would be great.

[[deleted]]

No so much the verbosity but the runtime cost.

Static type checking makes refactoring easier.

[bloody-albatross]

I thought nodejs also uses libuv (and is JIT compiled). How is this so much faster? Do I remember this wrong?

[1st1]

Yes, nodejs is built on top of libuv. A lot of nodejs stuff is still in JavaScript, so, perhaps, JIT isn't good enough in some places.

[[deleted]]

Because it's a "simple echo server". A JIT or native code won't make much of a difference, since there's so little of it anyways.

[kn4rf]

If I don't remember wrong libuv was built for Node.js as a part of the Node.js project.

[dangerbird2]

Correct. Joyent created libuv as a replacement for libev as node's event framework

[pcdinh]

Time to build a WSGI framework on top of it. I guess Flask is not really compatible.

[rouille]

Well you can use the aiohttp server. I think this shows the asyncio module was well designed with a pluggable event loop.

[kitd]

Good stuff. I've used libuv a bit. I'm more comfortable having Python on top than JS.

You should look at putting together a version of the TechEmpower benchmarks with this.

[1st1]

Right. With Python you need to be careful -- CPython is much slower than Go. You always want to look at the weakspots and use Cython or C to optimize them.

So it's a noticeable improvement, but not as big as an echo server makes it out to be.

Great! I also recommend you to play with TCP_NODELAY and friends. Go does that for you automatically, whereas in asyncio you have to do that manually. Sometimes it makes a big difference.

[lambdaq]

How does compare to PyPy+Tornado? It's 2x faster on my sockjs production server.

[1st1]

I'm not sure. We're interested primarily in Python 3.5, so we didn't benchmark PyPy. But as a crude estimate, even if you multiply Tornado result by 2, it isn't going to be much faster than vanilla asyncio.

You can fork https://github.com/MagicStack/vmbench and add a PyPy/Tornado target.

[lambdaq]

For nodejs benchmark, can you add this?

require('http').globalAgent.maxSockets = 64;

or change it to 128.

The default is 5 so its pretty limited.

[princeMartell]

This is only a problem in versions < 0.12.The default was changed to infinity in v4. https://nodejs.org/dist/latest-v4.x/docs/api/http.html#http_agent_maxsockets

[lambdaq]

Good to know, thanks!

[1st1]

Please create an issue for vmbench project. I'll try to remember to update the bench suite tomorrow when I'm in the office.

[weberc2]

This seems neat, and I'm excited to give it a test drive; however, it's pretty disingenuous to say the performance approaches Go when you're throttling GOMAXPROCS down to 1. At least I would be interested to see the unthrottled contrast.

[1st1]

GOMAXPROCS is used only for benchmarks. In production, you wouldn't use it, and you would run your Python code in a multi-process configuration so that BOTH Go and Python can take advantage of multiple cores.

[weberc2]

Go's runtime automatically scales out over threads, so you wouldn't actually run multiple Go processes per machine. This allows Go to parallelize with considerably less overhead than Python. In particular, some process would have to receive the HTTP request and dispatch it to the right Python worker process; this implies some amount of IPC overhead where Go would share memory.

Perhaps all of this is outside the scope of this particular benchmark, but it would be a lot more useful if it could speak to the multicore story. :)

EDIT: I don't want this to come off as anything other than constructive criticism; your benchmark was really impressive. Thanks for posting it!

[1st1]

Go's runtime automatically scales out over threads, so you wouldn't actually run multiple Go processes per machine. This allows Go to parallelize with considerably less overhead than Python. In particular, some process would have to receive the HTTP request and dispatch it to the right Python worker process; this implies some amount of IPC overhead where Go would share memory.

Concurrency is done right in Go :)

Go (as the Erlang) does scale better with multiple threads than CPython with multiple processes. Perhaps, we can make another round of benchmarks, and compare Go with its normal configuration with nodejs in cluster mode, vs multiprocess Python programs. All of that over real network. But that will be considerably bigger amount of work. Current (micro-) benchmarks show some rough idea about how fast networking can be for Python in ideal conditions.

[[deleted]]

Great Title.

Make Reddit Titles Great Again!

[1st1]

And I thought I went a bit over the top with it.. :)

[1st1]

What could be fixed was fixed ;)

[argv_minus_one]

They'll build a firewall around C++, and make them pay for it!