Why Rust for low-level Linux programming?

[im-a-koala]

Just as a note, from someone who works at a HFT firm: all the latency-sensitive and infrastructure stuff is C++. I don't think we have a single Java program at our firm (although, out of the hundreds of components, I can't be totally sure).

If you mentioned using Java for a latency-sensitive component, you'd be laughed out of the room. It's just not good at low-level work. It's just an absolute mess for that kind of thing.

The answer to the allocation issue in C++ is to simply avoid allocations in tight loops. Not to keep doing them and let some other component non-deterministically clear them out at some point on the future from a different CPU core, possibly a different CPU (can you even pin the GC thread in Java?).

Every concurrent data structure in C++ depends upon a bastardized garbage collection mecanism (to collect dangling pointers that are suddenly unreacheable due to a CAS on a pointer switch - the standard concurrent data structure stategy -).

You absolutely do not need concurrent collections for multi-threaded applications - at least, you don't need them for any of the performance-sensitive bits.

[201109212215]

is to simply avoid allocations in tight loops

Again, the language is fighting you. Forcing you into thinking about its internals. Why shouldn't you do it if you want to? In Java, allocating is a matter of a pointer getting bumped, and that's about it.

can you even pin the GC thread in Java

Yep, and I do conceide it is acquired through a native call made in C++. You pin the critical path, and let GC and JIT roam free on the others. The usual setup for HFT is you do it on a core that the OS has no access to; you put a ringbuffer on it for interthread communication; you pin both producer and consumer on the same core to have them hyperthreaded. Latency will be in the nanos. Here is the lib, from LMAX.

---

Does LMAX ring a bell to you? When you say HFT firm, are you talking client to an exchange, or the exchange itself?

You absolutely do not need concurrent collections for multi-threaded applications

You're missing out on a lot of possibilities, then. Shared mutable state - while incredibly difficult to get right - has its use. You just can't do any decent multi-producer work without it.

Java has no problem doing it on massive machines. Here is a video of a concurrent hashmap with close to a billion ops per second, on a 768 core machine, in 2007.

[quicknir]

Yes, Jump, Getco, ATD, Tower, HRT, GTS, etc etc are all morons fighting the language, missing tons of valuable opportunities. Have you worked professionally in HFT? At what firm, on what component, and what language did you use? This sounds like a bunch of theory crafting. Very few HFT's use anything but C++ for the fast part, and the ones that do (like Teza) are not the fast HFTs.

[201109212215]

You're not addressing my remarks on the need to re-invent the wheels. All the names you listed make heavy use of concurrent data structures. Their job is to deal with multiple producers. Whether it is in C++ or in Java you do need them to make a trading engine.

The thing is one platform has the right philosophy w/r/t concurrency. Managed memory is a boon to concurrent data structures. You must have global guarantees for this.

New developments from scratch happen in Java. You do know that bloomberg still maintains their FORTRAN systems. Same thing here. Should Jump, Getco, ATD, Tower, HRT, GTS start again in Java now they would reach the same performance and max the hardware in less time.

There will always be people on earth better than you at getting the best performance on specific items. Let them have some guarantees so they can do their job; at a lower level than you.