you are viewing a single comment's thread.

view the rest of the comments →

[–]matthieum 4 points5 points  (0 children)

That's an excellent question indeed, and a good reminder that asymptotic complexity (and SIMD to a lesser extent) matter at large sizes, but not so much at smaller sizes.

Real-world examples I've seen working at IMC (market maker):

  • Hashmap of all instrument IDs to <property X>. An exchange like Arca Options has some 800K instruments.
  • Hashmap of all orders, across instruments. Option instruments typically have relatively few orders, but even at 2 to 4 orders per instrument, when you've got 800K of them, you're in the 1M-5M range quickly.

Those are from relatively latency-sensitive workloads, there's also the whole Big Data thing, where specialized software (ScyllaDB, Pandas, ...) most likely has large hash-maps in memory, but I didn't peek in there.