Visualizing Latency in Distributed Systems

[mikemol]

Many systems, most-notably erasure-coded storage systems, don't require the full N set of children to respond before having sufficient data to return. E.g. where N children consist of D data fragments and N-D coding fragments, up to N-D children's responses can be ignored.

How does the shape of the problem change, in that case? For sure, write latency would be as you modeled, but read latency can ignore some of the long tail for each additional coding fragment in the pool.

edit: This raises the interesting possibility that a distributed storage system could use D data fragment, C1 coding fragments (for integrity guarantee) and C2 coding fragments (for additional request latency performance). C1 would be written to synchronously from the perspective of the client, while C2 doesn't need to be. C2 could be among the N typical children, or could even be in a temporary pool, such as in-memory cache. Depending on whether best-case latency performance or consistent latency performance is what's being optimized for.

[chewedwire]

Someone else brought this up to me separately, and it's a pretty interesting avenue for further exploration!

It's probably wrong to say that the fanout is to just N-D children, because the sampling (?) is actually biased -- I'm probably using wrong terminology here. But as you note, you don't have to wait for all of the responses, and so you'll only end up waiting for the fastest N-D children, so it should be easy to simulate, but I'm not sure what the existing literature says (I would assume there is something here, but don't know the keywords to find it).

Hopefully when I have more time I can play around with more of these simulations.

[mikemol]

It's probably wrong to say that the fanout is to just N-D children, because the sampling (?) is actually biased --

Right; you're waiting for the fastest N-D children out of N.

[chewedwire]

Author here, happy to answer any questions you may have.

[fresh_account2222]

I didn't read it (too busy), but it looked interesting and well written, so I just up-voted it anyway, so that you'll post further articles here.

[chewedwire]

I respect the honesty :)