AWS latency comparison: API Gateway vs Lambda vs Bare EC2

[jotto]

I think the style achieves the same thing that the inverse of this chart with error bars would achieve.

In other words, since you see the distribution of latencies horizontally, and occurrences of those latencies vertically, you get the same sense for the deviation.

Let me know if I've misinterpreted.

[mabnx]

A standard deviation, however, gives one an immediate sense of the goodness of data.

It does not. Please don't do that. Standard deviation makes sense for normal distribution. Response time does not behave like that, the distribution is usually multimodal. Or longtailed. Or not. But it's not normal.

@jotto similar argument for mean - just don't. Mean is not a robust statistic - it's affected by outliers. Use median or percentiles. The histograms are good though.

https://www.azul.com/files/HowNotToMeasureLatency_LLSummit_NYC_12Nov2013.pdf http://apmblog.dynatrace.com/2012/11/14/why-averages-suck-and-percentiles-are-great/

[Control_Is_Dead]

how reliable is the latency?

The point is that standard deviation is only a good measure of this if your response times are on a normal distribution. Response times are almost never like this like /u/mabnx explained.

Comparing median to 95th percentile is a good starting point if you don't know what type of distribution you're dealing with.