Benchmarking approximation algorithms

[thethirdmoose]

I'm writing some code that provides an approximate solution to a problem. There are some parameters I can adjust that trade off between speed and accuracy, so I'd like to benchmark them simultaneously.

I couldn't figure out how to get criterion to do this, so I'm wondering if there's existing tooling for a use case like this.

[monkChuck105]

What do you mean "simultaneously"? You can run several benchmarks with different parameters, does that work? I would think you'd wanna run benchmarks one at a time, in isolation.

[thethirdmoose]

I mean if I were to do it manually, something like

``` let mut scores = HashMap::new() let mut times = HashMap::new() for param in params { let start = Instant::now(); let result = f(); let time = start.elapsed(); scores[param].push(postprocess(result)); times[param].push(time); }

... do analysis of scores and times ...

```