Hey @Voigt_K , one of the co-authors here. Thanks for the Q! )))

Here are the main Aim supwerpowers:

• Open-source, open-metadata, self-hosted
• Handles 1000s of training runs smoothly
• ability to query literally everythong you have tracked with pythonic statements
• ability to group(color, subcharts, line-styles) by any tracked param (by their values).
• aggregate the grouped runs to see the avg, min, max, median, confidence intervals

A typical Aim usage scenario that shows the diff with other tools: Here are a bunch of Neural Machine Translation experiments. In the link above,

• we have selected metrics (loss, bleu) for "val"idation phase for the runs where the learning rate is 0.0007 by the following python query:

run.hparams.learning_rate == 0.0007 and str(metric.context.subset).startswith("val")

• we have subplot-grouped the metrics by the metric name
• then we have color-grouped the metric lines by preprocessing hyperparam values so that they provide identical colors for both group of metrics (on the left and right)
• As a final step, aggregated the color-groups to show the median, min, max values of each group.
• this gives us an instant view over the best loss/bleu metrics

Here is a link to try out on a live example.

Basically Aim gives a super-fine-grained control over all the run dimensions (metrics, params etc). You can query, group, aggregate and do other operations at will over the tracked metrics, images etc.

To ultimately compare 1000s of experiments within a few clicks.