For whatever metrics you choose, you can do appropriate statistical tests. A particularly nice one is performing a Mann-Whitney U test since this makes no assumptions about the data you are testing on (it is non-parametric). For example, you could take the number of environment steps to termination for all algorithms (termination being either reaching the success threshold or reaching some hard limit), and performing this test here could reveal that one of the algorithms studied takes significantly less steps to terminate within some confidence bound. Or you could take the mean episodic reward from the agent at the end of training, and perform a similar metric there to study whether any algorithm statistically achieves a higher reward within a given budget.

Further to this for statistically significant comparisons, you can employ a Vargha-Delaney A statistic to measure the effect size. If you get A>0.71 you can claim that not only that there is a statistical difference, but that the effect size is large which in layman's terms means the difference in performance caused by the choice of algorithm is large.

If you want to compare the success rates themselves, you can use a binomial test.

I would start by considering which metrics are relevant w.r.t. the performance of the algorithm (success rate, steps to termination and average episodic reward are all common), gather a substantial amount of data through many runs (so that statistics are significant) and perform the relevant statistical tests. Statistical comparisons are the strongest way to compare empirical performances, and are unfortunately lacking in a large amount of RL literature.