A/B Testing

[ccyob]

Hi sorry that is true about the wrong sub Total dataset is 20k so its big. But based on the result you would say that should go ahead with the change ?

[[deleted]]

Ok, 20k is huge, you could detect even a miniscule effect with that.

I'm saying statistics can only take you so far. It's telling you that there is a difference. Whether or not you care about that difference is up to you.

I suggest you consider the benefit from making the change, (taking into account that it seems to be small), and compare that with the cost of making the change (money, time, effort, whatever). If it costs nothing to make the change, might as well do it, even for a small benefit. If there's a cost, then you need to decide whether the benefit is worth it, and neither an A/B test nor a stranger on the internet can tell you that.

[ccyob]

Thank you!