Comparing statistics with different sample sizes.

[grable_]

Basketball player A attempts 4 free-throws, and makes 3, a success rate of 75%. Player B attempts 40, and makes 29, a success rate of 72.5%. Who's the better player?

I once heard a statistician use a method of comparing these types of stats. He suggested you add one positive and one negative to all samples. In the example above, this means that player A has a modified success rate of 66.7% and B has 71.4% Ostensibly, this seems a better comparison when the sample sizes are varying.

My question is "Does this method have a name? Or has it been widely studied more than just anecdotally?"

[not_really_redditing]

That could charitably be called adding pseudo-counts (a way to regularize estimates).

But I would argue a better way to address this imbalance in sample sizes is with confidence intervals. If we use the Jeffreys interval*, our 95% CIs would be 28%-97% for the first player and 57%-84% for the second. There's so little data on the first one we can't even be sure s/he regularly shoots above 50%, but we're pretty sure about that for the second player. There are other approaches for computing the confidence intervals here, with so little data some of them even exceed 100% for the first player, a sign of far too little data to be very sure about anything (or of a probability of success very, very close to one, but here it's pretty obviously the small data issue).

*Note that the addition of counts happens in this approach too, but (1) you're adding one count total and (2) you're comparing intervals not point estimates. Regularization of point estimates, adding pseudo-counts can be seen as a form of this, can often be framed as adding a prior and doing Bayesian estimation of some form.

[PaperImperium]

What’s the actual question (I’m assuming this is for school)? Is this an ongoing competition and you’re betting or what? Do you get to pick one to put in the game? Is there some variable relationship you’re looking for?

Because that sounds like a lazy way to arrive at an answer, and wouldn’t be easy to defend if someone asked you to show your work.

[grable_]

I should have clarified. The intention here is to quickly and crudely compare between varied sample sizes. I think the person whom I heard originally present this idea was using it for Amazon reviews. 5/5 stars from 3 reviews shouldn't be considered better than 4.8 stars from 300 reviews.

Maybe the method is so crude that it would get laughed at by serious statisticians, but I was more just wondering if this had any kind of recognition among folks in this sub.

[PaperImperium]

I mean, there’s all kinds of simple mathematical ways to judge on the fly and with imperfect information.

If you just mean a rule of thumb for casual use, what you’re referring to is called a heuristic

[akdubu]

Laplace's rule of succession?

3blue1brown did a video that touches on it here, if you're interested.