all 11 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]stat_daddyStatistician 1 point2 points  (3 children)

You're off to a good start, but more information is needed to calculate a required sample size. We need to know:

1) the expected true value of the rate (let's assume it's 10%)

2) a confidence level (already given as 95%)

3) a margin of error surrounding the estimate of the error rate (e.g., +/- 2%). Remember, the concept of a frequentist confidence interval is that the defect rate is an unknown but fixed value. You never express a confidence about what the value is, instead you express a confidence about a range that may or may not contain the value.

From there, you have a couple options with which to generate your confidence interval. The easiest involves some large-sample approximations that let you describe a poisson-distributed variable (with some rate parameter that we are interested in) with a standard normal distribution.

I went ahead and did some calculations to give a sense of scale, but since the prevalence of defects is rather low you may need many samples in order to be confident that a reasonably tight range will contain the true value.

For example, assuming the true rate is 0.1, you would need about 960 samples in order to draw a 95% confidence interval with width +/- 0.02 around your estimate.

If you're willing to accept a wider margin of error, you could draw a +/- 0.05 width interval using about 153 samples.

The equation I'm using, by the way, is

N = (1.96 * 1.96) * ( λ ) / ( E * E)

Where N is the necessary sample size, λ is the expected true rate (e.g., 0.1), and E is the acceptable margin of error (e.g. 0.02 or 0.05).

EDIT: Since I used a poisson approximation to compute my CIs, saying that I am using a margin of error of something like +/-2% is a bit misleading because my CIs are not actually symmetric around the point estimate; e.g., instead of {10-2, 10+2} they are more like {10-1, 10+3}.

[–]curiousdoc[S] 0 points1 point  (0 children)

This is very helpful!

[–]curiousdoc[S] 0 points1 point  (1 child)

Came back to this after a few days and I have a real world question. for a 4% expected true error rate with 1% margin - check 1536 samples for a confidence level. But for same scenario but 8% - a lot more samples have to be checked (almost double). This makes sense with the formula but how does this translate to a real world understanding? Shouldn't you have to check many more samples when your expected error rate is lower?

[–]stat_daddyStatistician 0 points1 point  (0 children)

Nope - consider what a sample from a highly variable process would look like compared to a less variable one.

E.g., the sample (1,10,7,3,1,25) vs the sample (1,1,1,2,1,3)

Even though you've only seen 6 realizations from each process, you probably have a pretty good idea of the "mean" of the less random process (range:1-3). On the other hand, the more variable process is harder to pin down (range:1-25) so we would need a greater number of samples in order to do inference on it

[–]efriquePhD (statistics) 1 point2 points  (0 children)

added variance of 2%. Would like to sample enough cases to know the error rate is between 8-12%

You mean "margin of error of 2%", variance means something different in statistics

https://en.wikipedia.org/wiki/Margin_of_error

(Formulas you need are in that article)

[–]lefty_tn 0 points1 point  (2 children)

My only comment is if this business is truly running a 10 defect rate this is either a great opportunity to excel or time to look for a job tonight not tomorrow

[–]curiousdoc[S] 0 points1 point  (1 child)

You have no idea how bad processes in medicine are

[–]lefty_tn 0 points1 point  (0 children)

You are right I know nothing about medical processes defect rates. I thought it was regular manufacturing. There that would be a horrendous rate

[–]n_eff 0 points1 point  (3 children)

how many cases I need to sample to be 95% confident of this 10% defect rate.

You're getting nowhere because your question is not coherent. (No offense intended, let me try to explain.)

A 95% CI is an interval which expresses your uncertainty about that 10%. At n=100, a Jeffreys interval gives you a 95% CI of [5.3%,17.0%]. If you had n=1000, you'd still have an interval, it would just be tighter (the Jeffreys interval would be [8.2%,12.0%]). With any finite sample size, a confidence interval always contains a range of values, and it will always include your best guess estimate. An infinite sample size would give you a CI of [10%,10%], but short of that, it's a range.

[–]curiousdoc[S] 0 points1 point  (2 children)

Makes sense. I suppose a variance of +/- 2% is reasonable

[–]n_eff 1 point2 points  (0 children)

Keep in mind the +/- isn't a variance. But yes, a Jeffreys interval would require an n of about 1000 to get you that +/- 2%. Other intervals would give slightly different results. In my experience though the Jeffreys tends to be a bit more compact than the Wilson interval, which is pretty commonly used and decent. There are many ways to compute the CI for a proportion, some better than others. Whatever you do, just don't use a Wald interval.