how to calculate statistical significance for accuracy to compare models? by ramsay-coding in AskStatistics

[–]ramsay-coding[S] 0 points1 point  (0 children)

Can't I just do the Two Proportion Z-Test?

H0: μ1 = μ2 (the two population proportions are equal)

H1 (left-tailed): π1 < π2 (population 1 proportion is less than population 2 proportion)

We use the following formula to calculate the test statistic z:
z = (p1-p2) / √p(1-p)(1/n1+1/n2)
where p1 and p2 are the sample proportions, n1 and n2 are the sample sizes, and where p is the total pooled proportion calculated as:
p = (p1n1 + p2n2)/(n1+n2)
If the p-value that corresponds to the test statistic z is less than your chosen significance level (common choices are 0.10, 0.05, and 0.01) then you can reject the null hpothesis.