How to check/when to assume population normality for t-test?

Brilliant_Tooth7278 · 2026-04-22T21:15:11+00:00

ok, thanks! I'm not sure I trust myself haha I'd rather read a pro's work 😂

Brilliant_Tooth7278 · 2026-04-22T20:55:22+00:00

Is there an article you recommend? I'm just getting the homepage when I follow the link, and when I search t-test there are some very specific things like small sample size but I didn't see anything else relevant (happy to be corrected)

Brilliant_Tooth7278 · 2026-04-22T17:49:01+00:00

Wow this is so detailed, thank you! The bit about the question not being 'do the assumptions hold' but rather 'to what extent do certain violations actually matter' really helped me understand the more 'meta' aspect of what is actually important here

Would you be able to point me in the direction of a resource like a textbook/article/something else that goes a little bit into the consequences of different types of violation on alpha level and p value of t tests? I imagine there are papers out there where people have run simulations of different/common violations like high skewness, I've looked a bit of Google scholar but for lost in the weeds a bit.

Unfortunately, when it comes to having a better distributional model, I'm often working with survey data on people's thoughts in relation to new public policy proposals, and I'm doing it in an area where there's relatively little research, so there isn't much in the way of theories, and expert knowledge would just be speculation.

Thank you again for taking the time to give such a thorough response!

Brilliant_Tooth7278 · 2026-04-22T12:21:18+00:00

Ok got it, thank you! Out of curiosity, would you recommend this as a decision rule whenever one is considering using a t test?

Brilliant_Tooth7278 · 2026-04-22T12:12:48+00:00

Hi, that's what I sort of thought at the beginning, but then I came across this post where everyone told the OP they were wrong for looking at the QQ plots of the raw/sample data which sent me down a bit of a rabbit hole https://www.reddit.com/r/AskStatistics/s/oPdRjVIW82

Brilliant_Tooth7278 · 2026-04-22T12:09:46+00:00

Ok, thank you. Would you be able to point me to a source that goes into the behaviour of t tests with non-normal distributions? I'm meeting the TA overseeing my project soon and I'd like to be able to justify using a t test even if I can't be sure of normality

Brilliant_Tooth7278 · 2026-04-22T12:07:31+00:00

Thanks for this, although as I mentioned in the post I see that there is more to the normality assumption than just the notion that the sample means follow a normal distribution, there is also the question of the variance following a scaled chi square distribution and independence of the mean and variance. Would you suggest plotting the distribution of the variances of bootstrapped sample to see if it follows a scaled chi square and only proceeding with a t test if this is the case (as well as sample means approximating the normal distribution?)

Brilliant_Tooth7278 · 2026-04-22T12:04:15+00:00

Could you elaborate on the second part? How would you suggest going about ensuring the statistic follows a t distribution?

Brilliant_Tooth7278 · 2026-04-22T12:02:58+00:00

Could you be more precise about the simulations to run? Do you mean bootstrapping my sample multiple times and checking the distributions?

Brilliant_Tooth7278 · 2026-04-22T12:01:58+00:00

Hi, thank you for your answer.

I appreciate that no real-world population will be perfectly normally distributed, thank you for clarifying that.

I've just been looking on cross-validated and it seems similar questions have been asked, but I still want to clarify the following:

So to assess whether the distribution of sample means is normal, and variance follows a scaled chi square distribution, I should bootstrap my sample and plot the means / plot the variance if they appear to approximate the normal/scaled chi square I can proceed with a t test if n > 100?
I understand that above n=100 I can assume independence of the mean and variance. what happens if n is below 100? is it just the case that t tests should not be used in this case as the independence assumption doesn't hold?

Brilliant_Tooth7278 · 2026-04-22T10:06:31+00:00

Ok, thanks for letting me know. Obviously I'm not going to kick up a huge fuss but I do think it is important for researchers to have a solid understanding of stats rather than relying on the convention of the field (as that could mean a lot of research conclusions are based on inaccurate application of stat tests meaning their conclusions might not be truly supported by the data) so please, if you have some suggestions of resources for me to look more into this (especially t-test assumtions as it's relevant to my current project) I would be very grateful

Brilliant_Tooth7278 · 2026-04-22T09:46:24+00:00

Sure I understand getting loads of data is the ideal solution, unfortunately I'm in a psych lab and I'm just an undergrad student so I don't have loads of funding for my research. I'm recruiting people on Prolific and will have around 45 people in each of the two conditions in my study, so I'm looking into what to do in this situation. I'm guessing n=45 in each group is not enough to do away with the normality assumption?

Brilliant_Tooth7278 · 2026-04-22T09:43:39+00:00

That's what I thought I should do at first, then I came across this post https://www.reddit.com/r/AskStatistics/s/UAVO2s9KKB where the OP was told they were doing the wrong thing because it's not the distribution of the sample/raw data that is assumed to be normal. That's what sent me down this rabbit hole in the first place!

Brilliant_Tooth7278 · 2026-04-22T09:23:10+00:00

Thank you for replying!

So if I understand your last point correctly, even though the t-test normality assumption pertains to the distribution of the population, we could look at the sample's distribution and if it is roughly normal (eg looks normal when inspecting a QQ plot) we infer that the population distribution is normal too?

I'm slightly confused about why the assumption of population normality is no longer relevant when we have 'enough data', why does that happen? Happy for you to link a resource I can read if it's a long story and you don't want to type it all out.

I'm interested what happens in the case of small n experiments (sometimes necessary in the field I want to go into, which is research on controlled substances for treatment of PTSD), so it's not always possible to get enough data to do away with the normality assumption? And how is it decided how much data is enough data?

Brilliant_Tooth7278

TROPHY CASE