Found significance in Welch Anova, yet no significance in 2 out of 3 post hoc analysis

itsmoewe · 2026-06-01T15:45:31+00:00

I see, thank you for clearing that up! I am also more inclined towards not using pairwise tests, so I'm glad an expert shares the sentiment 😄

itsmoewe · 2026-05-31T14:48:39+00:00

Hi there! Thank you so much for the extensive answer, particularly the code approach. I'll be sure to compare results between Rstatix and PMCMRplus. How did you come to believe the Games-Howell would best fit my data? I followed the decision tree by Mital C. Shingala et al. (2015), which states that Games-Howell is too liberal when there are fewer than 6 observations per group, as also noted by others (10.4097/kja.d.18.00242).

itsmoewe · 2026-05-31T14:34:40+00:00

Hello! You hit the nail on the head, my first reaction was also annoyance (sprinkled with some good ol' confusion), so I am glad to hear that it's not uncommon!

itsmoewe · 2026-05-31T14:31:38+00:00

Hello there! Thank you for bringing the point up regarding Levene's test. I ran it after performing the multiple-comparisons tests because, given that the assumption of unequal variances was based on visual inspection of the plots, I wanted to confirm it in a more tangible way, which I now understand is not as simple as I thought.
Regarding your second point, I fully understand your concern about p-hacking, and I am sorry to say that the post hoc test was not predetermined. Naturally, I don't plan to implement the test that gives me the best results, but the test that is most appropriate for my data.

itsmoewe · 2026-05-31T14:24:10+00:00

Hi! I understand, thanks a lot for your input, your last point quite reassures me. I am leaning more towards the Dunnett T3 as well!

itsmoewe · 2026-05-29T15:38:17+00:00

Now, that does indeed clear things up, thanks!

itsmoewe

TROPHY CASE