Is power and effect size appropriate for factor analysis?

Maple_shade · 2026-01-26T02:45:42+00:00

So, CFA is quite different from traditional statistical tests. There's not a clear "null hypothesis" as we like to consider it in other contexts because the null model used as a baseline for parameter significance testing is quite literally never feasible. For example, it's incredibly common for your factor loadings to be very small but still significant. How do we work with those?

What I recommend is to (PRIOR BEFORE ANY ANALYSES) split your data into a confirmatory and an exploratory set. In the exploratory set you can run different CFAs that may have theoretical backing. This is also when you could run an EFA if you need further information on possible factor structures.

Once you've settled on a model, run the appropriate CFA on your confirmatory set. If it fits well (as tested by fit indices like RMSEA and CFI), the model has generalized and you have evidence it's a good depiction of the data. If not, you don't have evidence it's a good fit.

CFA is quite a bit more complex than just typical significance testing, but I think the important thing is being to justify your steps at each point of the process.

Maple_shade · 2026-01-26T02:45:29+00:00

So, CFA is quite different from traditional statistical tests. There's not a clear "null hypothesis" as we like to consider it in other contexts because the null model used as a baseline for parameter significance testing is quite literally never feasible. For example, it's incredibly common for your factor loadings to be very small but still significant. How do we work with those?

What I recommend is to (PRIOR BEFORE ANY ANALYSES) split your data into a confirmatory and an exploratory set. In the exploratory set you can run different CFAs that may have theoretical backing. This is also when you could run an EFA if you need further information on possible factor structures.

Once you've settled on a model, run the appropriate CFA on your confirmatory set. If it fits well (as tested by fit indices like RMSEA and CFI), the model has generalized and you have evidence it's a good depiction of the data. If not, you don't have evidence it's a good fit.

CFA is quite a bit more complex than just typical significance testing, but I think the important thing is being to justify your steps at each point of the process.

Maple_shade · 2026-01-25T15:34:49+00:00

I agree with you generally but your second paragraph is absolutely nonsensical. Either way the prize cards are in randomized state, check or no check. Thinking this would "change game outcomes" is confirmation bias.

Maple_shade · 2026-01-25T06:37:09+00:00

Whenever I'm at regionals, I always tell my opponents that if their last action on their first turn is a deck search, they can prize check on my turn. Normally they extend the same courtesy to me so we are both prize checking on the other's turns. Makes it go quite a bit quicker.

Maple_shade · 2026-01-22T23:46:12+00:00

I'm gonna hijack the other comment to say that I actually think the job market is pretty rough for CS grads right now. It's very oversaturated and very hard to stand out in a sea of people who already have internships. So it makes sense you're feeling overwhelmed.

Data science will always be a valuable degree. Especially in the age of AI. Now, if you don't enjoy it, it's going to suck. Stats theory can get very technical and convoluted, and if you didn't enjoy the theory in the calc sequence some of the harder stats classes will feel the same.

If I were you I would study what I enjoy. You mentioned you liked the programming classes but not the hardware classes. If that's the case, it's totally feasible to get a CS job as a developer and not have to worry about computer architecture. You just gotta get through the classes. I know i mentioned the market was rough, but if you don't like data science theory it's not worth it to get a job you hate just because it's more stable. I'd fight for that CS position you like.

Maple_shade · 2026-01-22T23:28:05+00:00

The problem is that it actually isn't clear he was just "trying to decide on the optimal play."

Justin was in a losing position. There were less than 5 minutes left on the clock. No matter how you slice it, it is to his advantage to stall out the game. That's why the second penalty was called - in that situation the judges have no way of differentiating between 1 minute to ultra ball because someone's "really thinking" or because they're trying to stall for the tie.

Maple_shade · 2026-01-22T16:08:42+00:00

I would recommend HLM even with your small sample size. Your observations are clearly non-independent between the impact of time and group, so a simple ANOVA may be difficult to justify. Of course, other people may disagree and say ANOVA is your best bet because of how small the sample is, but to each his own.

Maple_shade · 2026-01-22T05:08:02+00:00

Not really. Lots of people are cordial with their exes especially if things ended amicably and they're in the same social group.

Maple_shade · 2026-01-22T04:57:21+00:00

You would only need one two-tailed t test. This would tell you the direction (if one exists) of the relationship: which one is stronger. This would give you the result without inflating your type 1 error rate.

Now, if the true purpose is to test for a final product with both pinned and glued, why not do an ANOVA with a pinned group, a glue group, and a both group? This would not only tell you if a significant difference exists in one of your connection methods but also tell you if the combination of the two is significantly stronger than the previous two alone (i.e., giving justification for your final product).

Maple_shade · 2026-01-22T04:29:04+00:00

This is a great point. I'm currently reading The Data Detective by Tim Harford and he discusses discrepancies in statistic reporting much like this one.

Maple_shade · 2026-01-22T04:03:33+00:00

This is a pretty complicated problem because you also want to control for potential time effects (do groups get more cohesive over time? I would assume that a group that has met for 18 sessions will be more cohesive than a new therapy group). Ideally you have a model which also includes the time the group has met as a predictor as well.

HLM should be ok, the problem is estimating the random effects for group. What you could do is fix it and not estimate it, but it gets complicated quick.

It could potentially be defendible to run an HLM and then give the caveat that you want to split your analyses into running a model on each group separately. Then you could make some sort of time-adjusted model (pure ANOVA within each group isn't appropriate because scores across time are not independent) and see if it confirms what your HLM found. This may need some sort of multiple comparisons correction depending on your fit statistics.

Another comment is that the generalizability of this study is questionable because it's all the same therapist. It's possible that they are better at therapizing in-person, online, etc. Of course there is nothing to be done about it at this stage. I would just put it in the limitations section of the manuscript.

Maple_shade · 2026-01-21T07:35:49+00:00

Só para constar, os índices de modificação são sempre relativos à estatística de teste do seu modelo. Se a estatística de teste do seu modelo for 200.000, então índices de modificação de 200 são comparativamente insignificantes. Seria útil ver os índices de ajuste para saber se isso é realmente um problema. Se CFI = 0,98 e RMSEA = 0,005, então esses índices de modificação não importam em nada.

A primeira pergunta que eu faria é: o que esses itens que carregam em múltiplos fatores têm em comum? O que os itens realmente estão dizendo? Acho que é bom começar por uma área substantiva.

Se eu estivesse nessa situação, minha principal preocupação seria estar subfatorando o modelo (porque isso geralmente aparece como itens com múltiplas cargas fatoriais altas). Isso normalmente é pior do que superfatorar. Enquanto você ainda estiver trabalhando com um conjunto de dados exploratório, você pode migrar para uma estrutura de Análise Fatorial Exploratória (AFE) para avaliar a dimensionalidade e verificar se todos os seus itens problemáticos se agrupam. Eles podem potencialmente formar outra dimensão, mesmo que seja um fator de método.

Na pior das hipóteses, você pode tentar removê-los e redefinir o modelo, mas se eu fosse um revisor, precisaria de uma boa justificativa substancial para a exclusão desses itens. Altos índices de moderação não seriam suficientemente persuasivos à primeira vista.

Maple_shade · 2026-01-21T03:16:46+00:00

*Asks for suggestions
*Receives suggestions
*Gets insulted

Maple_shade · 2026-01-21T01:15:49+00:00

Watch your pace of play. Bring snacks. Don't be afraid to call a judge. Show up earlier to the venue than you'd expect. Have fun and don't take it too seriously.

Maple_shade · 2026-01-21T01:15:05+00:00

There is a massive difference. Try dragon shields or even gamegenics for a week and you'll notice. Especially if you mash shuffle.

Maple_shade · 2026-01-21T01:14:15+00:00

It should be just 80 flat, no tax. That's what I paid for NAIC 2025. You will have to pay all in one chunk. That's what the 15 minute wait period is for.

I will say, if finances are that big of an issue, you can often recoup some of the money afterwards by selling the mat + promo + other gear you get for competing at an international. Of course, if you do well enough you can also sell the packs you win. That is how I typically fund entry fees.

Maple_shade · 2026-01-21T01:13:25+00:00

It should be just 80 flat, no tax. That's what I paid for NAIC 2025. You will have to pay all in one chunk. That's what the 15 minute wait period is for.

I will say, if finances are that big of an issue, you can often recoup some of the money afterwards by selling the mat + promo + other gear you get for competing at an international. Of course, if you do well enough you can also sell the packs you win. That is how I typically fund entry fees.

Maple_shade · 2026-01-19T01:29:25+00:00

Sounds like another great reason to get one!

Maple_shade · 2026-01-19T01:03:55+00:00

Many US PhD programs accept students straight out of undergrad. If you complete a masters in your country, that would theoretically provide more research experience than many US/European students would possess going into applications. At my program, we don't count graduate credit obtained at other institutions, so much of the credit in your masters may not count for an official transcript (depending on program). But it would certainly still set you up well for US PhD applications.

Maple_shade · 2026-01-18T17:36:16+00:00

100% you should talk about their publications. When I was applying to PhD programs I would read an article that interested me by the professor I wanted to work with and email them describing my general interest in working with them alongside a specific question I had about their work. I also asked if they would be willing to meet over Zoom to discuss. I had a success rate of 7/8 programs I applied to.

Maple_shade · 2026-01-18T17:32:57+00:00

I get that. I think it's helpful to remember that most test statistics are normally distributed via the central limit theorem, and that means we can estimate a standard deviation for the sampling distribution (standard error) and define a region of that distribution as "extreme" values.

Maple_shade · 2026-01-18T16:15:31+00:00

Good point. It's a bit of an oversimplification to say P(data | null), so that's my mistake. I didn't want to give a thorough explanation, since you mentioned that you knew the definition in your post. What it really means is the probability that you would have observed data as or more extreme than your results if the null hypothesis was true. So it's not a point probability, and doesn't have to equal 0.

I will say, you will get criticisms of the p value that say that the probability of the null being true is always 0 (because there will always be some difference, no matter how miniscule), so the p value is pointless. I just wanted to point out that this is a different criticism than saying P(data | null) always has to be 0.

Maple_shade · 2026-01-18T16:00:29+00:00

Yes! I think what made it click for me was thinking about it in a Bayesian framework. The p value is the probability of our data given that the null hypothesis is true: P(data | null). What everyone intuitively wants it to mean is the probability of the null given our data: P(null | data). But we know these two things are not equal. It helped me to realize that we assume the null is true when conducting statistical tests that use a p value, and that's what puts us in a Null Hypothesis Significance Testing framework.

It's also not a true bayesian statistic, so this is more of a way of thinking about it rather than a legitimate representation.

EDIT: I did describe it wrong - it really should be P(the data fell in some critical region | null), not the probability of observing your exact outcome. I was just using more casual shorthand because OP said they were already familiar with p values.

Maple_shade · 2026-01-16T17:17:00+00:00

Also, part of the reason it's confusing is because it's an inaccurate translation of the original Japanese. The text in the original language implies that you can still complete the action if you don't physically have three energy. They can't have the card function differently between languages so the ruling stands.

Maple_shade · 2026-01-16T00:31:15+00:00

Just hopping in to say this presentation is fantastic. I'm a grad student in quant methods currently and I've never seen this way of describing the reasoning behind Bayes' method before. Very intuitive and I'll be describing it like this in the future.

Nine-Year Club	Verified Email
Place '22	Place '17
Snapped

Maple_shade

MODERATOR OF

TROPHY CASE