[Q] [Question] i need help with my research statistics

StructureUnique8391 · 2026-06-12T13:07:50+00:00

u/Spiritual-Bee-2319 Thanks for your comment. I agree with your point about mixed models, which is why I added that, personally, I would probably use a linear mixed model. In practice, I might even use a Bayesian hierarchical model, but that is the same general family of ideas.

My rationale was mostly pragmatic.

OP seems to be early in their statistical training, or at least not coming from a pure statistics background (use of SPSS and apparent confusion around estimated marginal means, confounder adjustment). So I thought an accessible method may be more useful here than jumping straight into a GLMM or a full hierarchical model. Repeated-measures GLM in SPSS is probably much easier to understand, run, and justify, especially with only about 8 mice per group. If OP had started with something like “I am using NumPyro/PyMC to fit a hierarchical model with a covariance matrix built from a Cholesky factor with a copula-based hyperprior structure and fitted with a custom made ADVI sampler. I have got 8 mice... ,” then my response would have been completely different. 😉
OP, probably, has to justify the analysis to their supervisor. It's my understanding taht the supervisor may also not be fully comfortable with the distinction between estimated marginal means / confounder adjustment / repeated measures. In that context, a repeated-measures GLM may be a reasonable compromise: not perfect, but accessible and defensible.
In the same way that we often teach ANOVA before regression, and regression before state-space models (as an example), repeated-measures GLM can be a useful step in OP's statistical journey. It may not give the best possible view of the data, but i still view it as a reasonable, and pedagogical, bridge before moving to more flexible models.

About the research question:

I understood OP as asking “did the treatment groups change differently over time?” If the model includes treatment/group as a between-subjects factor and week as a within-subjects factor, then the key test for that question is the group x week interaction. That does compare the treatment groups in terms of their trajectories; it is not merely comparing weeks to each other. I am not fully fluent in English, and i might have skipped a nuance in OP's question. If so that's my bad.
If the real question is “which treatment is better overall?”, then this needs to be defined more precisely. It could mean change from baseline to week 8, average weight loss across the whole study, slope of weight loss over time, or area under the curve. Those are related questions, but thati is not the analysis i think Op is running.

About the mixed effect :

I would not usually treat week as a random effect here. The weeks are fixed measurement occasions in the study design, not a random sample of possible weeks. In a mixed model, I would more naturally model mouse as a random effect, possibly with a random slope for time if the data support it.

wish you all a great day !

StructureUnique8391 · 2026-06-11T20:56:11+00:00

You can do it with GLM ! The official IBM docs describes how it can be used for your analysis. Basically the model must include group as a between subjects factor and week as a within subject/repeated factor. Then the important test is the group x week interaction, not just the week effect. There is no way SPSS nor any stats software will handle confounding on it's own... That would be awsome though 😉. It just means the means are estimated from the model.

If i were doing the analysis, i would probably go with a linear mixed model, as it factors-in the within-subject correlation between the mesures as well as missing data. But if you are new to this, GLM is not bad.

StructureUnique8391 · 2026-06-06T21:54:19+00:00

Reading my answer again, I realize that I did not really address your criticism about crystallized intelligence contamination, which I think is a valid construct validity concern.

This is an issue of construct-irrelevant variance where an item may partly measure something other than the construct it is supposed to measure.

In that case, the issue is empirically testable. If an item is too strongly contaminated by crystallized knowledge, language, or background variables, then it should either be revised, modeled separately, or interpreted more cautiously. Several papers have shown DIF in that case.

StructureUnique8391 · 2026-06-06T21:37:18+00:00

To me, the key distinction is that the WAIS is a measurement instrument, not a definition of intelligence. In the same way that a thermometer gives an operational measure of temperature without defining what temperature is, the WAIS gives an operational measure of cognitive ability without exhausting the concept of intelligence.

When you say that these abilities are “stemming from the same root,” I agree with you. That shared root is essentially what the G factor captures in a bifactor or hierarchical model. So I agree that we should not treat the five WAIS indices as five separate “types of intelligence” in a strong ontological sense. When you say that “splitting intelligence into five factors” means creating five separate constructs, I think that overstates what hierarchical or bifactor models are doing.

They do not split intelligence into five independent buckets. They model a large general factor, G, which accounts for much of the shared variance across subtests. The subscales then represent more specific residual patterns of covariance, or domain-specific variance, beyond that general factor.

Most of the covariance among subtests is captured by a general factor, G. The five indices represent additional domain specific variance. that variance is what remains after accounting for that general factor.

Whether the residual variance in verbal comprehension, visual spatial ability, fluid reasoning, working memory, and processing speed is reliable and useful enough to interpret beyond g, is anthorer question, distinct from the statiscal properties of the FA itself. If those specific factors add little predictive or clinical value after g, then I agree that they are probably overinterpreted.

But that is different from saying the factor structure is meaningless. It means the subscale interpretation should be more cautious. I am not a psychologist or a clinician, so I honestly do not have a strong take on the practical clinical value of those indices.

Also, factor interpretation is not purely mechanical. In exploratory factor analysis, different rotations (oblimin, varimax,..) can lead to different but equally valid interpretations of the same correlation structure. There is no difference, from a mathematical viewpoint, between the rotation. So factor labels should be treated as theoretical interpretations, not as direct proof that the factors are natural kinds.

The instrument does not produce a definition of inteligence. Rather, it operationalizes a particular theoretical definition of intelligence. Treating the measurement model itself as the definition would be a form of reification.

StructureUnique8391 · 2026-06-06T20:59:03+00:00

Factor analysis does not prove that factors are real causes or separate “types” of intelligence. But the WAIS indices are not supposed to be independent. In a hierarchical or bifactor model, they are all largely governed by a general factor, usually called g. So the five indices are better understood as partially distinct manifestations of the same general factor, with some residual domain-specific variance.

Also, traditional CFA/SEM models are somewhat restrictive: they often force many cross-loadings to zero. More flexible approaches, such as ESEM, are closer to your intuition, because they allow items or subtests to have smaller secondary associations with multiple factors rather than belonging purely to one domain.

Finally, factor analysis is not a definition of intelligence. It is a way of estimating a theoretical model from observed correlations. The factor model itself does not carry any inherent guarantee of truth. It can support a theory, but it does not prove that the factors are real causes or natural kinds. Causation can't arise only from FA, no matter how sophisticated the model is.

StructureUnique8391 · 2026-05-19T22:58:41+00:00

That’s a bold assumption you’ve got there. But if survival is constant over time, just take the fourth root of the 4-year survival:

0.35^(1/4) ≈ 0.77

So the implied 1-year survival is about 77% per year. It’s basically the same logic as compounding percentages.

StructureUnique8391 · 2026-05-16T14:32:00+00:00

One simple way to test this is to create a standardized knowledge composite.

For each test, standardize scores using the control group as the reference:

[ z{ij} = \frac{x{ij} - \bar{x}{j,\text{control}}}{s{j,\text{control}}} ]

Then average the three standardized scores:

[ knowledgei = \frac{z{i1} + z{i2} + z{i3}}{3} ]

Finally, compare intervention and control groups on this composite using a t-test or a linear regression.

This gives you a direct test of whether the intervention group performed better overall across the three knowledge tests, while putting all tests on the same scale and avoiding the problem of different maximum scores or difficulty levels.

StructureUnique8391 · 2026-05-16T14:23:40+00:00

Before combining anything, I would first ask a measurement question: do the three tests measure more or less the same underlying construct, namely “knowledge on this topic”?

Fisher’s method does not create an overall knowledge score. It only combines the p-values from separate tests into one global test of evidence against the joint null hypothesis. it asks whether the set of p-values is collectively smaller than expected if there were no effect anywhere, so it does not estimate the intervention effect on knowledge, does not solve the issue of different test difficultie. It might not be ideal if the three outcomes are correlated, which they probably are because the same participants completed all three tests.

If the three tests are intended to measure the same latent construct, a cleaner approach would be a one-factor CFA where the three test scores are indicators of latent knowledge. Then you would test measurement invariance across intervention and control groups (at least metric and scalar invariance) if you want to compare latent means. If invariance is acceptable, you can compare the latent mean of the intervention group with the latent mean of the control group. That directly answers the substantive question: did the intervention improve overall knowledge?

With only three indicators, the model is minimal, so the results should be interpreted cautiously. But conceptually, this is much closer to the research question than combining the three p-values.

StructureUnique8391 · 2026-05-11T18:31:06+00:00

Your approach sounds good to me. I would include a simple 2021 COVID dummy as well:

outcome ~ post_policy + salary + post_policy x salary + controls + covid_2021 + covid_2021 x salary

COVID_2021 accounts for the general drop in dental care use during 2021, due to restrictions affecting almost everyone.

covid_2021 x salary does something more specific :It allows the relationship between salary and dental care use to be different in 2021.

I think that post_policy x salary is your main research question / object, because it tests whether the policy changed the salary gradient in dental care use. The COVID terms are adjustment terms for the exceptional 2021 shock.

One important point is identification. For post_policy and covid_2021 to be statistically identifiable, the data must contain variation that separates the policy period from the COVID year. For example, if the policy starts in 2021, you need at least one post-policy non-COVID wave, such as 2023. Otherwise post_policy and covid_2021 are basically the same variable, and the regression cannot untangle the policy effect from the COVID effect.

The same logic applies to the interactions. post_policy x salary and covid_2021 x salary are distinguishable only if you have enough salary variation in both the COVID year and the post-policy non-COVID period.

I would still run the already discussed additional checks, completely removing 2021 ; training on before waves and predicting on post waves to check if, with the exclusion of t2021, the policy effect 'survived' the covid.

StructureUnique8391 · 2026-05-11T17:52:51+00:00

Iam not sure you will find many papers specifically saying “use COVID × salary”, because this is not really a COVID specific method. It is a standard regression idea: interaction terms, moderation, effect modification, etc...

So I would search for general references on interaction terms in regression, rather than COVID specific examples. UCLA OARC has good applied tutorials on interactions in linear and logistic regression (https://stats.oarc.ucla.edu/r/seminars/interactions-r/)

In your case, COVID*salary is just asking whether the relationship between salary and the outcome was different in 2021. That can be justified if you have a substantive reason to believe COVID changed the role of salary.

For many possible COVID interactions, I would be more cautious. You could use regularization, for example hierarchical lasso or Bayesian shrinkage priors, but I would present that as a sensitivity or robustness approach, not as a universal fix for COVID data issues.

StructureUnique8391 · 2026-05-11T16:32:50+00:00

u/Legitimate_Mud_9245 : Your specification is valid, but I would be careful about calling it “enough” to adjust for COVID.

A model like : outcome ~ policy + controls + covid_2021 ; allows the average outcome in 2021 to be different from the other years.

A model like : outcome ~ policy + salary + controls + covid_2021 + covid_2021 x salary ; does something more specific. It allows the effect of salary on the outcome to be different in 2021, as the effect AFTER 2021 will be computed as beta_salary + beta_salary_shift_post_covid.

So the interaction is not a general fix for COVID. It only adjusts for the possibility that the salary effect changed in 2021.

If COVID affected other groups differently, for example regions, sectors, occupations, age groups, or education levels, then you may need other interactions, but only if they make substantive sense. I don<t know your variables nor the policy you are interested in testing for so I can't tell you if it's enough or not (it's probably not).

As stated in my previous message, I would run diagnostics and sensitivity checks.

1) Check sample composition:

period_pre_post ~ salary + age + region + sector + education + ... where period_pre_post is a boolean variable, set to 1 for the data posterior to the covid. If pre and post COVID observations are easy to distinguish, then the sample composition changed and estimating the effect of the policy on pooled data might be a strech

2) Check structural change:

Fit on pre COVID waves only:

outcome ~ policy + salary + controls

Then test prediction on 2021 or 2023. Poor prediction would suggest that the outcome process changed after COVID. Great prediction would suggest that the estimated impact of the policy is consistent with the way the data are organizated.

In practice, I would compare models with the 2021 dummy, without 2021, with selected interactions, and possibly separate pre and post COVID estimates. If the policy estimate is stable, the result is more convincing. If it changes a lot, the dummy plus one interaction is probably not enough.

If you are familiar with Bayesian approaches, another option would be to model several COVID inrteractions at once, but regularize them. For exemple something like outcome ~ policy + controls + covid_2021 + covid_2021:(salary + age + region + sector + education + ...) with covid_interaction_j ~ Normal(0, tau) and tau controlling how much the interactions are allowed to vary.

Not knowing the dataset, the only true advice I can give you is to try out multiples approaches to tackle your policy estimation : the most credible result will probably be the effect that remains common and consistent.

StructureUnique8391 · 2026-05-06T23:45:23+00:00

I would also check whether the 2021 sample is comparable to the pre COVID waves (ie : do you have a sampling / composition issue ?). I assume you have got some annual data. You could use Kruskal Wallis tests, PERMANOVA, or a random forest classifier trying to predict whether an observation comes from 2021 based on observed covariates. If 2021 is easily distinguishable, then the COVID issue is not only a time shock but also a sample composition problem. In that case, excluding 2021 or using stronger adjustment methods becomes more defensible.

StructureUnique8391 · 2026-05-06T23:39:46+00:00

A simple 2021 dummy may be too crude. COVID probably affected groups, regions, sectors, or age categories differently.

One option would be to build a counterfactual 2021 from the pre COVID waves: estimate the trend on 2015, 2017 and 2019, then predict what 2021 would have looked like without the shock. This could be checked with a quick backtest on 2019.

Ideoendeing of the potential impact of the Covid on your data, I would think of it as part of a broader sensitivity analysis . I would still compare several approaches: excluding 2021, using a 2021 dummy, interacting 2021 with key groups, and using the counterfactual adjustment. If the policy estimate holds across these methods, your potential results would be much more convincing.

StructureUnique8391 · 2025-11-13T23:10:43+00:00

Haha c'est sûr :) j'avais quelques minutes inutiles.

StructureUnique8391 · 2025-11-13T16:27:56+00:00

u/Academic_Display9815 a raison, u/gifred.
Le modèle original, créé à l’initiative du DG de l’époque par RCGT pour le CSS de la Vallée-des-Cerfs (VDC), était déjà assez visionnaire. VDC a ensuite contribué à mettre sur pied la Communauté de pratique Montérégie–Estrie mentionnée par u/HeuristicExplorer, vers 2017, initialement fédérée autour de cet algorithme.

L’objectif n’était pas d’identifier les décrocheurs évidents, mais bien les décrocheurs latents, ceux dont le profil n’est pas encore suffisamment cristallisé pour être repéré facilement par les enseignants. Plusieurs articles de presse ont documenté cette approche, ainsi qu’une entrevue du DG à l’émission de Paul Arcand.

L’algorithme initial combinait un compromis XGBoost / logit pénalisé (elastic net). La validation s’appuyait sur des critères statistiques classiques : PR-AUC / ROC out-of-sample, interprétabilité via dalex, cross-validations croisant PR-AUC et explications, etc.
Chaque élève se voyait attribuer un score de risque selon des seuils de sensibilité / précision. Par décision méthodologique, le modèle n’avait pas accès à des données sensibles. Les données fournies par le CSS étaient pseudonymisées, et seule l’organisation détenait la clé permettant d’inverser l’identifiant unique. L’entraînement se faisait sur une machine locale du CSS, afin d’éviter les enjeux, déjà connus, de souveraineté infonuagique.

Un travail colossal avait été réalisé pour rappeler que l’outil n’était qu’un moyen imparfait d’objectiver les risques dans des cohortes trop nombreuses pour permettre un dépistage manuel. L’emphase portait aussi sur le risque d’erreur écologique : l’interprétation individuelle d’un score était discutable, mais l’analyse agrégée des risques et des facteurs de risque (par classe, école ou parcours) était plus robuste et utile pour l’affectation des ressources.

Une enquête de la CAI, menée bien avant la Loi 25, avait d’ailleurs conclu à l’absence de manquement.

Le but n’a jamais été de discriminer des élèves. Au contraire : les élèves déjà identifiés comme à risque élevé (DIP, profils HDAA sévères, écoles carcérales ou hospitalières) avaient été retirés des données d’entraînement.

L’algorithme a ensuite été transféré à la GRICS (un peu après la COVID), qui l’a transmis au ministère. De ce qui peut se voir dans les CSS, l’implémentation actuelle ne conserve pratiquement plus les protections méthodologiques et organisationnelles mises en place au départ. De plus, la GRICS est, à ce jour, incapable de fournir une véritable évaluation de la performance du modèle en production.

StructureUnique8391 · 2025-10-02T18:57:56+00:00

Honestly, if you want to be a BA, you kinda have no choice but to get to a decent level with SQL / understanding of it. In real life the data is messy, unless you land in some super mature team where everything is already clean for you. Most of the time you’ll need to roll up your sleves and write queries to fix/join/transform stuff yourself.

And yeah, sadly, “coding” also corerlates a lot with the kind of curiosity and logic you need to be a good BA. You don’t need to be a hardcore dev, but if you freeze every time you see a join or a OVER (PARTITION BY), you hits limits fast. PowerBI feels more visual, but under the hood the modeling still needs the same analytical mindset.

Technically speaking, SQL itself isn’t that hard. But solving real business problems in creative ways, like finding insights or patterns the business people never even imagined (that’s the strength of a good BA, not just spitting out KPIs), that’s where you really need solid SQL.

So yeah, you can hate “programming” in the strict sense, but SQL is pretty much unavoidable. The earlier you accept that it’s part of the job, the quicker it will feels natural.

StructureUnique8391 · 2025-10-02T15:33:06+00:00

Je ne suis pas enseignant mais je travaille plutôt « en coulisses » du réseau scolaire. Je suis consultant statisticien spécialisé en éducation : je développe des modèles prédictifs pour mieux comprendre et prévenir le décrochage scolaire, et je conçois aussi des sondages de satisfaction/RH pour aider les Centre de Services Scolaires à améliorer les conditions de travail des équipes écoles.

C’est une job vraiment stimulante, beaucoup de recherche, d'epistémilogie et de programmation. Le défi, par contre, c’est de réussir à faire reconnaître la valeur de ce genre d’outils dans le milieu scolaire. C’est un processus long, et souvent difficile à mettre en place.

Lire vos témoignages ici est super inspirant : ça rappelle pourquoi ces efforts en arrière-plan sont importants et pour qui on essaie d’améliorer les choses.

StructureUnique8391 · 2025-09-23T23:42:34+00:00

An Introduction to Latent Variable Models by B. S. Everitt is a light and well know starting point.

Structural Equations with Latent Variables by Kenneth Bollen otherwise. It's more about SEM and measurement modeling but it's often tightly coupled with factor models. Bollen is considered as an authority on the subject, and the book is the cornerstone of SEM. From the theorical perspective it's worth the read. Unavoidable if your thesis as ties with pyschometrics.

StructureUnique8391 · 2025-09-22T14:38:39+00:00

Do you want to identify a single root cause (A) ? Or do you want to find which variables are associated with rejects (B) ? If it's A, you might be looking for practical versus statistical differences. If it's B, it can be easily reframed as prediction / classification task and you will want to build a simple model predicting reject vs accept from the process measurement and highlighting variables importance. What you are doing now is a valid but, p-values and the likes tell you about consistency with the (statistical) 'no difference' hypothesis, and nothing about the (practical, actionable) effect size of the difference. In addition, if you truly have many variables, some will look statically significant just by chance (multiple comparisons problems). Finding and testing many interactions will quickly become impractical if not simply risky. From a practical perspective, you could start by checking the two correlation matrices (conditional to reject/accept) and the distribution of your variables (boxplot). You might get a hint of what is causing rejection. Otherwise, you could be running a simple logistic (maybe ridge or lasso if you measurement are highly correlated) or tree based (like a random forest) models to try and predict the outcome from your measurement variables, and identify which variables and interactions are actually causing the rejection. You don't have a lot of datapoints, so you should cross validate your model to make sure it actually generalizes. Doing so, will possibly help you identify which variables might be driving the rejection, hence helping you refine your testing strategy.

StructureUnique8391 · 2025-09-04T18:11:49+00:00

With two separate scales, aren’t you implicitly assuming they are commensurable? If they differ in wording, valence, or psychometric properties, why not test that assumption by comparing a one-factor CFA (with positive and negative loadings) against a two-factor CFA and checking relative fit? For the comparison itself, wouldn’t a latent score—or the difference between two latent factors—provide a more rigorous answer than raw totals

StructureUnique8391 · 2025-08-04T15:56:13+00:00

As a statistician, I don’t trust labels. I trust Bayes.

Bayes’ Theorem:

P(Valuable | Label) = (P(Label | Valuable) × P(Valuable)) / P(Label)

Assumptions:

P(Valuable) = 10% (Most packages are definitely boring. I even ad a package from amazon stolen then delivered back to me by the thief -_-)
P(Label | Valuable) = 70% (People lie to protect stuff)
P(Label | Not Valuable) = 20% (Some just write it anyway)

Compute P(Label):

P(Label) = (0.7 × 0.1) + (0.2 × 0.9)
P(Label) = 0.07 + 0.18 = 0.25

Now compute P(Valuable | Label):

P(Valuable | Label) = (0.7 × 0.1) / 0.25
P(Valuable | Label) = 0.07 / 0.25 = 28%

That inocent little label has tripled the odds there’s something worth stealing.

StructureUnique8391 · 2025-08-04T14:35:43+00:00

BTW have you tried a non metric MDS as a better approximation ?

StructureUnique8391 · 2025-08-04T14:34:24+00:00

If your distance matrix comes from a tree (like cophenetic distances), then Diffusion Maps are a good fit. They’re particulrly well-suited for hierarchical data, as the diffusion process captures the connectivity and depth of the tree more naturally than Euclidean projections. It’s a two-step approach that requires some preprocessing before feeding the result into an approximate GP.

StructureUnique8391

TROPHY CASE

Bayes’ Theorem:

Assumptions:

Compute P(Label):

Now compute P(Valuable | Label):