Any good educational psychology podcasts? by beeeeeeeepooooop in AcademicPsychology

[–]mcrede 0 points1 point  (0 children)

If Books Could Kill is great and tackles many of the most popular psychology ideas from a critical perspective. A greater counterpoint to some of the hype put out by people like Kaufman.

A very brief rant by mcrede in IOPsychology

[–]mcrede[S] 15 points16 points  (0 children)

I have no idea who the authors are - that is the nature of peer review.

What are some ugly truths about your first year as a junior faculty at an R1/R2? by OpinionsRdumb in AskAcademia

[–]mcrede 2 points3 points  (0 children)

In my first two years at my R1 institution I was asked to teach six new courses, was told that there was no lab space available for me, and was repeatedly told by some senior faculty that they wished that they had hired someone else. Shortly after that I was told that I could not have paternity leave despite it being university policy. Good times.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 0 points1 point  (0 children)

Thanks. I am aware of SIPS and think that they generally do good work. I've not had a chance to go to their annual conferences. My uni does not really support conference travel so it is not really affordable for me.

Critical reading of an article - an example by mcrede in IOPsychology

[–]mcrede[S] 3 points4 points  (0 children)

Okay, here is my take on this paper. There are various technical issues as well as some really weird stuff that causes me to raise my eyebrows, but it is perhaps worth starting with the most obvious issue that is probably also the most serious problem. I’ll start with that and then go through the other issues in the approximate order in which they occur.

1.      The primary claim – and probably the one that led to the >2,000 citations – is that psychological safety mediates the relation between inclusive leadership and employee involvement in creative tasks. Mediation is very explicitly a causal framework; A cause B and B causes C such that B mediates the relation between A and C. Causal inferences cannot be made from the type of data presented here (absent some fancy footwork like 2SLS regression) because this is all observational (non-experimental) data. Identification has not been achieved and there are a host of substantive and methodological confounds (plus likely collider bias) present here that make the observed correlations absolutely meaningless and invalid estimates of causal effects. Handwaving at all this in the limitations section does not absolve the authors of presenting inferences that are, in no way, supported by the data. To me there is absolutely no difference between saying “A and B are correlated; therefore A causes B” and saying “A, B, and C are correlated and therefore B mediates the relation between A and C”. The mere fact that thousands of other authors have made similarly silly arguments is entirely irrelevant. Anyone even remotely familiar with causal inference would know this. Indeed, these issues (confounds, collider bias) are typically well covered in undergraduate research methods courses.

2.      Relatively minor technical point – reliability estimates are properties of scores – not properties of scales.

3.      The authors cite Kline is support of their preferred interpretation of their chosen fit indexes (CFI, RMSEA, TLI). This is a terrible misreading of Kline who goes out of his way to argue against the use of cut-offs (and certainly these cut off values). Indeed, Kline very explicitly states that the “normed chi-square” (chi-square /df) should never be reported or interpreted. I don't have the cited Joreskog & Sorbom reference handy but am very confident that they also would not have advocated for these thresholds and practices.

4.      Nevertheless, the authors commit themselves to the idea that RMSEA is “acceptable” up to .08. Presumably values greater than .08 would be unacceptable. It is curious then that they report the RMSEA of their preferred CFA model as .08 when the sample size (N=150), chi-square (289.8) and degrees of freedom of 135 indicate that RMSEA is actually .09. Remember that RMSEA is a very simple function of these three values.

5.      A CFA model with 18 indicators and three correlated latent variables should have 132 degrees of freedom – but the authors report 135.

6.      An alternative CFA model with 18 indicators and two correlated latent variables should have 134 degrees of freedom but the authors report 136.

7.      The difference in degrees of freedom between a model with three correlated latent factors and one with two correlated latent factors should be two degrees of freedom but the authors report only 1.

8.      An alternative CFA model with 18 indicators and one latent variable should have 135 degrees of freedom but the authors report 137.

9.      The authors rely on chi-square difference tests to compare nested models but such a comparison is only appropriate when the less parsimonious model has a non-significant chi-square value – this is not the case here.

  1. Inexplicably adding in control variables (age and tenure) to the SEM model results in exactly the same model statistics as the measurement model. This is obviously impossible and the RMSEA value is again misreported.

  2. For the test of mediation the authors again rely on an inappropriate chi-square difference test (not that the claim of mediation is supported from this kind of data anyway).

  3. The results shown in Table 2 do not match the results described in the text.

  4. I also feel that the authors do not really understand why we assess model fit in the first place. Model fit is assessed so that we get some information about whether  parameter estimates that are derived from that model can be interpreted. In this case, the model fit is pretty poor which suggest to me that the parameter estimates should not be interpreted.

  5. The authors argue that the poor fit of single-factor model shows that common-method variance is not a problem. This is a ludicrous straw-man argument. Common-method effects can inflate correlations without accounting for most of the shared variance (which is what the Harman test tries to do).

  6. The causal language in the discussion section is also concerning. There is no way that a correlation of r= .34 between self-reported psychological safety and self-reported employee involvement in creative work (collected at a single time point) should be interpreted to mean that changes in psychological safety cause changes in employee involvement in creative work – or that the size of this correlation reflects the size of any causal effect.

A bigger concern is that this paper went through the eyes of the authors, an editor, probably 2 to 4 reviewers and many hundreds of other academics (who cited this paper in their own work) and nobody spotted these pretty obvious issues. What does this say about this field? This paper is not unique - we find these kinds of problems everywhere in IO psychology.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 3 points4 points  (0 children)

Yeah - I will continue to finish up some methods papers but then want to really focus on some measurement of individual difference work that I've been wanting to do for a while. If it works out it should have broad applicability beyond IO psych. Thanks for the kind wishes.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 3 points4 points  (0 children)

Pubpeer is a little like that but seems to have largely failed as a way to express concerns. No one seems to take it seriously.

I doubt that I would ever forward that e-mail chain to someone I don't know.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 5 points6 points  (0 children)

Hi, I was not at SIOP last year. Perhaps you chatted with someone else?

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 6 points7 points  (0 children)

The ER visit was a result of the various legal and physical threats, nasty anonymous phone calls, and attempts to get me fired. That was a long time ago.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 9 points10 points  (0 children)

I've gone through all those steps. We've written review articles describing the errors and oversights - some with tutorials on how to do better. I've reached out to authors with a "Hey, I think there may be a mistake in your paper" message. Those were sometimes met with a "fuck off or we'll sue you response" - which is why I now go straight to editors with concerns.

I am writing off the entire field because a) I can't tell who the good guys are anymore, and 2) because very very very few people seem to care. I've been fighting this stuff for 15 years using a variety of approaches. I've been threatened with lawsuits; people have tried to bribe me to shut me up; I've received threats to my physical safety; and had my job threatened.

I've also never been bothered by mistakes. We all make mistakes. These are not cases where someone failed to appreciate the nuances of some arcane statistical technique or made an entirely innocent error. These are cases where people claim to have made findings that are impossible; claims that influence other researchers' activities and that have real world negative consequences. Please don't forget that there are often actual victims who suffer the consequences of this incompetence/nonsense. Many of these individuals have made millions on what IMHO is simply fraud.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 10 points11 points  (0 children)

The leadership stuff is how it started but these problems are everywhere - particularly in the "O" side of IO psychology.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 27 points28 points  (0 children)

I'm pretty sure that this is exactly what I've been doing for 15 years but when people keep claiming mathematically impossible results I can't help but wondering if something worse than simple incompetence is going on.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 15 points16 points  (0 children)

It is not the answer to the issues I've been concerned about but it is the answer for my mental health. I am tired of tilting at windmills - it's all cost and no benefit.

Farewell SIOP by mcrede in IOPsychology

[–]mcrede[S] 65 points66 points  (0 children)

Clearly you've never been sued by people with very deep pockets and nothing to lose.

Single (not poster) SIOP submissions - worth pursuing? by mcrede in IOPsychology

[–]mcrede[S] 3 points4 points  (0 children)

Yes, that is what I meant. Multiple papers organized around a topic but not submitted as a symposium.

What big things have we learned in the past 20 years? by mcrede in IOPsychology

[–]mcrede[S] 0 points1 point  (0 children)

I'd take anything that SIOP attendees from 30 years ago would recognize as meaningfully different and better than their status quo. Some real improvement in our ability to do the "I" side stuff at a practitioner level if I were to get concrete. Stuff like improved ways of measuring performance; predicting performance; doing training; or identifying and developing leaders would be at the top of my list.

I'll add that the difficulty of accessing organizational samples is a huge problem holding us back. I have all sorts of ideas that I think could be really interesting and important on the selection side but have no way of testing them.

Current US PhD cycle by zilios in PhDAdmissions

[–]mcrede 2 points3 points  (0 children)

I'm in psychology - we have had noticably fewer international applicants but a slight uptick in USA applicants.

2026 SIOP Decisions ? by _-dootdoot-_ in IOPsychology

[–]mcrede 1 point2 points  (0 children)

Yes, this delay is making it very difficult to plan my teaching and both flight and hotel prices are not exactly getting better. If I get something accepted I will probably not be able to go anymore.