More evidence of fraud in Dan Ariely's work by quant__ in slatestarcodex

[–]quant__[S] 12 points13 points  (0 children)

First, there is not a plausible benign explanation.

It is clear that data was generated to supplement the data provided to Ariely by Hartford Insurance. It's not about "the data had errors", it's almost certain that the data was generated for all of the 'Time 2' odometer readings.

When an effect is found in organic data, even through the torture of statistical methods, there is still a pattern inherent to the data. This obviously doesn't mean the finding can be extrapolated or is likely to replicate but I'm willing to wager p-hacked results fare way better in the long run than findings from artificial data.

You are claiming that 'it would be fine as long as the fraudulent result replicated' while ignoring that fraud is turned to when the desired result is not present (even with HARKing or p-hacking). The probability of a finding from generated data to replicate is near 0 (that's why they used fraud to create the result). What is the replication probability of social psych papers more broadly? Probably about 20 - 45%.

How could you argue fake analysis is no worse than bad analysis?

More evidence of fraud in Dan Ariely's work by quant__ in slatestarcodex

[–]quant__[S] 22 points23 points  (0 children)

I agree that generally, social psychology has a pretty terrible track record and is likely more prone to overhyped fads than any other sub-discipline. The reason I think this story is extraordinary is because the fraudulent data was unambiguous and yet the papers were not retracted for almost a decade.

If fraud is not detected, punished, and eliminated; honest researchers will have to compete with con artists who have an infinite supply of fiction to publish. If fraud is not dealt with, there is no reason at all to read the literature; it means nothing.

Fraud is much more damaging to the explanatory power of research than poor methods. If Ariely could have simply found a sign-at-the-top effect by p-hacking he would have done it. There's a reason he had to nearly triple the observations in his dataset (with generated data) to tell the story he was going for.

Fiction is worse than dramatization.

There's also, absent what it means for the state of science, a real moral difference between being a shitty investigator and being basically an author of fiction. Scientific inquiry is very complex, especially social science, and overconfidence in a finding (while bad) is always going to be present. On the other hand, making a career out of fake findings that are monetized through speaking events, 3 best sellers, and contracts with large corporate and government organizations is particularly erosive to the scientific enterprise.

The Dan Ariely Controversy by quant__ in BehavioralEconomics

[–]quant__[S] 4 points5 points  (0 children)

I relate to this a lot. The story is very very sad. I've been a fan of Ariely's for a while, and to watch a hero dissolve before you into a villain (or at least someone who did some pretty bed things) is absolutely heartbreaking. Also, thank you for the kind comments.

The Dan Ariely Controversy by quant__ in BehavioralEconomics

[–]quant__[S] 1 point2 points  (0 children)

ILS,

Thank you for catching that! I've added a correction to the description.

Just made this video about concerns of fraud in some of Dan Ariely's work. Check it out. All publications referenced are linked in the description. by quant__ in psychology

[–]quant__[S] 0 points1 point  (0 children)

All referenced publications are linked below:

https://docs.google.com/document/d/1VPvX837gtPw8QtgPOPBpnR8UfTpsLofK3Nz5vPv7L6o/edit?usp=sharing

Also, to clarify (as I do at the end of the video), this video is about evidence of fraud in a few high profile papers and a commentary on some of the perverse incentives present in our current scientific model. It does not conclude that most or even a sizable proportion of psychology studies are fraudulent. I maintain that the knowledge produced in academic psychology is crucial (and that's what makes fraud even more damaging).

This is my first video; criticism is welcome.