Here are some recent extraordinary events:
The bloggers at Data Colada published a four-part series (1, 2, 3, 4) alleging fraud in papers co-authored by Harvard Business School professor Francesca Gino. She responded by suing both them and Harvard for $25 million.
Earlier, the Colada boys had found evidence of fraud in a paper co-authored by Duke professor Dan Ariely. The real juicy bit? There’s a paper written by both Ariely and Gino in which they might have independently faked the data for two separate studies in the same article. Oh, and the paper is about dishonesty.
Also, there’s this gem:
(Both Ariely and Gino deny any wrongdoing. Since we’re now in the business of suing blogs, let me state that I, of course, have no idea if Ariely, Gino, or anybody else ever engaged in research misconduct. There’s no evidence that I have any ideas at all! I’m just a bunch of bees!)
Gino’s coauthors are scrambling to either find out if their data is solid, or to assure others that it is. She has students who are trying to get jobs right now; God help them. Ariely still has his job, but he runs a big lab, is involved in multiple companies, and collaborates with a lot of people, so if he eventually does go down, he’ll take a lot of people with him.
All of that is bad. But there’s an extra uncomfortable fact that nobody seems to mention, perhaps because they don’t see it, or perhaps because they don’t want it to be true.
This whole debacle matters a lot socially: careers ruined, reputations in tatters, lawsuits flying. But strangely, it doesn’t seem to matter much scientifically. That is, our understanding of psychology remains unchanged. If you think of psychology as a forest, we haven’t felled a tree or even broken a branch. We’ve lost a few apples.
That might sound like a dunk on Gino and Ariely, or like a claim about how experimental psychology is wonderfully robust. It is, unfortunately, neither. It is actually a terrifying fact that you can reveal whole swaths of a scientific field to be fraudulent and it doesn’t make a difference. It’s also a chance to see exactly what’s gone wrong in psychology, and maybe how we can put it right.
Gino’s work has been cited over 33,000 times, and Ariely’s work has been cited over 66,000 times. They both got tenured professorships at elite universities. They wrote books, some of which became bestsellers. They gave big TED talks and lots of people watched them. By every conventional metric of success, these folks were killing it.
Now let’s imagine every allegation of fraud is true, and everything Ariely and Gino ever did gets removed from the scientific record, It’s a Wonderful Life-style. (We are, I can’t stress this enough, imagining this. Buzz buzz, I’m bees.) What would change?
Not much. Let’s start with Ariely. He’s famous for his work on irrationality, which you could charitably summarize as “humans deviate from the rules of rationality in predictable ways,” or you could uncharitably summarize as “humans r pretty dumb lol.” He’s a great popularizer of this research because he has a knack for doing meme-able studies, like one where, uh, men reported their sexual preferences while jerking off. But psychologists have been producing studies where humans deviate from the rules of rationality for 50 years. We’ve piled up hundreds of heuristics, biases, illusions, effects, and paradoxes, and if you scooped out Ariely’s portion of the pile, it would still be a giant pile. A world without him is scientifically a very similar world to the one we have now.
Same goes for Gino. Much of her work is also part of the big pile of cognitive biases, and, just like Ariely, that pile would be huge with or without her. For the rest, you can judge for yourself the four studies that were recently retracted:
-
Participants said they wanted cleaning products more after they were forced to argue against something they believed (vs. arguing for the thing they believed).
-
Participants either wrote about 1) a duty or obligation, 2) a hope or aspiration, or 3) their usual evening activities. Then they imagined networking at a corporate event. The people who wrote about the duty or obligation said they felt more “dirty, tainted, inauthentic, ashamed, wrong, unnatural, impure” while imagining the networking event than people who wrote about their hopes/aspirations or their evening activities.
-
Participants who were given the opportunity to lie about the outcome of a coin toss (they could get more money if they lied), and who did indeed lie, later came up with more uses for a newspaper in 1 minute.
-
Participants completed as many math problems as they could in 1 minute, and they could lie about how many they got right (they could get more money if they lied). Then they filled out a form where they reported how much time and money they spent coming to the lab, for which they were compensated up to a certain amount (here they could also get more money if they lied). Some participants signed at the top of the form, and some signed at the bottom. The participants who signed at the bottom lied more than the participants who signed at the top.
(I’m describing these studies in experimental history terms—as in, people doing things. The authors described these results as “inauthenticity causes feelings of impurity” and “dishonesty leads to creativity” and “signing makes ethics salient.” See what a difference it makes to talk about people and the things they did!)
Looking over the rest of Gino’s papers, these studies seem like pretty standard examples of her research. I’ll only speak for myself here: if I found out that every single one of these studies had been nothing more than Gino running create_fake_data.exe on her computer over and over again, I wouldn’t believe anything different about the human mind than I already believe now.
This isn’t specific to Gino and Ariely; I think you could It’s-a-Wonderful-Life most psychologists, even the famous ones, without any major changes to what we know. This was also true the last time we discovered a prolific fraudster. Diederik Stapel, a Dutch social psychologist, faked at least 58 papers. I mean really faked: the guy eventually admitted he would open up a blank spreadsheet and start typing numbers. Unlike Gino and Ariely, there’s no ambiguity here—Stapel’s entire scientific career got wiped out.
So what was the scientific fallout of Stapel’s demise? What theories had to be rewritten? What revisions did we have to make to our understanding of the human mind?
Basically none, as far as I can tell. The universities where Stapel worked released a long report cataloging all of his misdeeds, and the part called “Impact of the fraud” (section 3.7 if you’re following along at home) details all sorts of reputational harm: students, schools, co-authors, journals, and even psychology itself all suffer from their association with Stapel. It says nothing about the scientific impact—the theories that have to be rolled back, the models that have to be retired, the subfields that are at square one again. And looking over Stapel’s retracted work, it’s because there are no theories, models, or subfields that changed much at all. The 10,000+ citations of his work now point nowhere, and it makes no difference.
As a young psychologist, this chills me to my bones. Apparently is possible to reach the stratosphere of scientific achievement, to publish