The Fraud Detection Pipeline
In the recent data fabrication stories, the accusations are largely coming from bloggers and independent researchers
We’re going to start this with a follow-up from the previous piece on data fabrication, which ended with the question of why data fabrication would even make sense in these circumstances. There were a few comments that I wanted to highlight.
The first was from Kate, who suggested that the fabrication might not be about faking results but might be a short-cut to implying that the researchers performed the proper tests and got the expected results:
I work in a research lab. I know the vast temptation to do this sort of stuff, have been obliquely told to do this sort of stuff and managed to resist. Here's how it happens. These are big labs, dozens of postdocs and grad students. They all need a certain number of publications to get out. The PI has very little idea what it happening in the lab. Now reviews come back for the paper that you need to publish before you defend and they are asking for a bunch of superfluous experiments. You try to get them to work and they kinda do but not cleanly and the PI says they have to be perfect. You are staring don the barrel of a year passing just since you first submitted this paper, not to mention the three years before that of work, most of which were 60-80 hr weeks at less than minimum wage and now you just need this one band on a Western blot to look like the band in that blot and this figure is going to be buried in supplemental and no one will ever care...
The second is from
who notes that pressure to get the proper results along with pressure to publish results that will stick out and garner attention may lead to a temptation to fudge the edges of the results:Two experiences -- one from my undergrad as a student in a biology lab course (twenty years ago -- yikes), and one from the last couple of years as I was working on an abstract for a conference paper as a graduate student.
In the undergraduate lab, we were doing an experiment to practice running Western Blots -- something with fertilized sea urchin eggs at different time points post-fertilization. Perhaps reasonably or perhaps not, the graders for the class had a standard for what our Western Blots "should" look like if we did the experiment correctly. If our Western Blots didn't match theirs, then we got points deducted, even if we understood what we'd done wrong and explained it. At the time I was outraged because I felt it would lead to falsification of data. Now I'm a little more mellow about it -- there's a lot of ways to screw up a Western Blot and they did have to grade us on our technique somehow. But even so, it does lead to the idea that you need to get the "right" results from the experiment or be punished for it.
Much more recently, in the case of the abstract, while I found myself not falsifying data or saying anything untrue, I did feel pressure to sensationalize my results more than I'd like -- because otherwise, no one was going to be interested in selecting my paper for the conference. And that tendency feels ubiquitous in my program, because it's just what you need to do to get attention. I like to think that we're all ethically above outright falsifying anything, but the pressure to state things with more enthusiasm than they warrant is there and very easy to give into -- so why not also, maybe sometimes, making things up just a little?
It's a bad environment all along the teaching pipeline.
These explanations help because they give some insight not only into why these data fabrication stories are increasing but also because they give us a vision into how this might come about.
Keep reading with a 7-day free trial
Subscribe to Matt Shapiro's Marginally Compelling to keep reading this post and get 7 days of free access to the full post archives.