The results that did not survive the second look

Science works because results are supposed to be checked. In practice, for most of the last century, the checking happened informally and rarely: a finding was published, built upon, cited, and almost never run again from scratch. That changed in 2005 with an essay by John Ioannidis titled Why most published research findings are false, which argued, with probability theory rather than accusations, that the way studies were designed, selected and published guaranteed that many of them would not hold up. The essay was controversial. The decade that followed was spent testing whether it was right.

The numbers, study by study

The largest single test was a project that took 100 published psychology experiments and repeated them as faithfully as possible. Where the original studies had reported a statistically significant result in 97 percent of cases, the replications found one in 36 percent. The effect sizes that survived shrank, on average, to about half of the original. Industry checked itself as well: a Bayer team that tried to reproduce 67 landmark cancer biology studies reported success with roughly a quarter, and an Amgen effort using 53 papers did worse still. The exact percentages are not the point. The pattern is.

None of this demonstrated fraud. It demonstrated arithmetic. A study with a small sample and a slightly lucky result gets published while the boring null result stays in a drawer. When the same question is asked again with no stake in the answer, the luck averages out. Statisticians call part of this the winner curse, and it is not a scandal. It is what happens when the gatekeepers reward novelty and treat replication as career suicide.

What the failures do and do not mean

A failed replication is not a verdict of false. The second study may have run in a different population, measured the outcome slightly differently, or simply been too small itself. A null result can mean the effect is absent, or smaller than everyone assumed, or that the replication was done poorly. What the replication projects established is that the headline numbers in the literature were, in aggregate, too good to be true, and that the inflation was measurable.

What actually helps

The most promising fixes are procedural rather than moral. Preregistration obliges researchers to state their hypothesis and analysis before seeing the data, which prevents the quiet reshaping of a study around whatever comes out. Registered reports go further: a journal reviews the design first and commits to publishing the result whatever it shows, which removes the incentive to bury negative findings. Large collaborations pool many samples under one protocol so a single experiment can no longer be the whole evidence base. Each of these has grown steadily, and the early evidence is that they reduce inflated effects.

The constraint is the one this section exists to cover: money. Replication wins no grants, fills no keynote slots and wins no prizes, so it is funded at the margins by foundations and consortia even as it becomes more necessary. The same structure that decides which questions get asked also decides whether anyone is paid to check the answers.

Why this matters outside academia

The consequences do not stay inside the literature. Drug programmes are built on published targets, and a target that only works in one laboratory is a decade of clinical development spent testing a ghost. When a field corrects itself, the correction is not abstract: it is trials not run, patients not exposed, and budgets not burned. This is also why the replication problem is a story about capital. Checking the work of others has no product at the end of it, so the market will not fund it, and the institutions that could have treated it as part of the cost of doing science mostly did not.

The honest edge

Replication rates vary enormously by field, and the psychology figure of 36 percent should not be transplanted onto chemistry or physics. Some celebrated results were rerun and confirmed. And the reproducibility movement itself faces fair criticism about the cost of rerunning work, and about what a single replication can ever prove. What the last decade did establish, with data rather than essays, is that the scientific record is more fragile than its confidence suggested, and that checking it is a public good no market will supply on its own.