They're all wrong! Most published research findings may be false
"There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims," says researcher John Ioannidis in an analysis in the open access international medical journal PLoS Medicine.
In his analysis, Ioannidis, of the University of Ioannina School of Medicine, Greece, and Tufts University School of Medicine, United States, identifies the factors that he believes lead to research findings often being false.
One of these factors is that many research studies are small. "The smaller the studies conducted in a scientific field, the less likely the research findings are to be true," says Ioannidis.
Original paper here. Editorial here:
Hypotheses will inevitably change, as they depend not only on the study but also on the context of other relevant research and knowledge. Conclusions are also often based on current knowledge and assumptions, and, thus, subject to change. The data should be more robust; for instance, other researchers applying the same methods to study the same group of patients at the same time should be able to generate the same data. However, research progress depends on conclusions being tested elsewhere. The major issue about the truth of research findings would therefore seem to concern the conclusions, and Ioannidis's claim that most conclusions are false is probably correct. Is that a problem? Can it be avoided?
Some commentary. We are also involved in a certain amount of biomedical research and what is said is true: In most studies involving intervention or observational studies associating risk factors with outcomes, the ultimate conclusion almost invariably involves p-values. This really is as it should be, but what this paper points out is something most researchers already know, in part, implicitly: these are hypotheses and they are therefore tentative. And most biomed journal articles include (sometimes required) sections describing limitations. And, now that we've identified most of the major risk factors for diseases (though not always by additionally establishing a mechanistic cause-effect relationship), we're left with a lot of possible risk factors that are very, very small and in these cases sample size and other complicating factors assume a relatively large importance.
Sadly, the media do little to put most of this research in perspective. We're often treated to scary-important headlines of terrible new risks from some previously unknown pathogen or toxin that may double our risk of cancer! Of course, we're not often told that said risk went from .0001 to .0002. And risk is another subject altogether.
So maybe we shouldn't get too worked up about the fact that we still haven't quite figured out why lots of megafauna went extinct at the end of the Pleistocene.
Okay, well now we know why: Paleolithic McMansions Oldest homes were made of mammoth bone
CAMBRIDGE UNIVERSITY archaeologists are joining Czech colleagues to study the world’s oldest-known houses, some of them built from mammoth bones, Norman Hammond writes. The houses lie beneath the vineyards of southern Moravia, along with early evidence of such craft activities as modelling in clay and weaving, all carried out around a central hearth.
20th century threatens a medieval Muslim jewel
Archaeologists fight to save lost city of gold from home builders
To hear historians tell it, this buried city three miles west of Cordoba was the Versailles of the Middle Ages, a collection of estates and palaces teeming with treasures that dazzled the most jaded traveler.
"Travelers from distant lands, men of all ranks and professions in life, following various religions, princes, ambassadors, merchants, pilgrims, theologians, and poets all agreed that they had never seen in the course of their travels anything that could be compared to it," wrote the 19th-century historian Stanley Lane-Poole in his book The Story of the Moors in Spain .
Archaeologists are more hesitant, saying that while many of those marvels may have existed, physical evidence has yet to be found.