Volume 5, Issue 34 January 2008
Truth in Data
Professor of Biostatistics and Statistics Mark van der Laan. Photo credit: courtesy Mark van der Laan
Scientific findings influence everything from our eating habits-are bananas healthier than blueberries?-to cancer treatments that can spell the difference between life and death. But when it comes to scientific studies, results can be deceiving. According to a 2005 literature review, published research claims are more likely to be false than true.
The problem, says Mark van der Laan, lies neither with science nor data. A Berkeley professor of biostatistics and statistics, van der Laan develops data analysis methods that promise to make studies more accurate and reliable.
One major aspect of his research involves clinical trials. Often, these trials enroll hundreds or thousands of subjects, may last for many years, and cost hundreds of millions of dollars to administer. But, on average, roughly one in four subjects drops out before a study's official endpoint.
What to do with this incomplete, or censored, data constitutes a major dilemma. At present, most studies simply discard censored data, but van der Laan says ignoring this information is not only wasteful, it also can introduce a dangerous bias to study results.
Roughly a quarter of all patients tend to drop out of clinical trials evaluating drugs and other medical treatments. Photo credit: David Richfield
In drug trials, for example, dropouts often develop side effects or fail to respond to the treatment. "Maybe the people you're tossing out are very different kinds of people than the rest of the trial participants. So the sample you've used isn't representative of the group you were originally looking at," says van der Laan. "By ignoring them, you're tossing away all of the bad outcomes. Your treatment will look better than it really is."
He is developing statistical methods that allow censored data to be incorporated in study results. His methods first try to understand why subjects drop out. Medical files frequently indicate why someone switched to a different treatment. Once the reasons behind a patient's noncompliance are understood, it's possible to account for the bias incurred by omitting this data. Similarly, van der Laan's methods can help predict the outcomes for these patients if they had participated in the trial to its end.
Currently, says van der Laan, censored data in clinical trials is handled with naïve and often biased methods. To help change that, he has begun collaborating with the Food and Drug Administration to demonstrate that techniques to incorporate censored data are robust, efficient, and reliable enough for medical studies.
He is also studying how to estimate the proportionate effect of a particular intervention. He uses a technique called causal inference to address such questions as the effect of smoking on lung disease or the impact of hormone replacement therapy on heart disease.
One of van der Laan's research areas is causal inference, an area of statistics that resolve questions such as whether cigarette smoking causes lung disease. Photo credit: Tomasz Sienicki
The problem is that data often come from subjects who have elected a certain treatment, and they might not be a statistically representative pool. "Perhaps the women who took hormone replacement therapy were wealthier, or more educated, or more compliant than other women," says van der Laan. "Maybe we're seeing the effect of these confounding factors rather than the effects of the intervention."
Often, van der Laan says, disentangling such factors is impossible. But many scientists will continue to massage their data until they find a correlation to ensure the study appears in a journal. He hopes to overcome this "pressure to publish" by convincing journals to publish negative results and require scientists to declare their analysis methods before a study begins.
"It's why I often say statistics is a brutal field if you do it right," says van der Laan. As rough as getting to the truth might be, the result should be more informed decisions and better health for all.