More rigorous safeguards can protect against errors similar to the ones caused by disgraced former Duke oncologist Dr. Anil Potti, according to recommendations by a nonprofit health advisory group.
A committee of the Institute of Medicine of the National Academies released a report Friday to establish standards for genomics research and other developing medical sciences based on large quantities of data. The report was partly a result of the Potti scandal, and it suggests a framework for monitoring and ensuring research integrity in translational medicine.
“We’re talking about research on the frontier…. If we’d had this document in place and the Duke institution had followed these procedures, we probably wouldn’t have had the problems that happened,” said David DeMets, a member of the IOM committee that released the report and professor of biostatistics and medical informatics at the University of Wisconsin-Madison. “But of course [the procedures] weren’t in place, and people were learning as they were going.”
Because the field of genomics has only developed in the last decade—since the mapping of the human genome—there was no field-wide set of research guidelines in genomics when Potti’s work began, DeMets said.
It is at the discretion of investigators and research institutions to adopt the IOM framework to reduce the risk of clinical treatments based on flawed genomic science.
“The IOM report is the first comprehensive roadmap, reframed for the genomics era,” DeMets said. “All these steps are individually not new steps, but putting them together in a comprehensive way is the fundamental contribution of this report.”
The report, titled “Evolution of Translational Omics: Lessons Learned and the Path Forward,” suggests changes to the review process for genomics research and the computational processes necessary to make sense of massive genomics datasets. Computational procedures should be locked down—recorded and not changed—and confirmed with new set of samples different from the initial discovery set, according to the report.
Potti’s research passed through peer review and internal review before going to clinical trial. When an investigation into Potti’s credentials revealed issues of concern for the University Fall 2010, new scrutiny of his published work revealed central problems.
Duke Medicine enacted changes similar to the IOM suggestions in its Translational Medicine Quality Framework, which was finalized last May and is being implemented. The framework suggests a digital lockbox to track changes to genomics data sets, an integrity hotline for reporting suspicious research activity and clearer methods for managing conflicts of interest, among other measures.
Dr. Robert Califf, vice chancellor for clinical research and director of the Duke Translational Medicine Institute, said Duke has taken efforts to correct problems that the Potti case demonstrated in the genomics review process. He hopes leaders of other genomics research institutions examine the IOM report and apply the findings.
The IOM findings acknowledge, in detail, mistakes made at Duke, but the emphasis now is on prevention, Califf noted.
“We have residual mop-up issues, but I think we’re about done [with Potti retractions],” he said. “It’s about how to do things right going forward.”
The IOM report’s call for publication of full data and the code used to guide patient therapy appealed to Keith Baggerly, a biostatistician at the University of Texas MD Anderson Cancer Center. Baggerly and his colleague Kevin Coombes noticed inconsistencies in Potti’s paper—including mixed-up labels and discussion of genes that were not reproducible with Potti’s software and data. Their critique prompted the National Cancer Institute to investigate Potti’s clinical trials in 2009.
At the request of the National Cancer Institute, the IOM decided in October 2010 to investigate translational research procedures and specific clinical trials based on Potti’s work. “Had such data and code been available earlier, this would have greatly reduced the amount of effort required for others—including us—to check and potentially extend on the underlying results,” Baggerly wrote in an email Wednesday.
Additionally, the IOM test will require approval by the Food and Drug Administration for any omics-based tests that will lead to clinical applications. FDA review may have caught several of the problems at Duke, Baggerly noted, particularly if the agency had been alerted of analytical problems with the research.
The increased scrutiny after the Potti revelations should not lead to overly cautious research, Califf added.
“Hopefully it will trigger an intense desire to do things right. No one can assure that your theory is going to work. You have to do studies to find out,” he said. “I wouldn’t want people to refrain from moving forward if that was the right thing to do—we just need a procedure so that right answers can be obtained without bias.”
Get The Chronicle straight to your inbox
Signup for our weekly newsletter. Cancel at any time.