Balancing quantity and quality of data in oncology research

By Karla Feghali, senior manager, head of ConvergeHEALTH Services, Deloitte Consulting LLP

Real-world data (RWD) and real-world evidence (RWE) can be important when demonstrating the quality and safety of a medical product. RWD is pulled from sources other than traditional clinical trials and typically reflects routine clinical practices. Data sources can include electronic health records (EHRs), claims data, disease registries, and patient-reported outcomes. RWE is a product of the analyzed RWD and is usually included in regulatory submissions to demonstrate product safety and effectiveness and to offer therapeutic context (e.g., contextualize the natural history of a disease). About 90% of new drug approvals in the United States included RWE as part of the submission (see Real-world evidence use accelerates).

Pharmaceutical companies and medical device manufacturers usually purchase RWD from third-party vendors and might also buy the analytics needed to generate RWE. Companies that purchase data often need to strike a balance between volume and quality to help ensure the most effective insights and the best return on investment. While EHRs have generally revolutionized the way providers capture data, they are not always accurate or complete. And decisions based on inaccurate or incomplete data could result in flawed insights. But focusing too much on the quality of data could limit the breadth of analysis.

COTA, Inc.—a technology company founded by oncologists, data scientists, and engineers more than a decade ago—curates and analyzes clinical data. Along with extracting information for EHRs, the company culls information from public sources including obituaries and the Social Security Administration to generate longitudinal and de-identified datasets. I recently had an opportunity to speak with Sandy Leonard, the chief commercial officer at COTA, about the increasingly important role RWD plays in the development of therapies to treat cancer. Prior to joining COTA last year, she spent 17 years working in the pharmaceutical sector. Here is an excerpt from that conversation:

Karla: Why do you think it is important that pharmaceutical companies have a balanced strategy for RWD?

Sandy: There are several factors that go into a data strategy. Companies should consider both their immediate needs and their pipeline needs. Is data needed for a single product or is it for a deep portfolio of interventions? The data strategy also needs to consider the budget to make sure that money is being spent in a way that meets those needs.

Karla: Some pharma companies have much larger budgets than other companies. Does a company’s budget tend to drive their ability to acquire the most appropriate data?

Sandy: Companies with smaller budgets can usually meet very targeted needs and use-cases. If a company is using data to answer a particular question, a small budget may meet that need. But a broad investment in data often leads to more exploratory and innovative work. This can be used for secondary analysis and hypothesis generation. The data budget can help drive innovation. That’s not to say that a targeted budget isn't going to lead to innovations, but targeted data will have limited applications.

Karla: Pharmaceutical companies sometimes need to choose between volume and quality when purchasing RWE. What are some of the challenges in striking the right balance?

Sandy: The quantity-versus-quality question has gone through an interesting evolution. When RWE was emerging, companies were focused on accessing as much data as they could. The idea was that if you had the entire haystack, you were bound to find the couple of needles that were important for your study. For years, data buyers and researchers were focused almost entirely on quantity. I think the industry has evolved significantly as companies have gained more exposure to different types of data and now have technologies that can help improve the quality of the data. A company no longer needs to start with a large quantity in order to get to quality. Targeted data from the right 300 patients might be more useful than a cohort of 10 million patients.

Karla: Do you think there are concerns that smaller RWE volume might be seen as not having statistical significance when it comes to submissions?

Sandy: In my experience, the [Food & Drug Administration] FDA provides a lot of guidance and opportunities to have conversations on this topic. Whether the data is being used for external control arms or in safety contextualization studies, anytime you are using RWD, it should be part of your early conversations with FDA. High-quality data can override massive quantities that might be incomplete. The other piece is the contemporaneous aspect. The data being used has to be relevant to how care is being delivered. I find that the FDA also wants sponsors to be able to get back to the source data.

Karla: How difficult is it to determine if a data set is inaccurate or incomplete?

Sandy: Inaccurate and incomplete are two different considerations. We address the accuracy of our data through our quality management system (QMS) processes to ensure adequate abstractor training, correctness of data entry via our abstraction platform, technology audits, and data reviews. Incompleteness needs to be addressed in a more nuanced way given that no dataset can be 100% complete. What matters most are the timepoints and data elements of interest. That can be a challenge when working with clinical data, but a little less so within oncology because everything tends to be concentrated in a cancer center. The sharing of data into the ER tends to be very good. For example, we can access the notes from a community oncologist and the EHR data from an academic medical center if a patient is transferred there. That allows us to see continuity of care. We might have to go back to the EHR to determine if a gap in care means there was a pause in activity or if information is missing. We refer to that as the period of observation. Researchers might see that a cancer diagnosis was made in February, but nothing happened for the next three years. Does that mean that data is missing or is the cancer at a stage where the physician or patient chose not to actively treat it.

Karla: Can you share an instance where the depth of the data purchased was not enough to answer the critical questions being asked?

Sandy: There are always going to be situations where the depth of the data doesn't matter if the right patients aren’t included. That can be seen with rare cancers, with nuanced biomarkers, and with inclusion/exclusions where there aren’t enough patients to build out a usable cohort. That’s where you have to get creative and take multiple approaches. Stacking different clinical data assets together might be a solution. You might have to supplement the broader research question with additional RWD.

Karla: How do data choices impact the decision-making process and long-term strategic goals?

Sandy: A pharmaceutical company might purchase data to answer a question, but that one question can instigate additional questions. And that's the point around having access to data. It can be used to answer a specific question and help to inform and drive an organization’s evidence strategy. The value of that data starts to expand the more it gets used and can then help determine what types of additional data might be needed. Maybe researchers decide they want a financial view and look to claims data.

Conclusion

The landscape of research is continually evolving, and a well-crafted data strategy can play a pivotal role in navigating this terrain. Striking the appropriate balance between data volume and quality is important for insightful analysis and innovative breakthroughs. But this balance isn't solely about quantity vs. quality; it is about understanding the unique nuances of research needs, the capacity of the budget, and the potential of the available data.

The power of data, particularly in oncology research, tends to lie not just in its volume, but in its relevance, quality, and how effectively it is used to answer critical questions. The future of oncology research is ripe with possibilities. Companies should continue to make informed decisions, and work toward advancing cancer treatments and improving patient outcomes.

Latest news from @DeloitteHealth

The executive’s participation in this article is solely for educational purposes based on their knowledge of the subject and the views expressed by them are solely their own. This article should not be deemed or construed to be for the purpose of soliciting business for any of the companies mentioned, nor does Deloitte advocate or endorse the services or products provided by these companies.

This publication contains general information only and Deloitte is not, by means of this publication, rendering accounting, business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor.

Deloitte shall not be responsible for any loss sustained by any person who relies on this publication.

Return to the Health Forward home page to discover more insights from our leaders.