The importance of ASCO and oncology research
Imagine you’re an oncology physician. You decide to start treating one of your patients with a new cancer therapy based on promising early-stage clinical trial data. But while you are cautiously optimistic, you are also concerned about the reliability of the early findings and start to question your decision. What if these results differ from the final published outcomes, you wonder. Many oncology providers – as well as pharmaceutical companies – share this worry.
At ASCO 2024, IMO Health researchers will present a poster demonstrating how large language models (LLMs) can help remedy this problem, examining the consistency of oncology trial reporting and facilitating clinical decision-making.
ASCO, or the American Society of Clinical Oncology, holds a conference every year where oncology professionals from all over the world gather to explore cutting-edge research, innovative treatments, and shrewd insights – it is one of the most prestigious conferences in the oncology field. This year’s meeting, held in Chicago from May 31 to June 4, will feature over 200 sessions complementing the theme The Art and Science of Cancer Care: From Comfort to Cure.
“Less than 30% of abstracts submitted this year were selected for poster presentations, so we were very happy to be part of this group,” Kyeryoung Lee, PhD, IMO Health’s Principal Scientist and one of the project’s leads, said. Hunki Paek, PhD, IMO Health’s BioMed Real-World Evidence Scientist, worked alongside Lee as the other project lead.
According to Lee, “Oncology is one of the biggest clinical trial fields, and big pharma mainly focuses on oncology clinical trials.” To her point, there were over 22,000 clinical trials in 2023, most of which indicated oncology.1
Study objectives and methodology
Lee said the idea for the study – titled Unveiling consistency: A large-scale analysis of conference proceedings and subsequent publications in oncology clinical trials using large language models – originated at last year’s ASCO conference, where a big pharma company expressed interest in IMO Health’s NLP technology.
“The company was interested in how these techniques can actually solve their issues,” she said – issues meaning extracting clinical trial treatment outcome information from free text formats, or unstructured data, including conference abstracts and published articles, and transforming it into structured, usable content.
So, Lee’s team established a GPT-4 based LLM pipeline to do just that. But the work didn’t stop there.
The team also developed a process to analyze the consistency between initial conference results and subsequent reporting in published articles, focusing on treatment efficacy and safety.
For example, let’s say an oncology clinical trial recruits 20 patients in the early stages and 50% of them respond, with 10% reporting adverse events. Will those results remain consistent when 100 patients are involved in the later stages? That’s one of the overarching questions IMO Health sought to answer with this study.
“If the result demonstrates that early-stage trial results are consistent with final published results, doctors can use this evidence for immediate decision-making,” Lee emphasized.
To execute this, the team collected clinical trial abstracts from 2012 to 2023 from ASCO/ASH conferences and PubMed, comprising both solid and hematopoietic cancer treatments. They then conducted a statistical test called a two-proportional Z-test to assess the consistency between reported outcome values in earlier conference abstracts and final published articles, factoring in cohort size and other variables.
Key findings and impact on healthcare industries
The team’s LLM pipeline demonstrates high precision and recall in analyzing clinical trial data across diverse cancer types, underscoring the transformative potential of AI in enhancing clinical research and, ultimately, decision-making. The findings reveal that while dose-escalation phases might show variability, the key treatment outcomes at recommended doses remain consistent over time.
Per Lee and her team, this study offers providers increased assurance in the reliability of early-stage clinical trial results presented at conferences, which are crucial for making informed treatment decisions in oncology.
For health tech companies, the study showcases the potential of LLMs to efficiently analyze and extract meaningful data from extensive clinical trial records, highlighting new possibilities for software development in clinical data management and analysis.
Lastly, life sciences companies can benefit from the insights on the consistency of trial data reporting, which can inform better strategies for trial design, regulatory engagement, and effective dissemination of clinical findings, enhancing credibility and operational efficiency in bringing new therapies to market.
“This study is beneficial to every community member, providing valuable insights and helping improve clinical trial protocols and decision-making processes,” Lee said.
Can’t make it to this year’s ASCO conference? No worries. IMO Health attends, exhibits, and demos at industry events throughout the year. Click here to learn where you can find us next and here to dive into more of our groundbreaking research.
1 Beaney A. Most significant clinical trials of 2023. Clinical Trials Arena. 27 Dec 2023. Accessed via: Most significant clinical trials of 2023 (clinicaltrialsarena.com)
