ChatGPT was released during my first semester of medical school, and it immediately transformed the way that I learned. Suddenly, I was able to use AI to break down complex physiology, generate clinical vignettes, and walk through diagnostic reasoning step by step. As I progressed, I soon realized the same technologies helping me to study for exams were quickly being applied to patient care, helping with clinical decisions and treatment planning.

Rather than simply using these tools, I became increasingly curious about how they worked and what their limitations might be in clinical settings. With an interest in oncology, I was especially drawn to questions about whether AI could meaningfully help address some of the uncertainty that surrounds cancer care. This interest led me to pursue a dedicated research year at Yale, working with Dr. Sanjay Aneja. Our lab leverages deep learning to develop image-based biomarkers for cancer patients, and my work in particular focuses on improving prognostication for lung cancer patients with brain metastases.

The problem

For patients with lung cancer, the development of brain metastases is often a clinical and emotional turning point. Occurring in up to forty percent of this population, such intracranial tumors can cause neurologic symptoms, cognitive decline, and highly variable survival.¹ Some patients live for years with aggressive therapy, while others decline rapidly despite similar treatment, making high-stakes decisions, such as surgery, radiation, or systemic therapy, especially difficult. Indeed, brain metastases vary widely in size, location, growth pattern, vascularity, and histology, even among patients with the same primary tumor subtype. Traditional prognostic tools, based on clinical factors, capture population-level trends, but they are less informed by the underlying biology driving outcomes in individual patients. Rich quantitative information embedded in radiologic scans, tissue slides, and longitudinal clinical data remains underutilized, which led us to ask: what if AI could integrate these diverse datasets to produce patient-specific predictions that meaningfully inform clinical decision-making?

AI and Multimodal Modeling

A Published Dataset

To explore the potential of AI in brain metastases, our team partnered with Dr. Don Nguyen and Dr. Darin Dolezal in the Department of Pathology at Yale to assemble a dataset designed to capture the complexity of lung cancer brain metastases. A key strength of our dataset is its multimodal design, which integrates multiple types of information from the same patients, including imaging, pathology, and clinical data. Each of these modalities provides a different window into the biology of the tumor, and integrating them allows a more complete, patient-specific view than any single type of data could provide. The fully annotated dataset is publicly available through The Cancer Imaging Archive at the National Institutes of Health, offering a resource for reproducibility and collaboration.²

The dataset links three core components:

• Pre-operative brain MRI, including multiple imaging sequences.
• Matched whole-slide histopathology images from brain metastasis tissue.
• Detailed clinical data, including patient demographics, tumor characteristics, key molecular markers, and established prognostic scores.

By aligning these modalities at the patient level, we sought to create a more faithful representation of the disease. For radiologic imaging, brain metastases were segmented to delineate tumor boundaries, and radiomic features were extracted to quantify tumor shape, intensity, and texture, properties that may reflect growth patterns, vascularity, and surrounding edema. Digitized whole-slide histopathology images were also included, providing the foundation for AI tools that could analyze subtle histologic patterns beyond traditional descriptors such as grade or morphology.

Making the dataset publicly available supports external validation, methodological comparison, and broader participation, particularly for trainees and early-career researchers who may lack the resources to assemble large multimodal datasets independently. In a rapidly evolving field like AI in oncology, shared benchmarks are essential for meaningful progress, and the integration of imaging, pathology, and clinical data makes this dataset uniquely positioned to advance patient-specific modeling.