Multimodal Artificial Intelligence in Oncology: Integrating Radiomics, Pathomics, and Genomics

Cancer remains one of the leading causes of morbidity and mortality worldwide despite major advances in molecular biology, targeted therapeutics, and immuno-oncology. The biological complexity of malignant diseases arises from dynamic interactions among genomic instability, epigenetic modifications, microenvironmental remodeling, and spatiotemporal tumor heterogeneity that collectively influence disease progression and therapeutic response.[1,2] Conventional clinical decision-making in oncology has historically relied upon fragmented interpretation of imaging findings, histopathological examination, and limited molecular biomarkers. Although these approaches have significantly improved patient management, they often fail to comprehensively capture the multidimensional architecture of tumor biology.[3] The rapid digitization of healthcare systems and high-throughput biomedical technologies has generated unprecedented quantities of multimodal cancer data, creating new opportunities for AI-driven computational oncology.

The integration of AI into oncology has accelerated substantially over the past decade because of parallel advancements in computational power, data storage infrastructure, and deep learning algorithms.[4] Modern oncology datasets now encompass radiological imaging, whole-slide pathology images, next-generation sequencing, transcriptomics, proteomics, metabolomics, wearable sensor outputs, and longitudinal electronic health records.[5] Individually, these modalities provide partial representations of tumor phenotypes; however, multimodal AI aims to unify them into coherent predictive frameworks capable of uncovering latent biological relationships inaccessible to human interpretation.[6] This paradigm shift has stimulated growing interest in multimodal foundation models that leverage transformer architectures, self-supervised learning, and attention mechanisms to learn generalized representations across heterogeneous biomedical domains.[7]

Radiomics, pathomics, and genomics constitute the foundational pillars of multimodal computational oncology. Radiomics enables quantitative extraction of imaging biomarkers from computed tomography, magnetic resonance imaging, and positron emission tomography, thereby characterizing intratumoral heterogeneity and treatment-associated phenotypes.[8] Pathomics extends computational pathology by applying machine learning to digitized histopathological slides for automated tissue classification, spatial cellular analysis, and microenvironmental characterization.[9] Genomics and multi-omics integration further provide molecular-level insights into oncogenic signaling pathways, mutational landscapes, immune regulation, and therapeutic resistance mechanisms.[10] The synergistic integration of these modalities has demonstrated significant promise in improving diagnostic accuracy, prognostic modeling, and therapeutic stratification across multiple malignancies.

Despite impressive technological progress, the implementation of multimodal AI in routine oncology practice remains constrained by substantial methodological and translational challenges.[11] Data heterogeneity, annotation variability, limited interoperability, domain shift, and inadequate external validation continue to limit generalizability across institutions and populations.[12] Moreover, the opaque nature of deep neural networks raises concerns regarding interpretability, fairness, accountability, and regulatory acceptance.[13] Consequently, there is increasing emphasis on explainable AI, federated learning, privacy-preserving computation, and prospective clinical validation frameworks to ensure safe and equitable deployment of AI systems within healthcare environments.[14]

This review comprehensively examines the scientific evolution, computational foundations, clinical applications, and translational implications of multimodal AI in oncology. Particular emphasis is placed on radiomics, pathomics, genomics integration, transformer-based architectures, multimodal fusion strategies, explainable AI, ethical considerations, and the future development of foundation models for precision cancer medicine.

2. Evolution of Artificial Intelligence in Oncology

The evolution of AI in oncology has progressed through several interconnected computational eras characterized by increasing algorithmic complexity and biological integration. Early oncology AI systems primarily relied on rule-based expert systems and conventional machine learning algorithms designed to support clinical diagnosis and survival prediction.[15] These methods utilized handcrafted features derived from radiological images or clinicopathological variables and employed classifiers such as support vector machines, random forests, and logistic regression. Although these approaches demonstrated moderate predictive capability, their performance was constrained by limited feature representation and poor scalability across heterogeneous datasets.[16]

The emergence of deep learning transformed computational oncology by enabling hierarchical feature extraction directly from raw biomedical data. Convolutional neural networks (CNNs) rapidly became dominant in radiology and pathology because of their ability to automatically identify spatially complex image patterns associated with tumor morphology, vascularity, necrosis, and stromal architecture.[17] Landmark studies demonstrated that CNN-based systems could achieve diagnostic performance comparable to expert radiologists and pathologists in breast cancer, lung cancer, and dermatologic malignancies.[18,19] Simultaneously, advances in genomic sequencing technologies enabled AI-driven analysis of mutational signatures, transcriptomic profiles, and pathway interactions associated with oncogenesis and therapeutic resistance.[20]

Recent years have witnessed a transition from isolated modality-specific models toward integrated multimodal frameworks capable of simultaneously processing imaging, molecular, and clinical data.[21] This transition has been accelerated by transformer architectures originally developed for natural language processing. Vision transformers and multimodal transformers utilize self-attention mechanisms to model long-range dependencies across heterogeneous biomedical features, thereby improving contextual understanding and predictive performance.[22] Foundation models trained on massive multimodal datasets are increasingly capable of transfer learning across cancer types and clinical tasks, raising the possibility of generalized oncology AI systems.[23]

Concurrently, self-supervised learning has emerged as a critical strategy for addressing the scarcity of annotated medical data. By learning latent feature representations through pretext tasks without extensive manual labeling, self-supervised models have demonstrated robust performance in pathology, radiology, and genomics applications.[24] The integration of large language models into oncology workflows has further enabled automated report generation, literature synthesis, and conversational clinical decision support.[25] These developments collectively signify a paradigm shift from narrow predictive algorithms toward comprehensive multimodal ecosystems capable of supporting precision oncology at scale.

3. Concept of Multimodal Artificial Intelligence

Multimodal AI refers to computational systems designed to integrate heterogeneous data modalities into unified predictive or interpretive frameworks. In oncology, these modalities commonly include radiological imaging, histopathological whole-slide images, genomic sequencing, transcriptomics, proteomics, and clinical metadata.[26] The biological rationale for multimodal integration derives from the intrinsic heterogeneity of tumors, which cannot be fully characterized by any single diagnostic modality. Imaging captures macroscopic anatomical and physiological phenotypes, pathology reveals microscopic architectural organization, while genomic analyses identify molecular alterations driving tumor behavior.[27]

Multimodal AI architectures are generally categorized according to the level of data integration. Early fusion approaches combine raw or low-level features before model training, thereby enabling simultaneous learning across modalities. Intermediate fusion strategies integrate modality-specific embeddings generated through independent subnetworks, whereas late fusion approaches combine independently generated predictions from separate models.[28] Transformer-based architectures have become particularly influential because self-attention mechanisms allow dynamic weighting of modality-specific information while preserving contextual interactions.[29]

The clinical significance of multimodal AI lies in its capacity to model complex cross-scale relationships associated with tumor biology. For example, radiological imaging features may correlate with specific genomic alterations or immune microenvironment states, thereby enabling noninvasive prediction of molecular phenotypes.[30] Similarly, pathomics-derived cellular architecture may reflect transcriptomic signatures associated with metastatic potential or therapeutic response. By integrating these complementary information sources, multimodal AI systems can improve diagnostic precision, prognostic discrimination, and therapeutic selection compared with unimodal approaches.[31]

Another transformative aspect of multimodal AI is its potential to support longitudinal disease monitoring. Cancer progression is inherently dynamic, involving temporal changes in genomic evolution, treatment response, and microenvironmental adaptation. Multimodal frameworks capable of integrating serial imaging, longitudinal molecular data, and clinical outcomes may facilitate adaptive therapeutic strategies and real-time precision medicine.[32] Consequently, multimodal AI is increasingly viewed as a central technological foundation for future learning healthcare systems in oncology.

4. Radiomics in Precision Oncology

Radiomics has emerged as a major component of precision oncology by enabling extraction of high-dimensional quantitative imaging biomarkers from routinely acquired radiological scans.[33] The central premise of radiomics is that medical images contain latent biological information reflecting tumor heterogeneity, angiogenesis, metabolism, and microenvironmental interactions. Advanced computational pipelines extract textural, morphological, statistical, and wavelet-based features from computed tomography, magnetic resonance imaging, and positron emission tomography to generate quantitative tumor phenotypes.[34]

Deep learning has substantially expanded the scope of radiomics by replacing handcrafted feature engineering with automated representation learning. CNN-based radiomic models can identify subtle imaging patterns associated with molecular subtypes, metastatic potential, and treatment response that are often imperceptible to human observers.[35] In lung cancer, radiogenomic studies have demonstrated associations between CT-derived imaging signatures and EGFR, KRAS, and ALK mutations, thereby enabling noninvasive molecular characterization.[36] Similar approaches have shown promise in glioblastoma, breast cancer, and hepatocellular carcinoma for predicting immune infiltration, hypoxia, and survival outcomes.[37]

The integration of radiomics with clinical and molecular datasets has significantly improved prognostic modeling. Multimodal radiogenomic frameworks combining imaging features with transcriptomic profiles have demonstrated enhanced predictive performance for recurrence and therapeutic response across multiple malignancies.[38] Moreover, temporal radiomics derived from serial imaging enables dynamic assessment of treatment response and tumor evolution during chemotherapy, radiotherapy, or immunotherapy.[39]

Despite considerable progress, radiomics faces substantial reproducibility challenges related to imaging acquisition variability, segmentation inconsistency, and feature instability.[40] Differences in scanner protocols, reconstruction algorithms, and preprocessing pipelines can substantially influence extracted features, limiting generalizability across institutions. Standardization initiatives such as the Image Biomarker Standardisation Initiative and federated learning approaches are therefore essential for reliable clinical translation.[41]

5. Pathomics and Digital Pathology

Digital pathology has revolutionized histopathological analysis by enabling high-resolution digitization of whole-slide images amenable to computational interpretation. Pathomics extends this paradigm through quantitative extraction of spatial, morphological, and cellular features using AI algorithms.[42] Histopathology remains the diagnostic gold standard for most cancers; however, conventional microscopic interpretation is inherently subjective and susceptible to interobserver variability. AI-driven pathomics offers opportunities for standardized, reproducible, and high-throughput tissue analysis.

CNN-based architectures have demonstrated remarkable capability in automated tumor detection, grading, and segmentation across diverse malignancies.[43] Deep learning models trained on whole-slide images can identify complex tissue patterns associated with tumor aggressiveness, lymphovascular invasion, and stromal remodeling. More recently, transformer-based pathology models have enabled improved contextual understanding of spatial cellular interactions within the tumor microenvironment.[44]

An important advance in pathomics involves spatial characterization of immune infiltration and tumor-stroma interactions. AI systems can quantify tumor-infiltrating lymphocytes, macrophage populations, and vascular architecture to infer immunological states associated with response to immune checkpoint inhibitors.[45] Integration of pathomics with transcriptomic and proteomic data has further facilitated identification of novel biomarkers associated with immune evasion and therapeutic resistance.[46]

Foundation models pretrained on massive pathology image repositories are increasingly being adapted for downstream oncology tasks through transfer learning.[47] These models enable efficient generalization across tissue types while reducing annotation requirements. Self-supervised learning strategies have proven particularly effective in pathology because manually labeled datasets are often limited and expensive to generate.[48]

Nevertheless, substantial translational barriers remain. Whole-slide images are extremely large and computationally demanding, requiring specialized hardware and efficient tiling strategies.[49] Additionally, staining variability, scanner heterogeneity, and limited interoperability between digital pathology systems complicate model standardization and external validation. Prospective multicenter studies are therefore necessary to establish clinical reliability and regulatory acceptance.

6.Genomics and Multi-Omics Integration

Genomic sequencing technologies have transformed oncology by revealing the molecular architecture of cancer and enabling targeted therapeutic development.[50] AI-driven genomic analysis facilitates interpretation of complex mutational landscapes, transcriptomic signatures, epigenetic modifications, and proteomic interactions associated with tumor initiation and progression. The integration of multi-omics datasets has become increasingly important because isolated genomic alterations rarely capture the full biological complexity of malignant diseases.

Machine learning models are widely used for mutation detection, molecular subtype classification, and pathway analysis. Deep neural networks can identify nonlinear interactions among genomic variables associated with survival outcomes and therapeutic response.[51] In breast cancer, AI-based transcriptomic analysis has improved prediction of endocrine therapy resistance and metastatic recurrence. Similar approaches have been applied in colorectal cancer, glioma, and melanoma to identify clinically relevant molecular signatures.[52]

The integration of genomics with radiomics and pathomics has generated the emerging field of radiopathogenomics. Multimodal frameworks combining imaging, pathology, and molecular data have demonstrated superior performance for tumor classification, risk stratification, and therapeutic prediction compared with single-modality approaches.[53] Importantly, these models may enable noninvasive inference of molecular phenotypes through imaging or pathology surrogates, thereby reducing the need for repeated invasive biopsies.

Single-cell sequencing and spatial transcriptomics have further expanded the dimensionality of oncology datasets by enabling characterization of intratumoral heterogeneity and microenvironmental dynamics at unprecedented resolution.[54] AI-based integration of spatial omics with histopathology and imaging may facilitate identification of spatially resolved biomarkers associated with immune escape, clonal evolution, and metastatic dissemination. Such approaches are particularly relevant in immuno-oncology, where spatial organization of immune cells strongly influences therapeutic efficacy.[55]

7. Transformer Architectures and Deep Learning Models in Oncology

Transformer architectures have emerged as a transformative computational framework in oncology because of their ability to model long-range dependencies across heterogeneous data modalities.[56] Unlike CNNs, which rely primarily on local convolutional operations, transformers employ self-attention mechanisms that dynamically prioritize relationships among input features. This capability is particularly advantageous for multimodal oncology applications involving complex interactions among imaging, pathology, molecular, and clinical variables.

Vision transformers have demonstrated competitive performance in radiological and pathological image analysis. In digital pathology, transformer models capture global tissue context more effectively than conventional CNNs, thereby improving tumor grading and biomarker prediction.[57] Hybrid CNN-transformer architectures further combine local feature extraction with global contextual reasoning, enhancing performance across classification and segmentation tasks.

Multimodal transformers integrate embeddings from diverse biomedical modalities into unified latent representations. Cross-attention mechanisms enable the model to learn biologically meaningful interactions between imaging features and genomic signatures.[58] Recent oncology foundation models trained on multimodal datasets have shown encouraging transferability across cancer types and clinical tasks, suggesting the possibility of generalized precision oncology systems.

Large language models are increasingly being integrated into oncology workflows for clinical documentation, evidence synthesis, and decision support.[59] When combined with multimodal inputs, these systems may facilitate automated generation of radiology reports, pathology summaries, and treatment recommendations. However, concerns regarding hallucinations, factual reliability, and clinical accountability remain substantial barriers to implementation.

Self-supervised learning and contrastive learning approaches are also reshaping oncology AI development. These methods enable robust feature learning from unlabeled datasets by maximizing similarity between related samples while distinguishing unrelated examples.[60] Such strategies are particularly valuable in medicine because high-quality annotations are limited, costly, and time-intensive to obtain.

8. Multimodal Data Fusion Strategies

Effective multimodal AI depends fundamentally on robust data fusion strategies capable of integrating heterogeneous biomedical information while preserving biologically meaningful relationships. Fusion methodologies are generally categorized into early, intermediate, and late fusion approaches.[61] Early fusion combines raw input features before model training, allowing comprehensive joint representation learning but often increasing dimensionality and computational complexity. Intermediate fusion integrates modality-specific embeddings generated through independent subnetworks, enabling preservation of modality-specific characteristics while facilitating cross-modal interactions. Late fusion aggregates predictions generated independently by separate models and is often computationally efficient but may overlook latent biological relationships.[62]

Attention-based fusion architectures have gained prominence because they dynamically weight modality-specific contributions according to contextual relevance. Cross-modal attention mechanisms enable models to identify interactions between imaging features and genomic signatures associated with prognosis or therapeutic response.[63] Graph neural networks have additionally emerged as promising tools for multimodal oncology because they can model relational structures among molecular pathways, cellular interactions, and clinical variables.

Federated learning represents another important strategy for multimodal AI development. Because cancer datasets are often distributed across institutions and constrained by privacy regulations, federated learning enables collaborative model training without centralized data sharing.[64] This approach enhances model generalizability while preserving patient confidentiality. Nevertheless, challenges related to data heterogeneity, communication efficiency, and institutional bias remain unresolved.

Table 1. Comparison of Radiomics, Pathomics, and Genomics in Oncology

Applications of Multimodal AI in Early Cancer Detection

Early cancer detection remains among the most promising clinical applications of multimodal AI because prognosis is strongly dependent on disease stage at diagnosis.[65] Traditional screening modalities often suffer from limited sensitivity, specificity, or interobserver variability. Multimodal frameworks integrating radiological imaging, pathology, and molecular biomarkers have demonstrated enhanced diagnostic performance across several malignancies.

In lung cancer screening, AI systems combining low-dose CT radiomics with genomic and clinical data have improved discrimination between benign and malignant pulmonary nodules.[66] Similarly, multimodal mammography models integrating imaging with genomic risk scores and histopathological features have shown improved breast cancer detection accuracy. Liquid biopsy integration with imaging-based AI further enables minimally invasive detection of circulating tumor DNA and early metastatic dissemination.[67]

Foundation models trained on large multimodal screening datasets may eventually facilitate population-level cancer screening with improved scalability and personalization. However, false-positive findings, algorithmic bias, and insufficient prospective validation remain important concerns that must be addressed before widespread implementation.

9. Tumor Classification

Accurate tumor classification is critical for therapeutic selection and prognostic assessment. Multimodal AI has demonstrated superior capability for molecular subtype prediction compared with conventional clinicopathological approaches.[68] In gliomas, integrated radiopathogenomic models can predict IDH mutation status and MGMT promoter methylation using MRI and histopathology data. Similar frameworks have been developed for breast cancer receptor status classification and colorectal cancer microsatellite instability prediction.

The integration of pathology and genomics is particularly important for identifying molecularly defined tumor subgroups with distinct therapeutic vulnerabilities.[69] Deep learning algorithms can infer molecular phenotypes directly from whole-slide images, thereby enabling rapid and cost-effective stratification in resource-limited settings. Multimodal classification systems may ultimately reduce diagnostic uncertainty and support standardized oncology workflows.

10. Prognostic Prediction

Predicting survival outcomes and disease recurrence represents a major challenge in oncology because of substantial interpatient heterogeneity. Multimodal AI systems integrating imaging, pathology, molecular profiles, and clinical variables have consistently demonstrated improved prognostic performance relative to conventional staging systems.[70]

Radiogenomic models can identify imaging signatures associated with aggressive molecular phenotypes and metastatic potential. Pathomics-based analysis of tumor architecture and immune infiltration further provides insights into recurrence risk and therapeutic resistance.[71] Importantly, longitudinal multimodal models incorporating serial imaging and temporal genomic evolution may enable dynamic risk prediction throughout the disease course.

Several studies have shown that transformer-based survival models outperform traditional Cox proportional hazards approaches by capturing nonlinear interactions among multimodal variables.[72] Nevertheless, prospective validation and calibration across diverse populations remain essential before integration into clinical decision-making.

11. Precision Therapeutics

Precision oncology seeks to tailor treatment strategies according to individual tumor biology and patient characteristics. Multimodal AI has become increasingly important for predicting therapeutic response and identifying optimal treatment combinations.[73] Integrated radiomic, pathomic, and genomic signatures can identify patients likely to benefit from targeted therapies, chemotherapy, radiotherapy, or combination regimens.

AI-driven drug response prediction models utilize multimodal molecular data to infer pathway dependencies and resistance mechanisms.[74] In breast and lung cancers, multimodal systems have demonstrated encouraging performance for predicting response to HER2-targeted therapy, EGFR inhibitors, and immune checkpoint blockade. Such approaches may reduce unnecessary toxicity while improving therapeutic efficacy.

The emergence of digital twins and adaptive treatment modeling further illustrates the translational potential of multimodal AI. By integrating longitudinal clinical, imaging, and molecular data, computational models may eventually simulate treatment trajectories and optimize individualized therapeutic strategies in real time.[75]

12. Immuno-Oncology

Immuno-oncology has transformed cancer treatment through the development of immune checkpoint inhibitors and cellular therapies; however, response rates remain highly variable across patients.[76] Multimodal AI offers opportunities to characterize the complex interactions among tumors, immune cells, and the microenvironment that determine immunotherapy efficacy.

Pathomics-based quantification of tumor-infiltrating lymphocytes and spatial immune organization has shown strong associations with checkpoint inhibitor response.[77] Radiomics can additionally capture imaging correlates of immune activation and tumor inflammation, while transcriptomic analyses identify immunological gene signatures and cytokine networks. Integrating these modalities may improve prediction of response and immune-related adverse events.

Spatial transcriptomics combined with digital pathology has become particularly valuable for understanding immune microenvironment heterogeneity.[78] AI-driven analysis of spatial cellular organization may reveal mechanisms of immune escape and identify novel therapeutic targets. These approaches are expected to play an increasingly important role in the development of next-generation immunotherapies.

Table 2. Recent Applications of Multimodal AI in Different Cancer Types