Multimodal Clinical Foundation Models

Multimodal AI (MAI) has enormous potential to provide useful tools for AI-integrated healthcare. To power clinical decision making, under NIH funding, we’re developing multimodal foundation models that integrate images, video, clinical imaging (CT, MRI, ultrasound), electronic health records, biospecimin, and genetics data. This project combines state-of-the-art techniques for multimodal learning and agentic architectures with vast amounts of clinical data and novel methods to enable ethical clinical reasoning within the MAI inference process.

Multimodal AI for clinical decision support.