Multimodal Deep Learning for Cross-Image Diagnostic Integration
Supervisor: Dr. Joddat Fatima, Snr Assistant Professor
Co-Supervisor: Dr. Adeel M Syed, Associate Professor
Campus/School/Dept: BUIC H-11/BSEAS/SE
Supervisory Record:
- PhD Produced: 0
- PhD Enrolled: 0
- MS/MPhil Produced: 4
- MS/MPhil Enrolled: 1
Topic Brief Description:
This research focuses on developing advanced multimodal deep learning frameworks that integrate data from multiple medical imaging modalities (e.g., PET/CT, MRI/CT, X-ray/Ultrasound) to enhance diagnostic precision and reduce errors in medical diagnoses. By combining the strengths of various imaging techniques, the approach aims to provide a more comprehensive understanding of a patient’s condition. Designing efficient fusion architectures (e.g., attention-based transformers or convolutional neural networks) to harmonize diverse image formats and leveraging auxiliary data such as electronic health records (EHR) or genomic profiles for holistic patient analysis. The ultimate goal is to create robust, real-time AI systems that support clinicians in making accurate and timely diagnostic decisions, particularly for complex cases such as cancer staging, cardiovascular abnormalities, or rare diseases.
Research Objectives/Deliverables:
- Develop a Multimodal Fusion Framework: Create a deep learning architecture capable of combining multiple medical imaging modalities (e.g., PET/CT, MRI/CT) for a unified diagnostic interpretation.
- Enhance Diagnostic Accuracy: Evaluate how multimodal integration improves sensitivity and specificity compared to single-modality models. Focus on diagnosing complex conditions, such as cancer staging, neurological disorders, and cardiovascular diseases.
- Incorporate Auxiliary Data for Contextual Understanding: Integrate non-imaging data (e.g., electronic health records, lab results, or genomic data) into the diagnostic pipeline for a holistic patient assessment.
- Prototype for Clinical Testing: A user-friendly interface or software application demonstrating the model’s capabilities in a simulated clinical workflow.
Research Questions:
- How can deep learning architecture effectively fuse data from multiple medical imaging modalities (e.g., PET/CT, MRI/CT) to improve diagnostic accuracy?
- How does the integration of multimodal imaging data impact the sensitivity, specificity, and overall accuracy of disease detection and classification?
- Can multimodal approaches reduce diagnostic errors (false positives and negatives) compared to single-modality AI systems?
- What visual or explanatory tools can be developed to make diagnostic decisions transparent and clinically actionable?
- How can non-imaging data (e.g., electronic health records, lab results, or genomic profiles) be incorporated with imaging modalities for a holistic diagnostic approach?
- What computational and infrastructure optimizations are necessary to enable real-time processing of multimodal data in clinical settings?
Candidate’s Eligibility Profile:
- Academic Qualifications: Master’s degree (or equivalent) in Software Engineering, Computer Science, Biomedical Engineering, Data Science, or related fields. Strong academic record with coursework or projects in Artificial Intelligence, Machine Learning, or Medical Imaging.
- Technical Skills: Proficiency in programming languages such as Python, R, or MATLAB. Experience with machine learning frameworks like TensorFlow, PyTorch, or Keras. Experience in working with large-scale imaging datasets and medical data preprocessing.
- Research Experience: Publications in relevant journals or conferences (e.g., MICCAI, CVPR, or similar).