International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 13 Issue: 02 | Feb 2026
p-ISSN: 2395-0072
www.irjet.net
Integrating Multi-Modal Healthcare Data Using Hybrid Deep Learning Techniques Dr. R.MURUGANANTHAM 1 , R. Jyoshita 2, K. Abhinav 3, N. Sai Priya 4, M.Nihal 5 1Professor, Department of IT, TKR College of Engineering and Technology, Telangana, India 2,3,4,5B.Tech Students, Department of IT, TKR College of Engineering and Technology, Telangana, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The rapid advancement of smart healthcare
creates major challenges in developing reliable prediction systems [8], [15]. Multi-modal healthcare learning has emerged as a powerful solution to combine different modalities into a unified prediction framework. By integrating structured clinical data, time-series signals, and imaging information, multimodal deep learning systems improve prediction accuracy and clinical reliability compared to single-modal models [1], [16]. At the same time, hybrid deep learning architectures that combine CNN, LSTM, and DNN models enable effective feature extraction across diverse data types [4], [5]. Despite these advances, missing data remains one of the most critical issues affecting multi-modal healthcare systems.
technologies has resulted in the continuous generation of heterogeneous medical data from wearable sensors, IoT devices, electronic medical records, and medical imaging systems. Although deep learning has shown strong potential for healthcare prediction, its performance is highly affected by missing or incomplete data, which is common in real-world clinical environments. This paper proposes a Hybrid MultiModal Deep Learning Framework that integrates multiple single-modal neural networks through a Collaborative Concat Layer (CCL) to achieve robust healthcare prediction under incomplete input conditions. The proposed framework employs dedicated deep learning models for time-series, structured clinical data, and image modalities, and combines them into a unified prediction architecture. To address missing features, a correlation-driven Weight Matrix and collaborative node mechanism are used to estimate unavailable variables using learned relationships among health parameters. Unlike traditional fusion methods that fail when modalities are absent, the proposed CCL dynamically adapts to missing inputs without requiring model retraining. Experimental evaluation shows that the framework achieves approximately 89–91% prediction accuracy even when key variables are missing, outperforming conventional deep learning fusion techniques. The proposed system is modular, scalable, and suitable for practical smart healthcare applications requiring reliable multi-modal integration and stable prediction performance.
1.1 Background and Motivation The increasing adoption of wearable sensors and IoT-based medical devices enables continuous monitoring of patient health, supporting early detection of diseases and personalized healthcare services [15]. At the same time, hospitals maintain Electronic Medical Records (EMR) containing patient history, clinical notes, diagnosis reports, and laboratory results. Medical imaging modalities such as Xrays, CT scans, and ultrasound images provide additional diagnostic evidence for clinicians [2], [11].Although these data sources offer comprehensive health insights, they are often collected asynchronously and stored separately. Many healthcare systems still process each modality independently, leading to reduced clinical effectiveness. Deep learning has proven to be highly effective in extracting meaningful patterns from complex data; for example, CNNbased models have achieved near-human performance in medical image diagnosis tasks [2], [5]. Similarly, LSTM networks are widely used for time-series physiological signals due to their ability to learn temporal dependencies [4], [8]. Therefore, integrating multiple modalities into a hybrid learning framework is essential for building robust and accurate healthcare prediction systems [1], [16].
Key Words: Multi-Modal Healthcare, Hybrid Deep Learning, Collaborative Concat Layer (CCL), Missing Data Handling, Correlation Analysis, Wearable Sensors, IoT Healthcare, Medical Prediction, Feature Fusion, Deep Neural Networks
1. INTRODUCTION Modern healthcare is rapidly evolving due to the integration of Artificial Intelligence (AI), Internet of Things (IoT), wearable devices, and cloud-based medical platforms. These technologies generate large-scale and heterogeneous health data such as physiological signals, clinical reports, laboratory values, lifestyle records, and medical images. Deep learning models have demonstrated strong performance in healthcare applications including disease prediction, diagnosis, and clinical decision support [1], [2]. However, real-world healthcare data is often incomplete, inconsistent, and collected from multiple independent sources, which
© 2026, IRJET
|
Impact Factor value: 8.315
1.2 Challenges Prediction
in
Multi-Modal
Healthcare
Even though multi-modal deep learning improves prediction capability, practical healthcare environments introduce several challenges. A major limitation is missing or incomplete data caused by sensor failure, irregular monitoring, device heterogeneity, and human errors in
|
ISO 9001:2008 Certified Journal
|
Page 444