International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 11 Issue: 01 | Jan 2024
p-ISSN: 2395-0072
www.irjet.net
Augmentation of Multimodal 3D Magnetic Resonance Imaging using Generative Adversarial Network for Brain Tumor Segmentation Bhavesh Parmar1, 2, Mehul Parikh2 1Research Scholar, Gujarat Technological University, Gujarat, India 2L. D. College of Engineering, Ahmedabad, Gujarat, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract – Gliomas are the most common malignancy in
3D UNet [6-9] are the most adopted method for brain tumor segmentation applications.
brain. Magnetic Resonance Imaging (MRI) images are widely used for medical imaging applications. Multimodal MRI can be used effectively for segmentation of brain tumors using Convolutional Neural Network (CNN). CNN accuracy depends on the large amount of data. Medical imaging datasets are mostly smaller in size. There is a chance of missing modality in MRI due to clinical settings or in case of patient not available for prognosis. This paper proposed a novel generator – discriminator architecture for development of cohort of MRI modality. Modified 3D UNet architectures were used as generator and PatchGAN for discriminator. In BraTS2020 dataset, there are four MRI modalities present. The modified 3D Unet provided with different three modalities as input and it produce the remaining modality. Discriminator block find the MSE, PSNR, and SSIM values to check before adopting as cohort. The generated modalities are used as input along with the original images in dataset for input to the modified 3D Unet. The model performance in segmentation improved up to 2 to 3 points in each tumor sub regions. The mean dice values achieved using the proposed work is 0.87. 0.81. and 0.78 for WT, TC, and ET respectively.
Model accuracy in segmentation for the deep learning network is data centric. Medical Imaging dataset are usually smaller in comparison to other computer vision datasets. The requirement of multimodal patient imaging might also not possible due to various clinical requirements. Additionally, prognosis and follow up of singular subject may not be possible. To cater the need, unavailable image data can be augmented using Generative Adversarial Network (GAN). In this paper, generative and discriminative method of GAN is proposed and compared its suitability and improvements in outcomes on existing methods of brain tumor segmentation application.
2. RELATED WORKS Due to success of deep learning in compute vision applications, data demand increasing day by day [10]. Unlike other computer vision dataset, medical imaging datasets are limited. The need of augmented data can solve the problem of inadequate dataset. Various application of GAN includes image to image translation [11], image synthesis from noise [12], style transfer [13], and image segmentation [14]. Due to smaller size of medical image datasets, GAN algorithms are getting popularity.
1. INTRODUCTION Magnetic Resonance Imaging (MRI) is the widely used imaging modality in the clinical practice. MRI can be very useful in assessing different insights in diagnosis and clinical planning. Multimodal characteristics of MRI scans can be very useful in neuroimaging, specifically brain tumor classification and segmentation. Gliomas are the most frequently occurring central nervous system (CNS) malignancy accounting 30% to all other CNS tumors [1]. World Health Organisation (WHO) classified brain tumors in grade I to IV considering aggression of malignancy. With amplifying order of antagonism, Grade I and II are Low Grade Glioma (LGG), while, grade III and IV are High Grade Glioma (HGG) [2]. Due to very high variations in size, shape, structure, infiltrative nature of growth and peritumoral edema, brain tumors are hard to delineate from its surrounding healthy tissues. Further, the large volume of MRI scan, Manual border segmentation with visual inspection is time consuming and prone to human error [3].
The Generative Adversarial Network (GAN) method was originally introduced in 2014, by Good fellow et al [10], for image generation. In Medical image synthesis, CycleGAN can be helpful for cohort of absent modalities [15]. GANS can also be used for segmentation of medical images. Han et al., has suggested GAN method for multiple spinal structure segmentation from MRI scans. Several methods suggested possibility of utilizing GAN in medical image segmentation [11, 16, 17, 18]. Another 2D GAN based method RescueNet was proposed for brain tumor segmentation from MRI images [19]. The detailed review of application of GAN is reviewed by Yi et al [20]. This work is motivated from the MM-GAN [21], a 3D extension of Pix2Pix GAN [22]. The method synthesise missing image of modality from the by the available images of another three modalities in multimodal brain tumor
Recently, deep learning methods have shown promising results in medical imaging [4]. 3D UNet [5] and variations of
© 2024, IRJET
|
Impact Factor value: 8.226
|
ISO 9001:2008 Certified Journal
|
Page 175