Improved UNet Framework with attention for Semantic Segmentation of Tumor Regions in Brain MRI Image

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

Improved UNet Framework with attention for Semantic Segmentation of Tumor Regions in Brain MRI Images

1 , Arimanda Chaitanya Sri2 , Saranu Charitha Sri3 1,2,3 Student of Dept. of CSE, Vignan’s Foundation for Science, Technology & Research, Vadlamudi, India ***

Abstract - BrainTumorSegmentationisacrucialtaskinmedicalimageprocessing.Braintumorsmustbedetectedearlyin ordertoimprovetreatmentoptionsandincreasepatientsurvivalrates.Achallengingandtime consumingtaskisdetecting tumorfromalargenumberofclinicalMRIimagesforcancerdiagnosis.Deeplearningalgorithmsforautomaticsegmentation haverecentlygainedtractionduetothefactthatthesemethodsproducecutting edgeresultsandarebettersuitedtothis problemthanotherapproaches.Deeplearningapproachescanalsobeusedtoefficientlyandobjectivelyprocessmassive amountsofMRI basedimagedata.SeveralreviewpapersonclassicMRI basedbraintumorimagesegmentationalgorithmsare available.BecauseSemanticSegmentationassignsaclasslabeltoeachpixelinagivenimage,itcanbeusedtosegmentbrain tumorimagesfromtheprovidedimages..Intheproposedmethodology,weperformabatchtrainingwhereeachrandomly created batch is passed to the variation of UNet, a popular Segmentation model. In this model, we have added batch normalizationsfollowingeveryconvolutionlayerwiththehopethatadeepernetworkhelpsextractingthebetterfeatures whichturnedouttobetrue.HereweprefertousethemetricasIntersectionoverUnion(IoU)[1]ratherthanaccuracybecause it is less influenced by the inherent class imbalances in foreground/background segmentation tasks. With the proposed methodology,weachieveanaveragedIoUof84.3anddicecoefficientvalueis91.4.

Key Words: BrainTumor,Segmentation,SemanticSegmentation,U Net,IntersectionoverUnion(IoU),Dicecoefficient

1. INTRODUCTION

Cancerisdefinedasuncontrollableandabnormalcelldivisionandproliferationinthebody.Abraintumorisanabnormalmass ofunnaturalcellgrowthanddivisioninbraintissue.Braintumorsareoneofthemostfatalcancers1,despitetheirrarity.

Braintumors[2]areclassifiedaseitherprimaryormetastaticbasedonwheretheyoriginate.Primarycancercellsoriginatein braintissue,whereasmetastaticcancercellsbecomecancerouselsewhereinthebodyandspreadtothebrain.Gliomasare brain tumors that develop from glial cells. While these modalities are used in conjunction to provide the most detailed information,becauseofitshighsofttissuecontrastandwidespreadavailability,MRIisroutinelyusedtoobtaininformation regardingbrainmalignanciesTheconventionalmethodMRI(magneticresonanceimaging)isanon invasivein vivoimaging techniquethatemploysradiofrequencypulsestoexcitetissues.

Figure1.1: Sample for segmenting the Brain Tumor images

Weusesegmentationtoefficientlylocateandsegmentbraintumorsinordertoperformsuccessfulsurgery.Braintumorscan beclassifiedintotwotypes.Thefirstismanualsegmentation,whichisasubjectivedecisionthatdoesnotproducethedesired resultsbecausecompletelyremovingbraintumorswithoutdestroyinghealthybraintissueisdifficult.Asaresult,automatic segmentationfortreatmentplanningandquantitativeevaluation,thesecondmethodisrequired.Itquicklyandaccurately diagnosesbraintumors.

Sinceboththelocationandsizeofthetumorsarerequiredtobeidentified,theproblemcomesunderthetaskofsegmentation anditparticularlycomesundersemanticsegmentation.SegmentationissomethingbeyondthetaskslikeImageclassification,

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

localizationandobjectdetection.Inimageclassification,wejustrequirethegivenimagetobeclassifiedintooneoftheclasses (binaryormulti).

Figure

1.2:

Levels of Understanding of an Image by a System

LocalizationisabitadvancementtoclassificationasWelocatetherequiredobjectintheprovidedimage.Objectdetection islikecombinationofbothofthosebecausehereweperformboththetaskslikeclassifyingtheobjectandlocatingit.Here locatingtheimageistojustcomeupwithaboundingboxandhenceitisnotthecasetobeusedwhenweneedtheexact shapeoftheImage..

2. Literature Survey

Sincethisbraintumorsegmentationisrelatedtothe,manyresearchersgetattractedtothiswork.Aspartoftheinitialstageof researchinthistopic,theresearchersusetoconsiderthehandcraftdifferentfeatureextractorsandusetheoutputsofthemfor theanalysisofthebraintumorimages.OneapproachisthemethodproposedbyNagashreeNandPremjyotiPatil[3].This system'smainideaistoworkontheencodinganddecodingphasesofUNet[4]modellingforefficientsegmentationofbrain images.Theinputimageisdividedintoseverallayerscalledconvolutionsinthismethodology,andtheCNNmethodisused. The process's convolution filter is the feature extraction of individual image layers. In the UNet approach, each layer is representedasanetworkencoderlayer.Alphabetpruning,anAIoptimizationalgorithmfordimensionalityreduction,was proposedasamodifiedformofUNet.Theprocessentailsbuildinga treenetworkoutofallthelayersoftheinputimage, retainingonlytheessentialimages.Theremainingimagelayersareprunedtosavetime..Theworkflowoftheirapproachisas follows:

Figure2.1: Workflow of Nagashree N and Premjyoti Patil

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

Convolutionalneuralnetworkshavebeenwidelyusedinthefieldofmedicalpicturesegmentationsincetheintroductionof deep learning for their great feature extraction capabilities, and have achieved good segmentation performance and robustness.Convolutional neural networkswereoriginallyusedinbraintumorsegmentationbyZikicetal.Thenetwork comprisesofaconvolutionlayer,amaximumpoolinglayer,afullconnectionlayer,andasoftmaxlayer.Ronnebergeretal suggesttheUnetnetwork,whichusesanencoder decodertopology.Convolutionwithsizeof33andstridesizeof1isusedfor 4timesdown samplinginthecodingphase;deconvolutionwithsizeof22andstridesizeof2isusedforup samplinginthe decodingphase.High resolutionandlow resolutioninformationareequallyrelevantduetothesimilarityofmedicalimaging andthefuzzinessoftumorregionboundaries.QingJunRuandGuangZhuChen[5]proposeanimprovedM Unetstructureto increasetheperformanceoffeaturefusionandtheaccuracyofnetworksegmentation.Thisapproachcanbeimprovedinthe followingways:

1.Amulti scalefeatureextractionmoduleisaddedtotheUnetnetwork'sfeaturefusionparttobetterextractthehigh leveland low levelfeaturesoftumorimages,whileredundantfeaturesareavoidedfrombeingintroducedintotheup samplingfeature map,furtherimprovingnetworksegmentationperformance.

2.Toacquirethebestnetworkweight,acosineannealinglearningrateattenuationapproachisutilisedinthetrainingphaseto makethenetworkjumpoutofthelocaloptimalsolution.

3. Methodology

Thisarchitectureoftheproposedmodel,trainingapproachandtheothertechnicaldetailsarediscussedinthissection.The proposed architecture is derived from the UNet architecture. Many changes like the number of filters at each layers, introductionofBatchNormalizationoperations,weremadetotheoriginalscore.ThedatasetusedforthisworkwaslggMRI segmentationdataset.Thisdataset[6]comprisesof3762imagesofsize256×256.Outofthe3762images,80%ofthemwere usedforthetrainingpurposeandtherestoftheimages(20%)wereusedfortestingpurpose.

Figure3.1: Sample Images

Asshowninthefigure3.2,thetrainingimagesandthecorrespondingmaskswereloadedandasavisualizationtechnique.In theproposedmethodology,weareusingrandom2828imagesfortrainingand708forvalidationand393fortesting.

TheperformanceofthemodelismonitoredinformofthemetricIntersectionofUnion(IoU),whichwasthemostcommon metricusedforthesegmentationtasks.CallbackslikeEarlyStoppingandModelCheckpointarefurtherimplementedonthe basisofaverageIoUofevery50batches.Thesecallbackshelpstosavethebestmodelandstopthetrainingprocessiftherewas nofurtherimprovement.Attheend,savethemodelweightssothattousethemlater.Therearemanyothercountlessefforts likeaugmentation,usingdifferentarchitecturesweremadebutnoneofthemprovedtobesuccessful.

4. Network Architecture

TheproposednetworkisderivedfromtheUNetarchitecture.UNet,beingapopularapproachtobeusedforsegmentationtask, appliesclassificationoneachandeverypixelinthegiveninputimageandtherebyproducingamaskofsamesizeasinput.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

Figure4.1: Proposed Architecture of varied UNet

Figure4.2: Proposed Architecture of varied UNet

Thenetworkshownabovetakesaninputofsize(256×256×3).Itfeaturesanencoder decoderstructure,andintheencoder part,weapplypoolingtechniquestoreducethesizeoftheimage.sothatwecanextracttheinformationof"what"ispresentin theimage.Weincreasethepicturesizeinthedecoderphasetoextracttheinformationof"where"inthegivenimage.Each blockinthecontractingpathhasthefollowingstructure:

Twoconvolutionallayersareusedinthefirstblock,followedbypoolingandbatchnormalizationprocesses,andthechannel countisincreasedfromonetoeight.Sincetheprocessofconvolutionincreasesthedepthoftheimage,therearefoursuch blocksintheentirecontractionpath,andbytheendofallfourblocks,thechannelcounthasincreasedto64channels.The imagesizeisreducedtobytheconclusionofthefourblocks(8,8)thankstothemaxpoolingprocedure,whichdecreasesthe imagesize..Fromhere,theexpansivepathwillbegin,inwhichtheimagesizeissteadilyraisedthroughupsamplingwhilethe channelcountisreduced.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

Transposedconvolutionisemployedhereasanupsamplingtechniquetoincreaseimagesize.Ontheinitialimage,apadding operationisdone,followedbyaconvolutionaction.Therearefoursuchblockshere,justlikeinthecontractionpath,andbythe endoftheseblocks,wewillhavetheoriginalsizeimage.Thefinalpredictionisobtainedbyapplyinga1Dconvolutionwith1 kernelandasigmoidactivationontheoutputofthelastblock.The1Dconvolutionreducesthenumberofchannelsnecessary forthenetworkoutput,whilethesigmoidactivationfunctionmapseverypixelintheoutputblocktotherangeoftherequired networkoutput(0,1).Theresultswillberoundedtothenearestinteger.

Themodelweightsareretainedattheendofthetrainingandusedinthetestingprocedure.Weusethesharpeningtechnique inthefinalstageofthetestingprocedure,aftertheoutputmaskprediction.Asapost processingapproach,sharpeningallows foragreaterviewofthesaltdepositspresentintheprojectedmask,resultinginahigherIoUscore.Lowpassandhighpass filtersarecommonlyusedonphotographstoimprovetheirviewingcapabilities.Smoothingisthetermusedtodescribetheuse ofalowpassfilter,whereassharpeningisthetermusedtodescribetheuseofahighpassfilter.Lowfrequenciesarefrequently attenuatedbyahighpassfilter,whichallowshighfrequenciestoflowthrough.Asaresult,thesaltpixelsintheexpectedmask passthroughthefilter,yieldingasuperioroutcome.Thekernelforsharpeninginoursuggestedmethodologyisrepresentedby thefollowingarray.

5. Results and Discussions

Inthetestingphase,weloadtheremaining20%datawithimagesandmasks.Theweightsthataresavedearlierareloadedinto themodelandthemodelisusedforthepredictionofthemasksforthegivenimages.Bycomparingtheanticipatedandoriginal masks,wecannowdeterminetheIoU(IntersectionoverUnion)value.TheaverageofIoUwasthencalculatedforarangeof criteriarangingfrom0.5to0.95,witha0.05stepbetweeneachisreported.IoUonathresholdtellsthataparticularIoUvalue hascrossedthatthreshold.ForExample,apredictedoutputmaskisconsideredtobevalidoverathresholdof0.7ifthevalueof IoUisabove0.7forthatparticularmask.ThefollowingistheinterpretationofIoUbetweengenuinesegmentationpixels,Y,and asimilarsetofpredictedsegmentationpixels,

Fig5.1: IOU Formula

WhichcanalsobeexpressedasafunctionoftheY Yconfusionmatrix

Fig5.2: General Confusion Matrix

IOUisthencalculatedas:(TP=Truepositives,FP=Falsepositives,etc.)

Fig5.3: IoU Formula in terms of confusion Matrix

HencetheresultsofthedifferentmethodsonvariousthresholdsandtheaverageIoUoverallthethresholdsfrom0.5to0.95 withastepvalueof0.05andthelossoftrainingisreportedinthefollowingtable.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

Table5.1:

Table5.2: Comparison of results with citations TheInputimagealongwiththetruemaskandthepredictedmaskareplottedhereforvariousimages.

Figure5.4: Predicted Masks comparison with the original mask

6. Conclusion

Weperformbatchtrainingintheproposedmanner,whereeachrandomlyformedbatchissubmittedtoavariantofUNet,a popularSegmentationmodel.Weaddedbatchnormalizationsaftereachconvolutionlayerinthismodelinthehopesthata

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

deepernetworkwillassistextractbetterfeatures,whichprovedouttobeaccurate.Insteadofaccuracy,weopttoutilisethe measureIntersectionoverUnion(IoU)[1],Thisislessinfluencedbytheinherentclassimbalancesinforeground/background segmentationtasks.Weget anaveragedIoUof84.3and a dicecoefficientvalueof91.4usingtheprovided methods.The proposedmodelwillbeimprovedinthefuturebyemployingdifferentfiltersizesandincludingallmodalitiesofMRIimagesin tumorsegmentation.Byraisingthemini batchsizefrom16to64andthemax epochfrom80to120,thesegmentationresult willbeimprovedevenmore.

7. References

[1] Rezatofighi, Hamid, et al. "Generalized intersection over union: A metric and a loss for bounding box regression." ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition.2019.

[2]ILong,J.,Shelhamer,E.,&Darrell,T.(2015).Fullyconvolutionalnetworksforsemanticsegmentation.InProceedingsofthe IEEEconferenceoncomputervisionandpatternrecognition(pp.3431 3440).

[3]AlphaBetaPrunedUNet AModifiedUNetFrameworktoSegmentMRIBrainImagetoAnalysetheEffectsofCNTNAP2Gene towardsAutismDetection.In20213rdInternational ConferenceonComputerCommunicationandtheInternet(ICCCI)(pp.23 26).IEEE. 240).IEEE.

[4]Kermi,A.,Mahmoudi,I.,&Khadir,M.T.(2018,September).DeepconvolutionalneuralnetworksusingU Netforautomatic braintumorsegmentationinmultimodalMRIvolumes.InInternationalMICCAIBrainlesionWorkshop(pp.37 48).Springer, Cham.

[5]Ru,Q.,Chen,G.,&Tang,Z.(2021,August).BrainTumorImageSegmentationMethodBasedonM UnetNetwork.In20214th InternationalConferenceonPatternRecognitionandArtificialIntelligence(PRAI)(pp.243 246).IEEE.

[6] Datased Used in The Discussed Model [Online]. Available: https://www.kaggle.com/datasets/mateuszbuda/lgg mri segmentation.

[7] Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A Deep Convolutional Encoder Decoder Architecture for Image Segmentation.IEEETrans.PatternAnal.Mach.Intell.2017,39,2481 2495.[GoogleScholar][CrossRef][PubMed]

[8]Guo,Meng Hao, Tian XingXu, Jiang JiangLiu,Zheng NingLiu,Peng Tao Jiang,Tai JiangMu, Song HaiZhang, RalphR. Martin,Ming MingCheng,andShi MinHu."Attentionmechanismsincomputervision:Asurvey."ComputationalVisualMedia (2022):1 38.

[9] Noh, H.; Hong, S.; Han, B. Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE InternationalConferenceonComputerVision(ICCV),Boston,MA,USA,7 12June2015;IEEE:Piscataway,NJ,USA,2015;pp. 1520 1528.

[10] Wu, X.; Liang, L.; Shi, Y.; Fomel, S. FaultSeg3D: Using synthetic data sets to train an end to end convolutional neural networkfor3Dseismicfaultsegmentation.Geophysics2019,84,IM35 IM45

[11]Ronneberger,O.;Fischer,P.;Brox,T.U Net:Convolutionalnetworksforbiomedicalimagesegmentation.InLectureNotes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer:Cham,Switzerland,2015;Volume9351,pp.234 241.

[12]McCaffrey,J.(2014).UnderstandingNeuralNetworkBatchTraining: ATutorial.VisualStudioMagazine.

[13]Daimary,Dinthisrang,etal."BraintumorsegmentationfromMRIimagesusinghybridconvolutionalneuralnetworks." ProcediaComputerScience167(2020):2419 242

[14]Rehman,MobeenUr,etal."Bu net:Braintumorsegmentationusingmodifiedu netarchitecture."Electronics9.12(2020): 2203

[15]Deb,Daizy,andSudiptaRoy."BraintumordetectionbasedonhybriddeepneuralnetworkinMRIbyadaptivesquirrel searchoptimization."Multimediatoolsandapplications80.2(2021):2621 2645

2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 07 | July 2022 www.irjet.net p ISSN: 2395 0072

8. Biographies

Heena Kousar

DepartmentofComputerScience&Engineering

Vignan’sFoundationforScience,Technology&Research

Arimanda Chaitanya Sri

DepartmentofComputerScience&Engineering

Vignan’sFoundationforScience,Technology&Research

Saranu Charitha Sri

DepartmentofComputerScience&Engineering Vignan’sFoundationforScience,Technology&Research

2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal