A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING

1,2,3,4 Student, Information Science and Engineering, Vidyavardhaka College of Engineering, Karnataka, India 5 Assistant Professor, Information Science and Engineering, Vidyavardhaka College of Engineering, Karnataka, India ***

Abstract The Content Based Image Retrieval aims to find the similar images from a large scale dataset against a query image. Generally, the similarity between the representative features of the query image and dataset images is used to rank the images for reclamation. Also, the cornucopia of online networks for product and distribution, as well as the number of images accessible to consumers, continues to expand. Thus, in numerous areas, endless as well as wide digital image processing takes place. Thus, the quick access to these large image databases as well as the extraction of identical images from this large set of images from a given image (Query) pose significant challenges as well as involves effective ways. A CBIR system's effectiveness depends basically on the computation of point representation as well as similarity. For this purpose, we present a introductory but important Machine learning algorithm like Convolutional Neural Networks (CNN) or DCNN which has further delicacy and we can train further and further images, which has comparatively bigger database. CBIR systems allow another image dataset to detect affiliated images to such a query image. The search per picture function of Google search has to be the most popular CBIR method.

Key Words: CBIR, Content based Image Retrieval, CNN, Convolutional Neural Networks, DCNN, Image processing, Query image.

1. INTRODUCTION

Intherecentpasttheadvancementincomputerandmultimediatechnologieshasledtotheproductofdigitalimagesand cheap large image depositories. The size of image collections has increased fleetly due to this, including digital libraries, medicalimagesetc.Toattackthisrapid firegrowth,it'sneededtodevelopimagereclamationsystemswhichoperatesona largescale.Theprimary endistomakearobustsystemthatcreates,managesandqueryimagedatabasesin anaccurate manner. CBIR is the following advancement to the stride of keyword grounded systems in which images are mended groundedonthedata oftheircontents. The reclamation prosecutionof a CBIR framefortheutmost part relies upon the two variables; 1) feature representation 2) similarity estimation. The fundamental substance of CBIR is the point birth process. CNNs made out of a class of learnable models which can make use in operations like Image Retrieval, Image Bracket, Image Annotation, Image Recognition and so forth. With the provocation of the extraordinary success of deep learningalgorithmstotheinventioninthispaper,they'veusedforthereclamationtheimages.CBIRissubstantiallyused for looking through grounded on the content as opposed to the image reflections. It incorporates the system of representingandsortingoutimagesgroundedonthewordqueryimage.

Contentbasedimageretrieval(CBIR)isacomputervisionfashionthatgivesawayforsearchingapplicableimagesinlarge databases.Thishuntisgroundedontheimagefeatureslikecolor,textureandshapeoranyotherfeaturesbeingdeduced fromtheimageitself.TheperformanceofaCBIRsystemsubstantiallydependsonthenamedfeatures.Theimagesarefirst represented in terms of features in a high dimensional point space. Also, the similarity among images stored in the database and that of a query image is measured in the point space by using a distance metric e.g., Euclidean distance. Hence, for CBIR systems, representation of image data in terms of features and opting a similarity meassimilure, are the most critical factors. In CBIR, certain image features incorporate shading, face, and shape that can be resolved from the images. CBIR can be acted in two different ways. for illustration the primary fashion is ordering and second are looking. Exercising the strategy ordering, the birth of the features from the image and can be used to store this feature in point database as point vectors. In the alternate strategy for illustration in looking, the birth of the point vectors from the information images and these separated features are taken for correlation with point vectors accessible in the database. Andthisoutgrowthisusedforrecoveringmostmatchedimagesfromthedatabasetothequeryimage.Unnaturally,there are two kinds of images recovery live, (1) exact image retrieval and (2) applicable image retrieval. For exact image reclamation,thecoordinatingof100%withthequeryisdoneandinapplicableimagereclamation,thereclamationrelies

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page153

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072

uponthecontentsorfeaturesofimage.Inthisweprobe,MachinelearningalgorithmlikeConvolutionalNeuralNetworks (CNN)orDCNN whichhasfurtherdelicacyand wecan train further and further images,whichhascomparativelybigger database.

2. LITERATURE SURVEY

ThereviewsbasedonthemethodologyusedbyvariousauthorsintheirresearchworksforContentBasedImageRetrieval (CBIR),followingare:

2.1 Content based Image Retrieval using Deep Learning Technique with Distance Measures

The exploration work carried out in [1], Sirisha Kopparthi, Dr. N. K. Kameswara Rao, used Convolution neural networks (CNN) with deep learning performed an excellent performance in numerous operations of image processing. The use of CNNbasedtechniquestoextractimagefeaturesfromthefinallayerandtheuseofasingleCNNstructuremaybeusedfor identifyingmatchingimages.LearningfeatureextractionandeffectivesimilaritycomparisoncomprisestheContent Based Image Retrieval (CBIR). In CBIR feature extraction, as well as similarity measures, play a vital role. The experiments are carriedoutintwodatasetssuchasUCMercedLandUseDatasetByusingapre trainedmodelthatistrainedonmillionsof imagesandisfine tunedfor thereclamationtask.Pre trainedCNN modelsareused forgeneratingfeaturedescriptorsof images for the retrieval process. This method deals with the attribute extraction from the two fully connected layers, which is present in the VGG 16 and VGG19 network by using transfer learning and retrieval of feature vectors using various similarity measures. The proposed architecture demonstrates an outstanding performance in extracting the featuresaswellaslearningfeatureswithoutapriorknowledgeabouttheimages.Byusingvariousperformancemetrics.

2.2 Image Retrieval based on the Combination of Color Histogram and Color Moment

Theresearchworkcarriedoutin[2],S.MangijaoSingh,K.Hemachandran,usedanoveltechniqueforContentbasedimage retrieval (CBIR) that employs color histogram and color moment of images is proposed. The color histogram has the advantagesofrotationandtranslationinvarianceandithasthedisadvantagesoflackofspatialinformation.Inthispaper, toimprovetheretrievalaccuracy,acontent basedimageretrievalmethodisproposedinwhichcolorhistogramandcolor moment feature vectors are combined. For color moment, to improve the discriminating power of color indexing techniques,aminimalamountofspatialinformationisencodedinthecolorindexbydividingtheimagehorizontallyinto three equal nonoverlapping regions. The three moments (mean, variance and skewness) are extracted from each region (inthiscasethreeregions),forallthecolorchannels.Thus,foraHSVcolorspace,27floatingpointnumbersareusedfor indexing.TheHSV(16,4,4)quantizationschemehasbeenadoptedforcolorhistogramandanimageisrepresentedbya vector of 256 dimension. Weights are assigned to each feature respectively and calculate the similarity with combined featuresofcolorhistogramandcolormomentusingHistogramintersectiondistanceandEuclideandistanceassimilarity measures.

2.3 Content Based Image Retrieval Using Deep Learning

Theexplorationworkcarriedoutin[3],AnshumanVikramSingh,useddeeplearningapproachesespeciallyConvolutional NeuralNetworks(CNN)inworkingcomputervisionoperationswhichhasinspiredauthortoworkonthisthesissoasto break theproblemofCBIR usinga datasetofannotated images.Heworkedwithonly3000imagesfrom41ordersand8 classes.Infutureto makethesystemmore generalized andeffectivethedatasetcanbeincreasedandfurther numberof classes similar as man, person, aeroplane etc can be added. The neural network was trained on the dataset for each marker.Itcanbeobservedthattheconfirmationerrorrateandtestingerrorrateforeachmarkerwasrelativelylow.The stylish test and confirmation error rates are achieved on replication 3, 6 or 9. Training runs for 500 duplications just to validatethatthere'snochangeintheerrorrateafterevery100duplicationsandonceitreaches500withaconstantrateit stopsandreturnsthestylish error rates.Theimagesin ourdatasetcontainreflectionsofdifferent regionsintheformof XML lines.TheExtensibleMarkupLanguage(XML)reflectionsgivetheannotatedimagedescription ofeachimageinthe dataset.Asemanticgapexistsbetweenlow positionimagepixelscapturedbymachinesandthehigh positionsemantics perceivedbyhumans.Thethesisshowsthatdeeplearningwillproducebetterresultsforannotatedimagesandwhichwill affectinmoreaccurateimagereclamation.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072

2.4 Fast content based image retrieval using Convolutional Neural Network and hash function

The exploration work carried out in [4], Domonkos Varga and Tamas Sziranyi, habituated success ways of deep learning similar as Convolution Neural Network (CNN), it has motivated them to explore its operation in their own environment. Due to the explosive increase of online images, content based image retrieval has gained a lot of attention. The main donation of their work is a new end to end supervised learning frame that learns probability grounded semantic positionsimilarityandpoint positionsimilaritycontemporaneously.Themainadvantageofnewmincingschemethatit's suitable to reduce the computational cost of reclamation significantly at the state of the art effectiveness position. They report on comprehensive trials using publicly available datasets similar as Oxford, Leaves and ImageNet 2012 retrieval datasets.

2.5 Medical Image Retrieval using Deep Convolutional Neural Network

The explorationwork carriedoutin[5],AdnanQayyum, Syed MuhammadAnwar,MuhammadAwais,MuhammadMajid, used a frame of deep learning for CBMIR system by using deep Convolutional Neural Network (CNN) that's trained for bracketofmedical images. AmajorchallengeinCBMIR systemsisthesemanticgapthatexists between thelowposition visualinformationcapturedbyimagingbiasandhighpositionsemanticinformationperceivedbymortal.Theefficacityof similarsystemsismorepivotalintermsofpointrepresentationsthatcancharacterizethehigh positioninformationfully. The learned features and the bracket results are used to recoup medical images. For reclamation, stylish results are achieved when class grounded prognostications are used. An average bracket delicacy of 99.77 and a mean average perfection of 0.69 is achieved for reclamation task. The proposed system is best suited to recoup multimodal medical imagesfordifferentbodyorgans.

2.6 Content based image retrieval using deep learning process

The exploration work carried out in [6], R. Rani Saritha, Varghese Paul, P. Ganesh Kumar, used the deep belief network (DBN)systemofdeeplearningwhichisusedtoprizethefeaturesandbracketandisanarisingexplorationarea,because of the generation of large volume of data. A multi feature image reclamation system is introduced by combining the features of color histogram, edge, edge directions, edge histogram and texture features, etc. In this model, the content groundedimagewillbeuprootedfromacollectionofintendedimagegroups.Afterperformingsomepre processingway likeselectionjunking,itsbelowfeaturesareuprootedandarestoredassmallhandlines.CBIRusesimagecontentfeatures to search and recoup digital images from a large database. A variety of visual point birth ways have been employed to applythesearchingpurpose.Duetothecalculationtimedemand,somegoodalgorithmsaren'tbeenused.Thereclamation performance of a content grounded image reclamation system crucially depends on the point representation and similarity measures. The ultimate end of the proposed system is to give an effective algorithm. The proposed system is testedthroughsimulationincomparisonandtheresultsshowahugepositivedivagationtowardsitsperformance.

2.7 Content-based Image Retrieval from Videos using CBIR and ABIR algorithm

Theexplorationworkcarriedoutin[7],VrushaliA.Wankhede,usedAnnotation basedimageretrieval(ABIR)andContent basedimageretrieval(CBIR).Videotapereclamationcanbeusedforvideotapehuntandbrowsingwhichareusefulinweb operations. Selectionofuprootedfeaturesplayanimportantpartincontent basedvideotapereclamation. Therearetwo types of point birth, low position point birth and high position point birth. Low position point birth grounded on color, shape,texture,spatialrelationship.Themainthingofthispaperisthat,stonercangivethetwodifferenttypesofinputin theformofimagequeryandthetextbookquery.TheABIRismorepracticalinsomeothersphere.Inthat,theywillgetthe set of frames or set of images from the below step, also give the labelling of one image from every videotape. The information store in the XML train information contains their path, images assign with the word or textbook. Reflection meanswhat'sintheimage,what'sitabout,whatdoesitbring?Givethedetaildescriptiontothatimage.Prepareannotated data. Take any one word or textbook as an input from the annotated data. Find the keyword from pre processed data if they match take that image as an affair image. Display the result means retrieves the image by using the textbook query hunt. With the help of labelling, they got the image from the particular path. The main thing of this paper is to apply the multiqueryimageretrieval.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072

2.8 Context aware Recommendation System using Content Based Image Retrieval with Dynamic Context Considered

Theexplorationworkcarriedoutin[8],YutaMiyazawa,YukikoYamamoto,TakashiKawabe,proposedpaperconceptually proposesa environment apprehensiverecommendationsystemthatgivesoptimal informationfordruggiesgroundedon 1)acontent basedimageretrieval(CBIR)mediumtosearchthesimilarimagesaimingtoprizethedetailedinformationto thetextbook indescribableimages2)thecontextualinformationofsimilaranalogousimagessearchedfromtheWeb,and 3)stoner’sdynamicenvironmentorsituationconsideringtime variantfactorsaswell asspacefactors.It'santicipatedto increase the perfection or optimality of recommendation by matching and fusing the environment of analogous images attainedbyCBIRwithtextualandstarinformationaboutstoner’ssituationordynamicenvironment.isdescribedjustata abstract position; thus, as a coming step for the exploration, the prototype system will be developed grounded on more detailedperpetrationdesign.Also,theutilityandeffectivenessoftheproposedideaanditsperpetrationwillbevalidated throughtheevaluationtrialusingtheprototypesystem.

2.9 Content based image retrieval: A review of recent trends

The exploration work carried out in [9], Ibtihaal M. Hameed, Sadiq H. Abdulhussain & Basheera M. Mahmmod, says that there are adding exploration in this field, this paper checks, analyses and compares the current state of the art methodologies over the last six times in the CBIR field. This paper also provides an overview of CBIR frame, recent low position point birth styles, machine literacy algorithms, similarity measures, and a performance evaluation to inspire fartherexplorationsweats.TonegotiateaneffectiveCBIRframe,theframe’sfactorsmustbechoseninabalancedway;this study helps in probing these factors. To add up, an algorithm that elevates the semantic gap is largely demanded. The design of the algorithm should consider the following first, the algorithm needs to consider the point birth as well as similarity measure as they impact the performance of the CBIR. Second, further features can be uprooted to enhance the delicacyoftheCBIR andmaintainthecomputational costasit'sconsideredimportantfactorinthereal timeoperations. Third,incorporatingoriginal andglobal featureswill leadtoa balanceddesignbecauseoriginal featuresaremorerobust against scale, restatement and gyration changes than global features; and global features are briskly in point birth and similarity measures. Fourth, machine literacy algorithms can be used in different stages of CBIR to increase system delicacy but need further attention to be paid to their calculation cost. Eventually, there's a dicker between system’s delicacyandcomputationalcost.

2.10 A Decade Survey of Content Based Image Retrieval using Deep Learning

Theexplorationworkcarriedoutin[10],ShivRamDubey,thischeckalsopresentsaperformanceanalysisforthestate of the artdeepliteracygroundedimagereclamationapproaches.TheMeanAveragePrecision(chart)reportedforthe differentimagereclamationapproachesisepitomized.ThemAP@5000(i.e.,5000recapturedimages)usingcolorfulbeing deeplearningapproachesisepitomizedoverCIFAR 10,NUS WIDEandMSCOCOdatasets.TheresultsoverCIFAR 10, ImageNetandMNISTdatasetsusingdifferentstate of the artdeepliteracygroundedimagereclamationstylesare collectedintermsofthemAP@1000.ThemAP@54000usingmanystylesisreportedovertheCIFAR 10dataset.The standardchartisalsodepictedbyconsideringalltherecapturedimagesforCIFAR 10datasetusingsomeoftheavailable literature.Ingeneral,informationreclamationalgorithmsinrecenttimesattainedthebenefitsofusingdifferentmachine learningalgorithmssimilarasdeepliteracy,SVM,andk means.Thus,they'reprognosticatedtoadmitfurtherattentionin theforthcomingtimes

Table 1: Summaryofpapersreviewed

Sl.No Title Year of Publication Result

1 ContentBasedImageRetrievalusingDeep LearningTechniquewithDistanceMeasures

2 ImageRetrievalbasedontheCombination ofColorHistogramandColorMoment

2020

EuclideandistanceandCosinesimilaritymeasuredseparately. Precisionrateof0.86obtainedforvgg16and0.89forvgg19.

2012 Two features are introduced color histogram and color moment. The retrieval efficiency of the proposed method is testedforbothfeaturesseparatelyandcombined. Increase in retrieval efficiency with both combined is observed.

International Research Journal of Engineering and Technology (IRJET)

e ISSN: 2395 0056

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072

3 Content BasedImageRetrievalUsingDeep Learning 2015 UsedCNNandDeeplearningmethods.

Numberofiterationswereperformedinwhichlowesttest andvalidationerrorswereobtainedin3rd,6th and9th iterations.

Trainingperiodrunsfor500iterationstoreturnbesterror rates.

4 FastContent Basedimageretrievalusing ConvolutionalNeuralNetworkandHash Function

5 MedicalimageRetrievalusingDeep ConvolutionalNeuralNetwork

2016 Resultsobtainedbasedonqueryimageaccordingtothe datasetsused.

Semanticsarepresentwhichmakestargetimagessimilarto queryimage.

Torch7isusedasatoolfordevelopmentandtraining. Systemisevaluatedbasedonclassificationandretrieval.

6 ContentbasedImageRetrievalusingDeep LearningProcess 2021 Accuracycalculatedbasedonprecisionratesandmaps obtained.

mAP=85.23% mAR=88.53%

7 ContentBasedImageRetrievalfromVideos usingCBIRandABIRalgorithms 2015 CBIRalgorithmisusedtotryandextractimageformultiple queries.

Multi queryimageretrievalisintroduced.

8 Context awareRecommendationSystem usingContentBasedImageRetrievalwith DynamicContextConsidered

2013 CBIRmechanismisusedtosearchsimilarimages. Contextualinformationofimagessearchedfromweb. Precisionrateisobtainedthroughfusingtheinformation.

9 Content basedImageRetrieval:Areviewof recenttrends 2021 StudiesinCBIRdomainispresent. GeneralCBIRframeworkstagesalsodiscussed. FactorsaffectingCBIRperformanceishighlighted.

10 ADecadeSurveyofContentBasedImage RetrievalusingDeepLearning 2021 Differentsupervisiontypes,differentnetworksusedare explainedindetail.

Evolutionofdeeplearningmodelsisalsoshown. Summaryoflarge scaledatasetscommonlyusedinCBIRis alsopresent.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072

5 AdnanQayyum

SyedMuhammadAnwar MuhammadAwais MuhammadMajid

24 Publiclyavailablemedicaldatabases 6 R.RaniSaritha VarghesePaul P.GaneshKumar

6 Imagedataset 7 VrushaliA,Wankhede PrakashS.Mohod 8 ImageDatabase 8 YutaMiyazawa YukikoYamamoto TakashiKawabe

5 ImageandText 9

IbtihaalM.Hameed SadiqH.Abdulhussian BasheeraM.Mahmmod

10 10 10 26

Corel CIFAR WANG Olivia 10 ShivRamDubey 10 10 10 80 397

CIFAR 10 MNIST SVHN MSCOCO SUN397

4. CONCLUSIONS

Theneedtofindaneffectiveimagereclamationmediumgroundedonimagecontentismotivatedbythelargequantumof imagedatabasesandtheabsenceofaneffectivetext basedimageretrievalsystem.Thispaperpresentsacomprehensive check of deep learning styles for content based image retrieval. As utmost of the deep learning grounded developments arerecent,thischeckmajorlyfocusesovertheimagereclamationstylesusingdeeplearninginadecade.Theexploration trend in image reclamation suggests that the deep learning grounded models are driving the progress. The lately developed models similar as generative inimical networks, autoencoder networks and reinforcement learning networks haveshownthesuperiorperformanceforimagereclamation.Thediscoveryofbetterobjectivefunctionshasbeenalsothe trend in order to constrain the literacy of the hash code for discriminative, robust and effective image reclamation. The semantic conserving class specific point literacy using different networks and different quantization ways is also the recenttrendforimagereclamation.ThispaperalsobandiedthegeneralCBIRframestagesandthemostrecentwaysused to reduce the semantic gap. To negotiate an effective CBIR frame, the frame’s factors must be chosen in a balanced way; this study helps in probing these factors. Similar effective CBIR architecture will contribute to numerous real world operations,similarasmedicaloperations,webquests,andsocialmedia.

REFERENCES

[1] Sirisha Kopparthi, Dr. N. K. Kameswara Rao, “Content based Image Retrieval using Deep Learning Technique with DistanceMeasures”,Science,Technology&HumanValues9(12):251 261,2020.

[2] S. Mangijao Singh, K. Hemachandran, “Image Retrieval based on the Combination of Color Histogram and Color Moment”,InternationalJournalofComputerApplications,Volume58 No.3,November2012.

[3] Anshuman Vikram Singh, “Content Based Image Retrieval Using Deep Learning”, Thesis. Rochester Institute of Technology,RITScholarWorks,2015.

[4]DomonkosVarga,TamasSziranyi,“FastContent BasedimageretrievalusingConvolutionalNeuralNetwork andHash Function”,IEEEInternationalConferenceonSystems,Man,andCybernetics(SMC),2016.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

[5]AdnanQayyum,SyedMuhammadAnwar,MuhammadAwais,MuhammadMajid,“Medical ImageRetrieval usingDeep ConvolutionalNeuralNetwork”, Neurocomputing,2017 Elsevier.

[6]R.RaniSaritha,VarghesePaul,P.GaneshKumar,“Contentbasedimageretrievalusingdeeplearningprocess”,2018.

[7] Vrushali A. Wankhede , and Prakash S. Mohod, “Content based Image Retrieval from Videos using CBIR and ABIR algorithm”,GlobalConferenceonCommunicationTechnologies,2015.

[8] Yuta Miyazawa, Yukiko Yamamoto, Takashi Kawabe , “Context aware Recommendation System using Content Based Image Retrieval with Dynamic Context Considered”, International Conference on Signal Image Technology & Internet BasedSystems,2013.

[9] Ibtihaal M. Hameed, Sadiq H. Abdulhussain & Basheera M. Mahmmod , “Content based image retrieval: A review of recenttrends”,ISSN,2021.

[10] Shiv Ram Dubey, “A Decade Survey of Content Based Image Retrieval using Deep Learning”, IEEE Transactions on CircuitsandSystemsforVideoTechnology,May2021.

Volume: 09 Issue: 06 | June 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal