AUTOMATIC TRANSFER OF DATA USING SERVICE-ORIENTED ARCHITECTURE TO NoSQL DATABASES

AUTOMATIC TRANSFER OF DATA USING SERVICE-ORIENTED ARCHITECTURE TO NoSQL DATABASES

Abstract - Since a few years ago, there has been a meteoric increase in the number of databases that are not genuine relational databases, which may be described as exponential growth. No one term accurately describes these kinds of databases; rather, they can only be characterised by a collection of qualities that are shared by all of them, such as the lack of a predefined schema, inherent scalability features, high performance, data, and so on. NoSQL is the acronym that has been used to refer to these types of databases. Many businesses are beginning to see the value of NoSQL databases and are interested in making the transition to using them. On the other hand, they struggle to move their data since the process requires a great deal of research and thought. The nomenclature and query language used with each kind of database is unique to that database. These businesses will be able to effortlessly move to the NoSQL databases of their choice with the assistance of our innovative automated migration approach, which makes use of the power of serviceoriented architecture. We make use of web services that encapsulate some of the most well-known NoSQL databases, such as MongoDB, Neo4j, and Cassandra, among others. This allows us to conceal the inner workings of these databases while still enabling the efficient migration of data with very little or no prior knowledge of how these databases function. To demonstrate the viability of the idea, relational data was successfully moved from the Apache Derby database to the NoSQL databases MongoDB, Cassandra, Neo4j, and DynamoDB, respectively. Each vendor represents a unique NoSQL database type.

Key Words: NoSQL, Service Oriented Architecture, Cassandra,MongoDB,Neo4j,Amazon

1. INTRODUCTION

Relationaldatabaseshavebeenanintegralpartofmanyof our web-based programmes, websites, and apps for quite some time. This custom has been around for quite some time.Problemswithrelationaldatabasesincludeimpedance disparity (the gap between what application developers needasdataandhowdataisstoredonadisc),theinability to naturally support clusters, and the difficulty in horizontallyscalingasdatagrowthoccurs.Whenthereisa mismatch between what application developers need and how data is stored on the relational database, a problem knownastheimpedancegaparises.NoSQL(whichstands for"notonlySQL")databaseswerecoinedtoindicatethat

they did not primarily use the SQL query language, and eventuallymanyothercompaniesfollowedsuitwiththeir solutions.

TheInternationalDataCorporationestimatesthatby2022, micro-servicearchitectureswillbeusedinthedevelopment of90%ofnewapplications.Currenttendencieswereusedto constructtheseforecasts.Whenmicroservicearchitecture andtheDevOpsmethodologyareusedtogether,thesoftware maybedevelopedwithmorespeedandadaptability.As a result,companiesmaymorequicklyintroducetheirdigital goods and services to highly competitive marketplaces. Businesses are starting to modernise their outdated monolithicsystemsbysplittingthemupintosmaller,more manageablemicroservicessothattheycankeepupwiththe competition and safeguard their competitive edge. While academics and engineers have looked at the problem of migrating monolithic software to a microservice architecture, not as much attention has been paid to how databasesshouldchangetoaccommodatethischange.This isabiggapintheknowledgebase.Partoftheinvestigation has gone missing in this area, and it must be pieced back together as soon as possible. Experts in the field agree, however,thatdatamanagementisoneofthemicroservices' mostsignificantlimitations.Microservicediscoverywithin monolithic programmes and microservice dissection of sourcecodehavebeentheprimaryresearchfoci.Thistopic hasbeenthefocusofagreatdealofresearch.Theresearch will be focused on these two primary goals. While transitioning from a monolithic design to a microservice architecture,thereislittletonoguidanceonhowtomodify currentdatastoragetoaccommodatethenewarchitecture providedbyanyoftheexistingmigrationapproaches.Thisis becausetherevisedarchitecturewasnotconsideredduring the development of any migration strategies. The author believes that, except A. No migration techniques have investigated theadaption of data storage to micro-service architecture outside the one proposed by Levcovitz et al. That's in contrast to A's method. It was proposed by Levcovitzetal.thatmicro-servicesbeextractedfromlarge, centralised enterprise systems. Unlike strategy A, though. For extracting micro-services from monolith business systems,thisstrategydepartsfromtherecommendationsof Levcovitzandothers.ContrarytoA.'sassertions,Levcovitz andhisgrouphavediscoveredthistobetrue.

1.1. NoSQL

Severalinnovativemethodsofdatabaseadministrationhave emergedinthelastdecade.Duetotheever-evolvingnature ofinformation,relationaldatabaseshavestruggledtokeep upwiththelikesofGoogle,Amazon,Facebook,andTwitter since the rise of Web 2.0. Social media and other internet records are one such example of the kind of data we are talking about here. These companies have come to this decisionbecauserelationaldatabasessimplycannotkeepup with the volume and velocity of information. The sheer amountofdata minedfrom social mediaandotheronline activity logs simply overwhelms traditional relational databases.Moreover,therelationshipbetweentheitemsis notpresentedstraightforwardlybyrelationaldatabases,and asignificantnumberofjoinqueriesareneededtocollectall oftheinformationthatisrelatedtotheobjects.Keepinmind that horizontal expansion is not a natural feature of relationaldatabases.Youshouldkeepthisinmind.Web2.0 businesses now almost always employ clusters as their default backend infrastructure. The integration of a relationaldatabaseintotheclusterwasdescribedasoneof the primary challenges that needed to be met. This was a majorcontributortothesuccessofNoSQLdatabasesystems.

1.2. Aggregate Data Modeling

Data in relational databases is often organized into rows (tuples).Norowsmaybeincludedinsideoneanother,nor mayarowcontainalistofvalues.Can'thaveitbothways, sorry. Both of the alternatives you listed are not possible. This is because a row can only process a finite amount of information at once. The use of nested data is crucial in disciplines like biology, which deals with living things. Aggregate orientation should be considered as a solution. Theaggregateperspectivebroadensourviewofthedataso thatwemayexamineitsunderlyingstructures,ratherthan itscomponentrecords.Inthisway,thedataismoreeasily digestible.Thisfindingstronglysuggeststhatnestingcould beapossibility.

From its origins in Domain-Driven Design, the word "aggregate"isusedtodescribeacollectionofinterconnected partsthatareoftentreatedasawhole.Researchofthiskind is where the term originates. Databases on a cluster are greatlyfacilitatedbyseeingdatainaggregatessincetheyare the natural units for sharding and replication. As a result, there is more database compatibility. Aggregates are the mostnatural methodtostructuremassivedatasets,hence theyworkwellforthispurpose.Consideredinthisview,it also facilitates the handling of enormous data sets. Aggregatesareeasierforprogrammerstowork withthan tuples. The information in many NoSQL databases is aggregate-oriented[5],buteachofthesedatabaseshasits uniquewayofstoringthisinformation.MongoDBisagood exampleofthistypeofdatabase.

Over the last several years, the popularity of document databases known as "MongoDB" has skyrocketed. A documentinMongoDBservesasimilarpurposetoarowina relational database. A collection in MongoDB is the equivalent of a table in a relational database, just as a schema is the equivalent of a database and a table is the equivalentofacollection.Becauseofitsuseofreplicasets, theMongoDBdatabasecanprovideveryreliableavailability. The master-slave setup is used for replicating data inside clusters. Adding new nodes to a graph database has no impact on the database's functionality, unlike relational databases.Allsortsofenterprises,largeandsmall,havethis need.

2. SERVICE-ORIENTED ARCHITECTURE

In no way is it a revolutionary idea to place greater importance on service. It achieves this by using the timetested strategy of "split and conquer" and the principle of "codereuse."Applicationsbuiltontopofaservice-oriented architecture (SOA) may be seen as a set of interrelated services.Incontrast,theseservicesdon'tneedtobedirectly ownedbythesamecorporation,whichisakeydistinction. Youcan'toverlookthisonebitofinformation.Tothispoint, solutions to each issue may be classified into one of two groups.

3. DATA MIGRATION MODEL

Withtheendaimofcreatingamigrationmodelinmind,the approach of using the construction technique known as ServiceOrientedArchitecturewasselected.Duringtheplan's implementation, a broad range of service options will be drawnfromacommonpool.

relationaldatabase.Toshowhowwelloursystemperforms, we constructed this demonstration system. Figure 4 is a visual representation of the system in action and gives further context for the discussion that follows. Additional referencesmaybefoundin[thefollowing]Duringtheactual implementation,OpenESBversion2.3wasused.Thissample demonstrates the system's "Business Process Enterprise Logic."Tocondensethewholemeaningof"BusinessProcess Enterprise Logic." Information from the previous user is usedtorequestthereadDBservice,whichgrantsaccessto the database. This happens after you have acquired the information from the previous user. Next up in the procedure is this step, which follows right after the one beforeit.Neo4j,MongoDB,Cassandra,orAmazonDBmayall be used as the underlying database and their respective insertionservicesmaybeused.Theultimatedecisionrests withtheuser.Toputitsimply,multi-serviceserversarenot necessaryatanytime.

The algorithm-01 examines the database's structure and drawsajudgementonthepresenceorabsenceofaprimary key-foreignkeyrelationallink.Iftherelationshipdoesexist, thentherelevanttablesshouldbejoinedtogetherandthe resulting dataset should include both the joined and individualresults.Iftheconnectionismissing,thefindings should be included in the dataset without any further processing.AmodifiedJSON stringcontainingthisdataset willbesentovertheinternet

4. IMPLEMENTATION OF MIGRATION MODEL

Asaproofofconcept,wedevelopedamechanismtomove datafromApacheDerby,arelationaldatabase,toMongoDB, Cassandra,AmazonDynamoDB,orNeo4j.Thedesignofthis systemwasinspiredbythestructureoftheApacheDerby

5. CONCLUSION

Distributed computing, in which information is processed overadecentralisednetwork,hasgainedprominenceover thelastdecadeasapromisingapproachtothecreationof modern online applications. The value of the cloud computingmarkethasskyrocketedinrecentyears.Because DBMSs are in charge of storing and displaying an application'sdata,informationhasbecomeacrucialpartof web-based programmes. It is the goal of this thesis to

provideaframeworkfordesigningasystemthatcanmigrate data from traditional relational databases to the most populartypesofnon-relational databases.Thecreationof suchasystemwillalsoformthebackboneofthisthesis.The second objective of this thesis is to provide a tool for migrating data from relational to non-relational storage systems.Thisresearchanalysesastrategyandapproaches thatmighthelpsoftwarecompaniesmigratetheirpresent relational databases to NoSQL providers utilising serviceoriented architecture. The University of Washington conductedthestudy.Bothmethodsandanoverallplanare detailedinthisstudy.Thereisanextralevelofabstraction built into Asp. The intention was to facilitate the shift to NoSQL databases as much as possible. Web services have beencreatedtoexaminetheschemaofrelationaldatabases andimplementchangesautomatically.Throughtheuse of webservices,wewereabletoprovidesupportforavariety ofNoSQLdatabases,includingMongoDB,Cassandra,Neo4j, andAmazonDynamoDB;thisalsoallowedformulti-vendor compatibility.Someoftheinformationthatmaybefoundin thesedatabasesincludes:Roughlyone-fiftiethofallsoftware applicationstodayareNoSQLsystems,andthereareovera hundreddistinctorganisationsacrosstheworldthatprovide thisservice.

REFERENCE

1. K.Mehra, Y.Yan and D.Lemure, Automatic data migration to the cloud in the Sixth International Workshop on Cloud Data Management (CloudDB 2014)

2. JingHan,EHaihong,GuanLe,andJianDu.Surveyon NoSQL database. In Pervasive computing and applications (ICPCA), 2011 6th international conferenceon,IEEE,2011.

3. P. Howard and C. Potter., Bloor research: Data migration in the global 2000 - research, forecasts andsurveyresults

4. Andre Calil and Ronaldo dos Santos Mello, Simplesql: a relational layer for simpledb In Advances in Databases and Information Systems, Springer,2012.

5. SadalageP.JandFowler.M,2013,NoSQLDistilled, Pearson,p.99-109

6. J. Henrard , M. Hick, P. Thiran , and J. Hainaut Strategies for data engineering. In Reverse Engineering, 2002. Proceedings. Ninth Working Conferenceonpages211220.IEEE,2002

7. M.A.JeusfeldandU.A.Johnen.Anexecutablemetamodel for re-engineering database schemas. Springer,1994.

8. J.H.JahnkeandJ.Wadsack.Varlet:Human-centered toolsupportfordatabasereengineering.InProc.of WorkshoponSoftware-Reengineering,1999.

9. A. Maatuk, A. Ali , and N. Rossiter . Relational databasemigration:Aperspective.InDatabaseand Expert Systems Applications, pages 676683. Springer,2008.

10. K. Haller. Towards the industrialization of data migration: Concepts and patterns for standard software implementation projects. In Advanced Information Systems Engineering, pages 6378. Springer,2009.

11. B. Bordbar, D. Draheim, M. Horn, I. Schulz, and G. Weber. Integrated model-based software development, data access, and data migration. In ModelDrivenEngineeringLanguagesandSystems, pages382396.Springer,2005

12. Amazon. Amazon DynamoDB. “http://docs.aws.amazon.com/amazondynamodb/l atest/developerguide/”.Retrieved on November 2014.

13. Apache Cassandra. http://docs.datastax.com/en/cassandra/2.0/cassan dra/gettingStartedCassandraIntrohtml.Retrieved onDecember2014.

14. Neo4j graph database. http://neo4j.com/developer/get-started/. RetrievedonDecember2014.

15. MongoDB. http://docs.mongodb.org/manual/. RetrievedonJune2014.

16. A.ThakarandA.Szalay.Migratinga(large)science database to the cloud. In Proceedings of the 19th ACM International Symposium on HighPerformance Distributed Computing, HPDC 10, pages430434,NewYork,NY,USA,2010.ACM.