A Review on the Determinants of a suitable Chatbot Framework- Empirical evidence from implementation

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

A Review on the Determinants of a suitable Chatbot FrameworkEmpirical evidence from implementation of RASA and IBM Watson Assistant Frameworks

1 Professor, Department of CSE, G.Narayanamma Institute of Technology and Science, Hyderabad, India

2Student, Department of CSE, G.Narayanamma Institute of Technology and Science, Hyderabad, India ***

Abstract - The rapid emergence and evolution of AI chatbots has been phenomenal. There are countless frameworks out there that are trying to catch up to each other in order to be the best. From modest start-ups to significant partnerships, these conversational professionals are utilized in a variety of industries. On the market, there are a variety of code-based and interfacebased chatbot development solutions. However, they lack the adaptability and agility required to create sincere conversations. Chatbots are currently developed utilizing rule-based techniques, rudimentary machine learning algorithms, or retrieval-based techniques, however the results are not adequate. It can be difficult to decide which one is most suited to your requirements. The purpose of this paper is to look into the factors that influence the choice of a chatbot platform between RASA and IBM Watson Assistant. This paper presents a survey of these frameworks for researchers in identifying the areas of development and methodology. This study offers a critical examination of these frameworks, with current tactics thoroughly examined and analyzed. 30 publications from well-known digital databases were analyzed using a systematic review approach. In this paper, an extensive comparative analysis is carried out using evaluation models for chatbot performance. This survey concludes with curiosity to know why would we prefer one over the other and what are the future aspects of each. The data is collected from several resources including 50 respondents from 2 MNC’s dealing with chatbot providing services.

Words: Chatbots, RASA, Machine learning, Deep Learning, IBM Watson Assistant

1.INTRODUCTION

A chatbot is an artificial intelligence software. It can communicate with a customer in natural language via informative applications,websites,andavarietyofapplications.Itcanhaveasimulatedinteractionwiththeuserinsuchawaythatthey don’tfeelliketheyaretalkingtothemachinedirectly.Theyaredesignedtohelporganizationsmaintaintrackoftheirclient interactions.It'subiquitousonpopularchatappslikeFacebookMessenger,Telegram,RocketChat,andGoogleHangoutsChat, amongothers.Despitethefactthatchatbotsappeartobearelativelynewconcept,75percentofwebcustomersusecourier stages,accordingtoresearchfromtheGlobalOnlineIndex.Itisapieceofcorrespondenceprogrammingthatmimicswrittenor voicecommunicationwithhumans.

Thechatbotestablishedinthepastmaintainsarudimentaryconversationalstreamwithcustomersintheformofasimple solicitationandresponsestream.Asresearchprogressed,chatbotshavebeenabletorecognizethecustomers'settingsandthe flowofinteractionsandrespondappropriately.AccordingtoFortuneBusinessInsights,thechatbotmarketwillreach$721 millionin2022.Thisnumbermayprojecttoreach3billiondollarsbytheendofthedecade,basedonitscurrentcompound annualgrowthrate(CAGR)ofroughly22%.Smallerfirmsarecurrentlyusingchatbotsinlargenumbers.Addingathird-party customercarebotpoweredbyoneofthepopularchatbotbuildersisfairlysimple.Largercompanies,ontheotherhand,tend totakeamorestrategicapproach.Thispushesthemtocreatetheirownin-housesolution,whichprolongsthedevelopment process.Conversationalbots,accordingto61percentofexecutives,booststaffproductivitybyautomaticallyfollowingupon scheduled tasks. (According to Accenture, 2018). Chatbots are expected to provide consumers with 24-hour service (64 percent)andrapidresponses(55percent).(2018,Drift).Chatbotsorcomparabletechnologywillautomate29%ofcustomer serviceactivitiesintheUnitedStates.(Tableau).DuringtheCOVID-19epidemic,AI-poweredchatbotsplayedacrucialrolein handlingpatientdemands.TheWorldHealthOrganizationestimatesthat4.2billionpeoplemightpotentiallybereachedbythe WHOHealthAlertMessengerAppandotherrelatedcommunicationchannels.2020(WorldHealthOrganization)

1.1 Chatbot Usage and Engagement Statistics

AI advancements enhance chatbots’ ability to mimic human agents in conversation Contrasting with human-human conversations,human-chatbotcommunicationisdistinguishedbynoticeablevariancesinbothcontentandquality.Ahuman–

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

chatbot conversation can last a long time. It's common for people to speak in brief sentences with few words or even in poorlanguage(Hill,RandolphFord,&Farreras,2015).Whilechatbotsarelesscapableoflinguisticinterpretationthanhumans, thefundamentaldifferencebetweenthemandhumansishowtheydetectempathy.However,breakthroughshavebeenmade, andchatbotsarebecomingincreasinglysensitivetotheirusers'emotions.(2018)(Fernandes) Artificialintelligenceandother technologiesarebeingusedbybusinessestohelpthemmakequicker,moreinformedchoices,increaseefficiency,andprovide more customized and relevant experiences for both consumers and staff Conversational bots, like Apple's Siri, Google Assistant, and Amazon's Alexa, are intelligent virtual assistants that enable third parties to create "skills" or original conversationalinteractionsbyutilizingtheAI,NLP,andMLAPIs/servicesofferedbytheseplatformproviders.Theconceptof utilizinghumanlanguagetocommunicatewithmachinesemergedintheearly1950s.However,atthetime,noonecould fathomcomputersthatcouldreactoroperatelikehumans.Thelastfewdecades,however,haveseenasignificantchangeinthe situation.EventhoughpeoplestillhaveirrationalexpectationsaboutAI,itcanbesaidthatwehaveadvancedinourabilityto interact with robots. In a number of industries, including healthcare, business, education, and finance, AI technology is currentlyusedtoprovidevirtualassistance(Tidio).With1.4billionpeopleregularlyutilizingchatbots,itsusehasspreadtoa widerangeofindustries(Acquire).Thefactthatchatbotscanrespondtothemajorityofquestionsthatcustomersmayask themisoneofthefactorscontributingtothistechnology'sincreasingpopularity,perdatafromthechatbotindustry.Itis neverthelessvitaltohavesomeknowledgeablecustomersupportpersonnelonstandbyformorecomplexinquiries.However, achatbotservicereducescostsandimprovesresponsetimesforcommonissues.Thisallowscustomerserviceworkerstofocus onhardertasksandgetabiggerpicture(IBM).

1.2 Technologies and Conversational Bot Frameworks

Tocreateachatbotbasedonnaturallanguageunderstanding,avarietyoftechnologiesandplatformsareavailable(NLU) Aroundthesametimethattheideaofachatbotbecamepopular,AlanTuringputuptheTuringTest("Cancomputersthink?") (Turing,2009,pp.23–65).Inordertoserveasapsychoanalystandrespondtocustomerquestions,Eliza,thefirstknown chatbot,wascreatedin1966.(Weizenbaum,1966).Itusedpatternmatchingalgorithmsandatemplate-basedresponsesystem to react to the user's query (Brandtzaeg & Flstad, 2017, pp. 377–392). A Chatbot dubbed PARRY was created in 1972 in additiontoELIZA.(Colbyandcolleagues,1971). Aprize-winningchatbotnamedALICEwascreatedin1995.Theannual TuringTestaward,theLoebnerPrize,wasconferredtoit.Itwasthefirstchatbotthatwaswidelyrecognizedasa"human Computer". (Wallace,2009,pp.181–210).Itusedpattern-matchingandArtificialIntelligenceMarkupLanguage(AIML)to defineitscoreoperations(Mariettoetal.,2013).CurrentChatbotsdevelopedastechnologyadvancedincludeSmarterChild (Moln'ar&Szuts,2018),Siri,AmazonAlexia,IBMWatson,Cortana,andGoogleAssistant(Reisetal.,2018).Thedevelopmentof chatbotshassignificantlyincreasedsince2016,leadingtothecreationofawiderangeofconversationalsystemsforusein industry.TheScopusfindingsonChatbotdevelopmenthistoryareshowninFig.1,whichwasmodifiedfrom(Adamopoulou& Moussiades,2020).

Chart -1- Shows the timeline when people searched Scopus using the phrases "chatbot," "conversation agent," or "conversationalinterface"(Adamopoulou&Moussiades,2020).(CWOkonkwoandA.Ade-Ibijola)

Chatbotsmanagefictitiousinteractionsandlackmoralsandindependence(Murtarellietal.,2021).Athoroughinvestigationof thevariousChatbotplatforms,aswellasthelevelofinnovationandapplicationofalready-existingChatbots,isrecommended byAdamopoulouandMoussiades(2020) Hencethestudyaimstocriticallyanalyzetwobigchatbotframeworks-RASAand IBMWatson.WiththeendlesschatbotandAIhelpersonthemarket,itistoughtodeterminewhichisbetterandcompatible with the platform on which it will be implemented. Although many reviews have been done on the design, trends and applicationsofthechatbots,theirprimaryfocushasbeentoaddtothebodyofknowledgeonthevariouschatbotsdesignsand

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

itsapproaches.Thisresearch,ontheotherhand,proposestobefocusedontwoframeworks,showingtheon-the-groundreality oftheirimplementationspointingresearchersintopotentialfieldsoffutureresearch.

Thisstudycontributestotheexistingbodyofknowledgeonchatbotframeworksinthefollowingways(specifictoRASAand IBMWatson).



Itoffersstructuredandcurrentinformationaboutpriorresearchandtheirapplicationareas.  ItaddressestheprimaryobstaclesrelatedwiththedeploymentofChatbotsystems.

 ItoutlinesthekeyissueswiththeusageofChatbotplatformandhelpintheidentificationofcriticalareasthatrequire enhancement.

Giventhedepthandbreadthofpriorresearchonchatbotplatforms,thefollowingresearchquestionshavebeenaddressed usingasystematicliteraturereview(SLR)(RQ).

RQ1 -Whatisthecurrentresearchstatusorprofileofthetwochatbotframeworks?

RQ2 –Whataretheirmainadvantages?

RQ3 -Accordingtotheliterature,whatarethehurdlesobservedduringimplementation

Theremainderofthepaperisstructuredasfollows:ThetwoChatbotframeworksareexplainedinSection2,theresearch methodologiesarecoveredinSection3,andtheresultsofthesearcharediscussedinSection4.Section3alsocoversthe resultsofthestudy'sramifications.Section4ofthereportconcludeswithsuggestionsforadditionalresearch.

2. Overview of RASA and

IBM

Watson ChatBot Platform

Fig -1:GeneralArchitectureofaChatBot

Theillustration(Fig1)abovedepictswhathappenswithinachatbot.Despitethefactthatpracticallyeverychatbothasthese features,theycanbecategorizedintovariousdistinctcategories.

Achatbotcanbecategorizedasapersonalizedbot,acustomersupportbot,orafunctionalbot.Eachofthesecategorybotscan respondtooneoftwodeeplearningmodelsthatcanbeusedtodecidethechatbot'sdesignstructure.Thefirstoneisthe Generativemodel.GenerativemodelsareintelligentBotsthatareusedseldombutaremeanttocreatecomplicatedalgorithms. Thesebotsconverseinahuman-likemanner.Figure2showsanexampleofMicrosoftTay(Deshpandeetal.,2017)

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

Fig 2:Generativemodel

The Retrieval-based Model is the second. These are simple to construct utilizing pre-built APIs and established user conversationalcontext.(SeeIllustration4)(Deshpandeandcolleagues,2017).Thismodelissimpletoimplement.Itsdrawback isthattheuser'sinquirymaynotalwaysfitintotheexistingquestionandanswerdatabase.

Fig 3:Retrieval-basedModel

OneofthemostpopularArtificialIntelligencechatbotplatformsusedbydevelopersisWatson,whichusesaretrieval-based paradigm.Itoffersavarietyoftoolsforcreatingbots.Intermsofcapability,integration,andscalability,IBMWatsonholdsits ownagainstothersignificantrivalsinthehuntforanAIchatbotframework.Italsoincludesmachinelearningcapabilities.To developsophisticatedAIchatbotsforinternalusage,largeorganizationsareusingIBMWatson.Ithasconsiderablyenhanced reasoning,patternrecognition,computationallinguistics,andartificialintelligence.Additionally,itoffersdevelopersenhanced cognitivecapacities.Morethan10differentlanguagesarecurrently supportedbyWatson,anditalsohasabuilt-intranslator. Itcontainsatoneanalyzerthatcanhelpwithinterpretingandidentifyingpositiveandnegativeresponsesfromusersand customers.ItsTechnologystackandapproachprettymuchincludesNLP,IBM’sDeep-QAsoftwareandApacheUIMA.In2011, IBMunveiledWatsonasachatbot(WatsonAssistant|IBMCloud,2020).Onthegameshow"Jeopardy,"whereparticipantshad toguessthequestionsthatassociatedwithanswerstheyweregiven,Watsonwasabletounderstandrealhumanlanguagewell enoughtoovercometwopreviouschampions.Yearslater,Watsonaidedcompaniesincreatingbettervirtualassistants.Watson Healthwasdevelopedtohelpmedicalprofessionalswithdiseasediagnosis(EleniAdamopoulou&LefterisMoussiades.,2020).

Ontheotherhand,RASAisanopen-sourceimplementationoftheDualIntentandEntityTransformer(DIET)paradigmfor naturallanguageprocessing(NLP),implementstheDIETmodel.Inordertoboostefficiency,DIETemploysasequencemodel thatconsiderswordorder.Itisalsoavailableinasmaller,morecompactformwithaplug-and-play,modulardesign.For instance,DIETcanbeusedforbothintentcategorizationandentityextraction,oritcanbeusedforaparticulartask,suchas entityextractionalone.Thiscanbedonebydisablingintentcategorization.BeforeDIET,Rasa'sNLUpipelineutilizedabagof wordsmodelwithjustonefeaturevectorforeachusercommunication.Currently,RASAconsistsoftwomodules:RasaCore andRasaNLU.RasaNLUanalysesuserinput,categorizesintent,andextractsentities.Rasacoreacceptstheuser'sinputand usesseveralpipelinestoprovidearesponse.Rasaisapowerfulandtime-savingtoolforcreatingcomplicatedchatbotsthat worksrightoutofthebox.Intermsofdevelopment,itisclearandadaptable(Indiaai).Rasaisachatbotstructuretoconsider formoreaggressiveroles,withanupgradedNLUmotorcapableofmanipulatingimportantAIassociationsviatextorspeech. Rasaisalsofreeandopensource,unlikemanyotherbotsystems. It'squiterobustandiscommonlyusedbydesignerstocreate chatbotsandcontext-awarecoworkers.Insidethechatbot,onecandesign,interact,andexecute,makingitandthedataitlinks

2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal |

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

safer.Furthermore,itgivesbettercontrolandfreedomwhenrunningthechatbot.Intermsofcomponents,combinations,and generaladaptability,itrankswiththeothermajorchatbots,suchasAmazonLex,DialogStream,andMicrosoftBotFramework.

3. Review Of Literature

Designingachatbot'sobjectives,procedures,anduserrequirementsisthefirststepincreatingone.Thechatbotfunctionalities are evaluated locally after being built using a programming language or a chatbot application framework. The chatbot is subsequentlymadeaccessibletothepubliconawebsiteorinadatacenter,anditislinkedtooneormorechannelstosendand receivemessages.(EleniAdamopoulou&LefterisMoussiades.,2020).Theservicesachatbotshouldprovidetousersandthe categoryitfallsunderdictatethealgorithmsorplatformsutilizedtobuildthem(Nimavat&Champaneria,2017).Thebenefits of making the proper decision include connectivity, efficiency, quick and easy production tweaks, and less labor for the designer. It's important to remember that a chatbot is considered more effective if a user may use it directly without downloadingandinstallinganything.ComputerlanguageslikePythonandJava,aswellasapaidorfreechatbot creation platform,canbeusedtocreateachatbot(Nayyar,2019).RASA(Rasa,2019),Botkit:BuildingBlocksforBuildingBots(2019), andChatterbotareall open-sourceplatforms.SomeNLUcloudplatforms(Braunetal.,2017)fueledbymachinelearning includeGoogleDialogFlow(Dialogflow,2019),Facebookwit.ai(Wit.ai.,2019),MicrosoftLUIS(LUIS(LanguageUnderstanding) -CognitiveServices-MicrosoftAzure,2019),andIBMWatsonConversation(IBMWatsonConversation,2019).

TheRasaNLUandcorewerefirstpresentedunderanopen-sourcelicensebyBocklisch,T.etal.inapaper.Theobjectiveofthis studywastocreateacommunicationsystemfornon-experttechnologyenthusiastsbasedonmachinelearningandlanguage understanding.Thebundletheycreatedwassmallandincludedeverythingtheyneededtoprogress.244versionsofRasahad beenproducedwithatotalof18023contributions,thankstotheeffortsof344contributors Rasa'sAPIintegratesprinciples fromscikit-learn[1](consistentAPIswithvariousbackendimplementations)andKeras[2],andbothoftheselibrariesare (optional)componentsofaRasaapplication.ThetechniquetodialoguemanagementusedbyRasaCoreismostsimilarto[3], however it differs from the majority of previous research systems. End-to-end learning, as in [4] or [5,] in which natural languageunderstanding,statemonitoring,dialogmanagement,andanswerproductionarealllearnedcollaborativelyfrom discussiontranscripts,isnotcurrentlyenabled.Rasa'slanguageunderstandinganddialoguemanagementareentirelydistinct RasaNLUandCorecanbeusedseparately,andlearnedconversationmodelscanbeappliedtodifferentlanguages.Rasa's architectureisintendedtobemodular.Thismakesintegratingwithothersystemsstraightforward.Forinstance,RasaCorecan beusedasadialoguemanagerinconjunctionwithNLUservicesotherthanRasaNLU.Despitethefactthatthecodewasbuiltin Python,bothservicesmayofferHTTPAPIs,makingitpossibleforprogramswritteninotherprogramminglanguagestoeasily accessthem.InanexperimentbyAnranJiao[6],theRASANLUapproachoutperformstheNNmethodintermsofaccuracy. Additionally,theRASANLUmethodexcelsatextractingallentitieswhileanalyzingasinglewordasawhole.However,itwas discoveredthattheNNapproachhassuperiorfidelitywhenclassifyingthingsfromsegmentedwords.HeintegratedtheRASA NLUwithneuralnetworktechniquestocreateanentityextractionsystemthatcanrecognizeintentionsandtheentitiesthatgo alongwiththem.HealsocreatedapracticalframeworktoputtheRASANLUnotionintopractice.Lacerda[7]usedthecoreof RasainhisworkandproposedanewsoftwarestackcalledRasa-ptbr-boilerplatefornon-specialistswhodon'tknowmuch abouttheinternalsofthechatbotand treat chatbotslikeablackbox.

Tobefair,theexperiencesofthetopthreechatbotplatforms:Amazon,Google,andIBMareallextremelysimilar.Rasawasthe onlychatbotplatformthatnecessitates hands-onPythonscripting.AlthoughRasa'sdocumentationseemsthemostfun,IBM Watson’s appeared to offer the most comprehensive and easy-to-follow resources. The IBM RedguideTM article (2012) illustrates how Watson employs dynamic learning, natural language processing, hypothesis generation, and hypothesis assessmenttodeliverprompt,accurateresponses.Itrepresentsacognitivesysteminaction.Withaccuracycomparabletothat ofahuman,itcanparsehumanlanguagetoidentifyconnectionsbetweentextpartsatratesandscalesthataremuchfasterand greaterthanthosethatahumanalonecouldachieve.Itcanhandleahighlevelofaccuracywhenitcomestoknowingthe appropriate response to a question. Its pricing starts with the Lite edition, which is cost-free and supports up to 10,000 messagespermonth.Theotherfee-basedprogrammesareStandard,Plus,Premium,andDeployAnywhere.Standardoffersan infinitenumberoftextsfor$0.0025each AlthoughthepriceofthePlusplanisnotdisclosed,onecangetafree30-daytrialby contactingIBM.ThecostsofthePremiumandDeployanywhereprogrammesvary.

3.1 Chatbot Performance Evaluation Models

Evaluationofchatbotperformancecanbedoneinavarietyofways.Personalassistants,question-answerbots,anddomainspecificbotsaresomeofthedifferentinformationretrieval(IR)usesforchatbots.Theaccuracy,precision,recall,andF-scoreof thechatbot'sresponsesmustbemeasuredbytheevaluatorsaftertheyhaveaskedthechatbotquestionsandmaderequests (Cahn,2017).Accordingtotheuserexperienceperspective,thebot'sgoalisprobablytoincreaseusersatisfaction.Usersneed tobesurveyed(oftenviaquestionnairesonwebsiteslikeAmazonMechanicalTurk),andbotsneedtoberatedbasedontheir

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

usability and pleasure. To get close to speech, bots should be put to the test by linguists for their ability to put together complete,grammatical,andmeaningfulutterances.Thebotthatappearsthemostconvincinglyhuman thatis,onethatbest passestheTuringTest isalsothemostefficientfromtheperspectiveofartificialintelligence.

Therearemanymeasuresthatcanbeusedtodeterminewhetherachatbotwillbeeffective,suchastheBleuScore, which comparesacreatedwordsequencetoareferencesequence.TheBLEUscorewasfirstdevelopedspecificallyfortranslation assignmentswhenitwasfirstintroducedbyKishorePapineniin2002.TheBLEUscoreisdeterminedbycountingtheinstances inwhichn-gramsofusertextcoincidewithn-gramsofreferencetext.Thechatbot'sintelligenceincreaseswithitsBLEUscore b)TheTuringtestisaprominentwayofevaluatingamachine'scapacitytodemonstrateintelligentbehavioursimilartothatof ahuman.TheTuringtestindicatesthatamachinehassucceededifthetesterisunabletodistinguishbetweenhumanand computerresponses.c)Scalability:Ifachatbotacceptsalargenumberofusersandnewmodules,itissaidtobemorescalable. d)Interoperability,whichreferstoasystem'scapacitytoshareandutilizedata.Multiplechannelsshouldbesupportedbyan interoperable chatbot, and users should be able to move between channels rapidly. e) Speed: When it comes to speed, a chatbot'sresponseratemeasurementiscrucial.Chatbotsofhighqualityshouldbeabletorespondfast(PoojaGambhir,2019).

InrelationtoIBMWatsonAssistant,itcanassistinovercomingthehighlearningcurveandannoyinglanguageemployedby rivalvirtualagenttechnologies.Itschatbotdesigniseasyanddoesnotrequiretheneedforsophisticateddecisiontreesor scripting.WatsonAssistantisfoundtobeveryeasytouseandveryscalable.TheCEOofAdMed,JoanFrancystatesthatthe interfaceallowsanybodytobuildachatbot,whilealsoallowingourdeveloperstofullyutilizeWatson'scapabilities(IBM,2019). However,buildingasimplequestion-answerbotisveryeasybutestablishingacommunicationwithanexternalAPIcanbea complexprocess.Accordingtoafewrespondents,thelistofavailableintegrationsforthechatbotrevealsitsmajorflaw:itcan't beusedanyplace!Slackisanewadditiontotheiroffering,howeverMicrosoftTeamsisnotsupported(whichhasover115 millionusers).Rasa'sapproachtodialoguemanagementcannotbecomparedtoIBM's.Thedialoguemanagementenvironment inWAishighlystrict.Thedialoguehaselementssuchasdisambiguation,digression,numerousconditionedreplies,andalarge amountofbusinesslogic,resultinginacomplicatedenvironment.TheLiteplan'smetricsarenotablylightonspecifics,making itdifficulttoevennoticewherethechatbot'srecognitionislacking.Inordertomakethechatbotuser-friendlyanduseful,one mustfirstidentifytheshortcomings.IBMhasasolutionforit,whichiscalledpayforPluswhichcouldgodowntheexpensive roadespeciallyforsmallbusinesses.Furthermore,ifthechatbotrequiresanythird-partyAPIconnectors,onemustjoinupfor IBM'sCloudFunctionsservice,which,asidefromtime,costsalotmoremoney.Nonetheless,IBMWatsonisamarket-leadingAI solutionthatisacclaimedforhelpingbusinessesconductquickandin-depthresearch.Theycanfindpatternsandinsights thankstoitspowerfulAIandmachinelearningcapabilities

.Throughextensivestudiesofcomplicateddata,meaningfulandactionableinsightscanbegleaned.Itmaysoundmundanein today'sbot-buildingframeworks,butRasahasclaimedtobedoingsomethingalittledifferent.AccordingtoRasaTechnologies, thearchitectureenablesthecreationofcontextualchatbots trueAIassistantsthatdomorethanrepeatFAQreplies.Rasais morelikelytobeusedformoreambitiousapplications,suchasbotsthatcancomprehendandrespondtohighlysophisticated statements.Thatbeingsaid,theentrancehurdleisminimal.Onecanbeginusingtheframeworkforfree,asthereisminimalrisk inbrowsingtheRasadocumentationtoacquireanunderstandingforit.Accordingtothestudy[8],RASAismoreadaptablethan otherbusinesssoftwaresduetoitsscalabilityandopen-sourcelicensing.JollitychatbotbuiltinRasaintegratedwithTelegram helpsuserscopewithdepressionbygivinganunseenfriendonwhomtheycanrely,aswellastheabilitytointeractwiththe botthroughouttheday.Variouscriteria,includingintentaccuracy,narrativecorrectness,andconfusionmatrix,wereusedto evaluatethesystem.Experimentsshowedthatthesystemhada90%accuracyrateforintentidentificationandcouldretrieve pertinentresponses(KanakamedalaDeepika,VeerankiTilekya&JatrothMamatha,2020).Accordingtopeerspot,Rasaisplaced fourthinChatbotDevelopmentPlatforms,whereasIBMWatsonAssistantisrankedsecond. However,themajorityofones selectionwillbedependentontheirpreviousexperienceandwhattheywanttoaccomplishwiththebot.Themostsignificant differencebetweentheframeworksisthedevelopmentecosystemandassistancethattheybestenable.Thus,consideringthe advantagesanddisadvantagescanhelpinmotivatingusersandestablishingacceptablestandardsthatwillencouragethe growthandapplicationofchatbottechnology.

4. Discussion Of Results

ThepurposeofthisstudywastoconductasystematicreviewoftheliteratureonChatbottechnologies(RasaandIBMWatson Assistant)inordertogainabetterunderstandingoftheircurrentstatus,benefits,obstacles,andfuturepossibilities.Three majorresearchtopicswereidentifiedinrelationtotheobjectives.

RQ1investigatedthecurrentstateandprofileofboththeframeworks. Atotalof30publishedresearchpublicationswere analyzedinordertoanswerthisquestion. Thisresearch alsotookintoaccountavarietyofverifiedChatbotDevelopment Platform evaluations in order to uncover the real-world challenges that developers experience during implementation.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072

Accordingtoourfindings,ItisfartooearlytodetermineintheperspectiveofcomputinghistoryifIBMWatsonAssistantis primitive.Inanewlyreleasedbenchmark (IBM,2020),WatsonAssistant'snewandenhancedintentrecognitionalgorithm outperformedcommercialandopen-sourcecompetitors.Whentrainedonalltrainingsetsandtestedonin-scopeoccurrences, theupgradedversionofWatsonAssistantachievesanaverageaccuracyof73.8percentacrossalldatasets.AccordingtoJio HaptikTechnologies(2020),WatsonAssistantoutperformsRasaby9.3percentagepoints.Accordingtothestudy,Watson AssistantcompeteswithpretrainedLMsacrossawiderangeofdatasetsand situations,buttrainsfarfaster-animportant componentintheusabilityofacommercialconversationalAIservice.Rasa,ontheotherhand,allowsopen-sourcemodelstobe addedinthepipeline,suchasTransformer-based(Vaswanietal.,2017)models Rasa-basedchatbotshaveoutperformedanyof theotheropen-sourcecompetitors

RQ2outlinesthebenefitsoftheiruseintheindustry.Thereviewhasidentifiedandhighlightednumerousadvantagesgained fromtheuseoftheseChatbotframeworks.SomeoftheadvantagesofWatsonAssistantincludetheeaseofuse,doesnotrequire anycodingknowledge.Ahighcompletionandcontainmentrateisachievedthroughefficientandtargetedencountersthat includerichmedia.ItisdesignedforworldwidedeploymentandisbuiltwithIBMsecurity,scalability,andflexibility.RASA, however,fallsintotheopen-sourcecategory,whichallowsforabalanceofcontroloverdevelopmentprocesses. Ittakesa flexibleapproachtoChatbotdevelopmentinthattheNLUpipelinemaybetailoredtotheproblemthattheCAwilladdress. RQ3 identifies some of the primary issues that developers confront while implementing these frameworks. Various publicationsverifiedchatbotplatformevaluationwebsites,andresponsesfromrespondentswereusedinthisresearchto highlightsomeoftheprimarychallengesoftheseframeworks.

5. Conclusion

Chatbotscanaidintheimplementationofmajorchanges.Customers'interactionswithbusinessesareaffectedbythem.They influencehowclientsupportishandled.Theyinfluencehowleadsareproducedandhowsoonclientsarehelped.Chatbotsare oneofthemosthuman-likewaysforaproducttocommunicatewithcustomersandanswertheirinquiries.Wehaveanalysed theperformanceoftwoofthemostpopularcommercialservicesfordevelopingtask-orienteddialoguesystems.Inpractise, systemsdesignedfordesigninganddeployingvirtualassistantsmustaddressavarietyofscenariosandtrade-offs.These systemsmusttrainthebestmodelsinfew-shotscenarios,strikeabalancebetweentrainingtimeandaccuracy,andreadily adapttoawiderangeofdomains.InthisreviewithascometonoticethatWatsonAssistanthasoutperformedRasaatmany instanceshowevertheanswertowhichframeworktopreferliesintherequirementforit.Thestatedbenefitsandconstraints canbeexperimentallystudiedinfutureworkstoseehowtheyinfluenceChatbotdevelopment.

REFERENCES

[1] F.Pedregosa,G.Varoquaux,A.Gramfort,V.Michel,B.Thirion,O.Grisel,M.Blondel,P.Prettenhofer,R.Weiss,V.Dubourg,et al.Scikit-learn:Machinelearninginpython.JournalofMachineLearningResearch,12(Oct):2825–2830,2011.

[2] F.Cholletetal.Keras,2015.

[3] J. D. Williams, K. Asadi, and G. Zweig. Hybrid code networks: practical and efficient end-to-end dialog control with supervisedandreinforcementlearning.arXivpreprintarXiv:1702.03274,2017.

[4] T.-H.Wen,D.Vandyke,N.Mrksic,M.Gasic,L.M.Rojas-Barahona,P.-H.Su,S.Ultes,andS.Young.Anetwork-basedend-toendtrainabletask-orienteddialoguesystem.arXivpreprintarXiv:1604.04562,2016.

[5] A.BordesandJ.Weston.Learningend-to-endgoal-orienteddialog.CoRR,abs/1605.07683,2016.

[6] AnranJiao,AnIntelligentChatbotSystemBasedonEntityExtractionUsingRASANLUandNeuralNetwork.Journalof PhysicsConferenceSeries1487(1):012014,2020

[7] Lacerda,A.R.T.D.(2019).Rasa-ptbr-boilerplate:FLOSSprojectthatenablesbrazilianportuguesechatbotdevelopmentby non-experts.

[8] Singh, A., Ramasubramanian, K., &Shivam, S. (2019). Introduction to Microsoft Bot, RASA, and Google Dialogflow. In BuildinganEnterpriseChatbot(pp.281-302).Apress,Berkeley,CA.

[9] Bocklisch,T.,Faulkner,J.,Pawlowski,N.,&Nichol,A.(2017).Rasa:Open-sourcelanguageunderstandinganddialogue management.arXivpreprintarXiv:1712.05181.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

[10] IBM RedguideTM article -The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Workshttps://www.redbooks.ibm.com/redpapers/pdfs/redp4955.pdf

[11] Cahn,B.J. (2017). [Thesis] Chatbot Literature Review. Retrieved from https://static1.squarespace.com/static/569293741c1210fdda37b429/t/59160b6bff7c50104e601a85/1494616940469/ CHATBOT_thesis_final.pdf

[12] PoojaGambhir(2019)-ReviewofChatbotDesignsandTrends,Conference:ArtificialIntelligenceandSpeechTechnology 2019,IGDTUW

[13] HenryAmm-https://www.adenin.com/blog/common-problems-building-ibm-watson-assistant-chatbot/

[14] Nayyar,2019-ChatbotsandtheOpen-SourceToolsYouCanUsetoDevelopThem

[15] Rasa2019:open-sourceconversationalA.I,Rasawebsite:https://rasa.com/.(Retrieved18November2019)

[16] Botkit:Buildingblocksforbuildingbots,https://botkit.ai/ (Retrieved25November2019)

[17] Wit.ai.,https://wit.ai/.(Retrieved25November2019)

[18] KanakamedalaDeepika,VeerankiTilekya,JatrothMamatha,2020-JollityChatbot-AcontextualAIAssistant,CFP20P17ART;ISBN:978-1-7281-5821-1

[19] IBM,2020- Watson Assistant improves intent detection accuracy, leads against AI vendors cited in published studyWatsonBlog(ibm.com)

[20] JioHaptikTechnologies(2020),2012.03929.pdf(arxiv.org)

[21] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.2017.Attentionisallyouneed.InAdvancesinNeuralInformationProcessingSystems(NIPS),pages5998–6008,LongBeach,CA.

[22] RakeshKumarSharma&ManojJoshi(2020),AnAnalyticalStudyandReviewofopenSourceChatbotframework,RASA, http://www.ijert.orgISSN:2278-0181IJERTV9IS060723(ThisworkislicensedunderaCreativeCommonsAttribution4.0 InternationalLicense.)

[23] Drift,2018-Theultimateguidetochatbots,Chatbots-TheBeginnersGuidetoChatbotTechnology|Drift

[24] JenniferHill,W.RandolphFordandIngridG.Farreras-Realconversationswithartificialintelligence:Acomparisonbetween human–human online conversations and human–chatbot conversations, Computers in Human Behavior, Volume 49,August2015,Pages245-250

[25] EleniAdamopoulou&LefterisMoussiades-AnOverviewofChatbotTechnology,ArtificialIntelligenceApplicationsand Innovations,Published 6May2020

[26] Deshpande,AlishaShahane,DarshanaGadre,MrunmayiDeshpande,PrachiM.Joshi,2017-ASurveyofvariousChatbot implementationTechniques,ComputerScience

[27] Ketakee Nimavat, Tushar Champaneria- Chatbots: An overview. Types, Architecture, Tools and Future Possibilities, October2017,Conference:InternationalJournalforScientificResearch&Development,At:Ahmedabad

[28] VaswaniA,ShazeerN,ParmarN,UszkoreitJ,JonesL,GomezAN,KaiserŁ,PolosukhinI(2017)Attentionisallyouneed.In: Advancesinneuralinformationprocessingsystems,pp5998–6008

Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page365