Ameliorating Depression through Deep Learning Conversational Agent: A Novel Approach to Mental Health Intervention
Abstract - Depression is a prominent cause of disability worldwide, impacting millions of individuals. Despite the availabilityofmanytreatmentalternatives,manyindividuals still may not obtain proper care owing to societal stigma, financialrestraints,orashortageofmentalhealthexperts.To address these issues, we present a novel approach to depression intervention that leverages deep learning techniques to construct a conversational agent capable of alleviating depressive symptoms. Our proposed method incorporates a combination of natural language processing, sentiment analysis, and transfer learning to create a chatbot that understands the user's emotions and responds in a compassionateandsupportivemanner.Thesystemisdesigned to identify the user's cognitive distortions and provide responses that ameliorate the user’s mental circumstances to help them reframe their negative views. We assessed the system's performance through a peer-based study by implementing the chatbot on Discord, where many people, notablyyoungpeople,canutilisethebot.Thestudycomprised participants with mild to severe depression. The results demonstratethatthechatbotsubstantiallylowereddepressive symptoms and enhanced the overall mood in the majority of individuals.Theusersindicatedexcellentsatisfactionwiththe system and enjoyed the non-judgmental and empathic attitude. Overall, our study illustrates the promise of deep learning-based conversational agents as a scalable and accessible method to relieve depressive symptoms. Further researchisneededtoexplorethelong-termeffectivenessofthe system and its effects on clinical outcomes.
Key Words: DepressionAmelioration,ConversationalAgent, DeepLearning,TransferLearning,NLP,SentimentAnalysis
1.INTRODUCTION
Depressionisacommonmentalhealthillnessthatcanhave significant consequences on an individual's life. While different treatment alternatives are available, the accessibilityandeffectivenessoftheseinterventionsremain a concern, particularly for people living in distant or disadvantagedlocations.Conversationalagents,orchatbots, haveemergedasapromisingwaytoprovidementalhealth care and deliver evidence-based interventions. Deep learning-basedchatbotshaveshownspecialpromisedueto their capacity to learn from massive volumes of data and deliver more tailored and contextually appropriate
responses. Transfer learning, a technique that allows the modeltoexploitpre-existinginformationandadapttonew tasks,significantlybooststheeffectivenessofthesechatbots. Inthisresearch,weintroduceadeeplearningchatbotthat leveragestransferlearningtorelievedepressionsymptoms. Wefine-tuneapre-trainedbinaryclassificationtransformer to classify the text as normal or depressed. If the input is classifiedasdepressed,thechatbotusesanotherpre-trained multiclassclassificationtransformertoaccuratelychoosea responsefortheprovidedinput.Thechatbotleveragesthe pre-trained Blenderbot transformer pipeline to conduct informal, non-depressed conversation inputs. We analyse theefficacyofourchatbotthroughuserresearch,including interviews with persons with depression, and report significant reductions in depressive symptoms and user satisfaction.ThischatbotisdeployedonDiscord,afamiliar applicationformany,andsimulatesthesenseofconversing with a friend. Our method illustrates the promise of deep learning-based chatbots in providing accessible mental healthcaretoindividualswithdepression,particularlythose who may not have access to traditional mental health services. The use of transfer learning further boosts the chatbot's performance and may be relevant to various mentalhealthillnessesandhealthcarefields.
1.1 Transformers
1) RoBERTa: It stands for ‘Robustly Optimised BERT PretrainingApproach’andwasusedforboththebinaryand multiclass classification tasks in the chatbot. Precisely, RobertaForSequenceClassification was used, which is a RoBERTa Model transformer with a sequence classification/regressionheadontop(alinearlayerontopof thepooledoutput),e.g.,forGLUEtasks.
2)Blenderbot:Specifically,theBlenderbot-400M-distill modelwasusedforconditionalgenerationtoconductcasual conversations.Thishelpswhentheuserjustwantstohavea casualconversationandwantsnormalresponsesinsteadof therapeuticones.
1.2 Deployment Platform
Thedeploymentplatformforthechatbotwaschosentobe Discord.Notonlyisitincrediblypopularwiththeyouth,it alsohassignificantfeaturessuchasaccessibility,anonymity, 24/7availability,personalization,scalability&lowercost.
Italsoopensupa plethora ofpossibilitiessuchastext-tospeechduetoitsinbuiltfunctionsandsupportedAPIs.
2. LITERATURE REVIEW
A. Literature Review
1) "The Woebot Trial: A Randomized Controlled Trial of an Automated Conversational Agent for Depression" by Fitzpatrick et al. (2017)[1]: This study evaluated the effectiveness of an automated chatbot named Woebot in reducing symptoms of depression. The chatbot provided cognitive-behavioral therapy and was evaluated through a randomizedcontrolledtrial.Thestudyreportedsignificant improvementsindepressivesymptomsandusersatisfaction.
2) "Using Chatbots for Mental Health: A Systematic Literature Review" by Vaidyam et al. (2019)[2]: This paper providesasystematicliteraturereviewoftheuseofchatbots formentalhealthsupport.Thereviewidentified14studies that evaluated the effectiveness of chatbots in providing mental health support, particularly for depression and anxiety. The paper highlights the potential of chatbots in providingaccessible,personalized,andcost-effectivemental healthsupport.
3) "RoBERTa: A Robustly Optimized BERT Pretraining Approach"byYinhanLiuet al. (2019)[3]: Thepaperbeginsby introducing the limitations of the previous pre-training approach,BERT(BidirectionalEncoderRepresentationsfrom Transformers), and how RoBERTa overcomes these limitations by fine-tuning several hyperparameters in the pre-training process. The authors then describe the pretrainingcorpus,whichwasacombinationofbooks,articles, andwebpages,resultinginatotalof160GBoftextdata.The pre-trainingobjectivewasamaskedlanguagemodelingtask, wherethemodellearnedtopredictthemaskedwordswithin asentence.RoBERTaachievedstate-of-the-artresultsona range of NLP tasks, including natural language inference, question answering, and sentiment analysis. The authors conductedexperimentstoshowthatRoBERTaoutperformed BERT on several benchmarks, including the Stanford Question Answering Dataset (SQuAD) and the General Language Understanding Evaluation (GLUE) benchmark. Additionally,RoBERTawasshowntoberobusttovariations inthetrainingdataandhyperparameters,makingitamore reliableandflexiblemodel.
4) "Recipesforbuildinganopen-domainchatbot"byStephen Roller et al. (2020)[4]: The authors evaluate the chatbot's performanceonseveralmetrics,includingperplexity,human evaluation,andengagement.Thechatbotisshowntoachieve state-of-the-art performance on several metrics, outperformingexistingchatbotsonthePersona-Chatdataset. The paper's methodology and results demonstrate the effectiveness of the approach, which has since become a widelyusedmethodinthefield.Thepaperhasbecomeagotoresourceforresearchersandpractitioners,anditsimpact
canbeseeninthemanystate-of-the-artchatbotsdeveloped using the techniques and methodologies described in the paper.
B. Problems in Existing Systems
1) Lack of Personalization: Many mental health chatbots provideaone-size-fits-allsolutiontomentalhealthproblems. Theydonottakeintoconsiderationtheindividualvariances inpersonalities,experiences,andmentalhealthsituationsof theusers.Thiscanleadtoerroneousorineffectiveadvice.
2) Data Privacy: The use of mental health chatbots necessitatestheexchangeofsensitivepersonaldata,which might be a worry for users who are anxious about their privacy.
3) Inaccurate Responses: Mental health chatbots might delivererroneousorevenhazardousadviceiftheyarenot properlydevelopedortrained.Thiscanleadtobadresults forusers.
4) LimitedScope:Mostmentalhealthchatbotsaremeantto targetacertainsetofmentalhealthissues,suchasanxietyor depression. They may not be equipped to address more sophisticatedmentalhealthdifficulties.
5) Stigma:Despitethegrowingacceptanceofmentalhealth chatbots, there is still a stigma attached to mental health issues, which may prevent some individuals from seeking help through chatbots, making anonymity an important feature
6)Lackofavailabledata:Mentalhealthdatasetsarenotas widelyavailableasotherhealthcaredatasets,suchasthose for diabetes or heart disease, due to the sensitive and personalnatureofmentalhealthdata,whichrequiresstrict ethicalconsiderationsandprivacyprotections.Therearealso concerns about the quality and consistency of the data collected, as mental health conditions can be difficult to diagnoseandmayhavevaryingsymptomsacrossindividuals.
C Findings
1)RoBERTawasusedasitprovidesstate-of-theartresults and outperforms the other transformers on GLUE benchmark results. While it gives better results than the previous state-of-the-art transformers used for text classification, it also has a better speed-to-accuracy ratio, makingitthebestchoice.
2) Blenderbot was chosen as the model for casual conversationsasityieldedbetterresultsbothinansweringa single prompt as well as remembering context through conversationhistory.
2) Custom Reply Generation: Thisisthesecondstepifthe user input is classified as depressed. Here, the input is passed through a multiclass classification transformer to predictthetypeofmentaldistresstheuserhas,forexample, anxiety,suicidalthoughts,depression,etc.,andprovidethe correctresponse.ThemodelusedhereisagaintheRobertabasemodel.Thedatasetusedhereisamodifiedversionof theMentalHealthConversationalDataDatasetfromKaggle. Due to the extremely small size of the dataset, only the traininglosswasrecordedat98.2%.Thehyperparameters werethesameexceptfortheevaluationsteps,whichwere adjusted for the small dataset size. Since the dataset was small, traditional machine learning algorithms were also used as benchmarks. The results of traditional machine learningalgorithmswhencomparedwithRobertaareshown inTable-2.
3. METHODOLOGY
1) ClassificationTransformer: Thefirststepinthesystem’s workflow is passing the user input through the binary classification transformer to classify it as depressed or normal. The model used here is the Roberta-base model. ThismodelwasfinetunedusingtheSuicideandDepression Detection Dataset from Kaggle. The dataset has 7730 differentRedditpostsclassifiedasdepressedornormal.The trainandtestsetsweremadewitha75–25%distribution. Thetrainedmodelyieldedanaccuracyscoreof98.5%onthe testset.Thehyperparametersusedwhilefine-tuningandthe resultsareshowninTable-1.
3) Conversational Transformer: Thisisthesecondstepif the user input is classified as normal by the binary classification transformer. Here, we use Blenderbot for casualconversation.Everypairofinputandoutputisstored up to a maximum of three pairs. These pairs are passed throughtheconversationaltransformerbeforeeveryinput soastoretainthecontextoftheconversation.
5. CONCLUSION
To summarize, developing a depression amelioration chatbotthatemploysbothtransferlearningandpre-trained transformersisapotentialwayforenhancingmentalhealth results.Thischatbothastheabilitytogiveindividualswith customised and accessible help, allowing for early intervention and even preventing symptoms from deteriorating. The chatbot's use of natural language processingandmachinelearningtechnologiesenablesitto givetailoredresponsesthatadapttotheuser'sneedsover time.Furthermore,thechatbotisofferedonaneasy-to-use platform, making it more accessible than a personal therapist.Whilethechatbot'sperformanceisnotequivalent to that of a real-life therapist, more research is needed to improve the chatbot's effectiveness and user experience. These preliminary findings imply that a depression
ameliorationchatbotisa viabletoolforenhancingmental healthoutcomesandgivingpersonsinneedwithaccessible support.
ACKNOWLEDGEMENT
We wish to express our sincere gratitude to Dr. Sanjay U. Bokade, Principal, and Prof. S. P. Khachane, Head of DepartmentofComputerEngineeringatMCT'sRajivGandhi Institute of Technology, for providing us with the opportunity to work on our project, "Depression amelioration Chatbot using NLP and Deep Learning." This projectwouldnothavebeenpossiblewithouttheguidance andencouragementofourprojectguide, Prof.DilipKale,and the valuable insights of our project expert, Dr. Sharmila Gaikwad. We would also like to thank our colleagues and friendswhohelpeduscompletethisprojectsuccessfully.
REFERENCES
[1] FitzpatrickKK,DarcyA,VierhileM.DeliveringCognitive Behavior TherapytoYoung AdultsWith Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized ControlledTrial.JMIRMentHealth.2017Jun6;4(2):e19. doi: 10.2196/mental.7785. PMID: 28588005; PMCID: PMC5478797.
[2] VaidyamAN,WisniewskiH,HalamkaJD,KashavanMS, Torous JB. Chatbots and Conversational Agents in MentalHealth:AReviewofthePsychiatricLandscape. Can J Psychiatry. 2019 Jul;64(7):456-464. doi: 10.1177/0706743719828977. Epub 2019 Mar 21. PMID:30897957;PMCID:PMC6610568.
[3] YinhanLiu,MyleOtt,NamanGoyal,JingfeiDu,Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov (2019). RoBERTa: A RobustlyOptimizedBERTPretrainingApproach.ArXiv [Cs.CL].Fromhttp://arxiv.org/abs/1907.11692
[4] StephenRoller,EmilyDinan,NamanGoyal,DaJu,Mary Williamson,YinhanLiu,JingXu,MyleOtt,KurtShuster, Eric M. Smith, Y-Lan Boureau, Jason Weston (2020). Recipes for building an open-domain chatbot. ArXiv [Cs.CL].Fromhttp://arxiv.org/abs/2004.13637
[5] Khan, S. (2019, September 4). BERT, RoBERTa, DistilBERT, XLNet which one to use? Retrieved 29 March 2023, from Towards Data Science website: https://towardsdatascience.com/bert-robertadistilbert-xlnet-which-one-to-use-3d5ab82ba5f8
[6] LianaYe2, (n.d.). The future of_conver_ai[6933]. Retrieved 29 March 2023, from https://www.slideshare.net/LianaYe2/the-futureofconverai6933