International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 03 | Mar 2025
p-ISSN: 2395-0072
www.irjet.net
Optimizing Sentiment Analysis: A Scalable Approach with Logistic Regression and NLTK Daulat Tikhole1, Hittanshi Tikle2, Mehek Uikey3 , Prof. M. A. Chimanna4 1-3 UG Students, Department of Electronics & Computer Engineering,
Pune Institute Of Computer Technology, SPPU, Pune, Maharashtra, India. 4 Professor at Pune Institute Of Computer Technology, SPPU, Pune, Maharashtra, India.
---------------------------------------------------------------------***--------------------------------------------------------------------have significantly improved sentiment prediction by Abstract –
leveraging statistical patterns in textual data. However, some models, particularly deep learning-based approaches, suffer from interpretability challenges, making them difficult to deploy in real-world decision-making systems.
The exponential growth of digital content, analyzing sentiments expressed in text has become essential for businesses and researchers. Sentiment analysis, a subfield of Natural Language Processing (NLP), is widely used to determine whether a piece of text conveys a positive, negative, or neutral sentiment. This research focuses on sentiment classification of IMDB movie reviews using machine learning. A structured pipeline is developed, covering data preprocessing, exploratory data analysis, feature engineering, model selection, evaluation, and deployment. Various machine learning models, including logistic regression, decision trees, random forests, and AdaBoost, are analyzed for performance. Logistic regression with TF-IDF vectorization is identified as the most efficient and interpretable model. To enhance model transparency, explainability techniques such as SHAP and LIME are used to interpret feature importance. Finally, the trained model is deployed as a REST API using Flask, ensuring practical usability. This study provides an effective sentiment classification approach that can be applied to customer feedback analysis, social media monitoring, and automated opinion mining systems.
In this study, sentiment analysis is performed on IMDB movie reviews using machine learning. A structured methodology is adopted, involving data cleaning, feature engineering, model training, evaluation, and explainability techniques. Logistic regression with TF-IDF vectorization is selected as the primary model due to its balance between accuracy, interpretability, and computational efficiency. Explainability methods such as SHAP and LIME are used to uncover the influence of individual features, ensuring model transparency. Finally, the trained model is deployed using Flask as a REST API, demonstrating its potential for realworld applications.
2. LITERATURE SURVEY Sentiment analysis has evolved significantly over the years. Early approaches were based on lexicon-based methods, which relied on predefined word lists where words were assigned sentiment scores. While these methods were simple and interpretable, they struggled with handling negations, sarcasm, and context-dependent meanings.
Keywords: NLP, SHap, Lime, Data Preprocessing, Feature Engineering, Vectorization, TFIDF, Hyperparameter Tunning.
With the rise of machine learning, models such as Naïve Bayes, Decision Trees, and Support Vector Machines provided significant improvements in sentiment classification.
1.INTRODUCTION The digital era has transformed the way opinions and reviews are shared, leading to an explosion of usergenerated textual data. Online platforms, including movie review sites, e-commerce stores, and social media networks, serve as hubs for users to express their sentiments. Extracting valuable insights from these textual data sources is crucial for businesses to understand public perception, monitor brand reputation, and improve customer experiences.
S. S. Aasiya, S. S. Ahmed, S. S. Akram, S. S. Aslam, and S. S. Amjad et al., in their study "Sentiment Analysis using Logistic Regression Approach on E-commerce," analyze how sentiment analysis enhances user experience on e-commerce platforms like Amazon. They employ logistic regression combined with Python’s NLTK module to classify Amazon reviews into positive and negative sentiments. Their work highlights the importance of preprocessing and feature selection in improving sentiment classification accuracy, proving the effectiveness of NLP techniques in commercial applications.
Sentiment analysis, often referred to as opinion mining, enables automated identification of emotions within text. Traditional sentiment classification methods relied on manually created dictionaries of positive and negative words, but such approaches struggled with linguistic nuances, context, and negations. Machine learning models
© 2025, IRJET
|
Impact Factor value: 8.315
X. W. Qing Wu and W. Yun Xu et al., in their research paper "Sentiment Analysis of Yelp’s Ratings Based on Text Reviews," investigate Yelp customer reviews to predict sentiment
|
ISO 9001:2008 Certified Journal
|
Page 1055