International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072
AI Based Smart Data Analyst:
Mr. P. Nallusamy*1, Harsh Kumar Gupta*2, Jay Krishna*3, Prince Kumar*4, Priyanshu Kumar*5
*1 M.E., Assistant Professor, Department of Computer Science and Engineering DhanalakshmiSrinivasan Engineering College (Autonomous), P
*2,3,4,5 Students, Department of Computer Science and Engineering, Dhanalakshmi Srinivasan Engineering College (Autonomous), Perambalur
ABSTRACT - The data-driven economy requires intelligent, accessible, and intuitive data analysis systems to enhance decision-making. This study proposes a practical approach for developing a dynamic and conversational AI-powered Data Analyst Agent using cutting-edgeArtificialIntellige nce and Natural Language Processing principles. A structured modular architecture comprising intelligent components is designed to ensure high code maintainability and scalability. Key features include natural language to code translation and multi-turn conversational memory. The system supports CSV, Excel, PDF, image, and audio file formats and is validated on a carefully curated architecture built using Python, Streamlit, and the Groq API with the LLaMA 3.3 70B model, with optimization steps to handle large datasets and ensure efficient inferencedeliveryInthemoderndigitalage,organizations generate massive volumes of data that require efficient and accurate analysis to support strategic decisionmaking. However, traditional data analysis methods often demand skilled professionals, significant time, and advanced technical expertise. The AI Based Smart Data Analyst project aims to design and develop an intelligent system that automates the process of data analysis using ArtificialIntelligenceandMachineLearningtechniques.
Keywords: AI Data Analyst, Natural Language Processing, Large Language Models, LLaMA 3.3, Groq API, Python, Streamlit, Conversational Agent, Code Generation,Multi-FormatDataAnalysis.
I. INTRODUCTION
The global digital landscape has witnessed a significant riseinthedemandforsophisticatedandintelligentdata analysis systems, among which the enterprise analytics and business intelligence sector ranks as one of the leading categories of organizational decision-making. A poor data analysis experience occurs when the analytical process is interrupted or delayed, preventing users from extracting essential information and completing insight-driven tasks. This technical inadequacy demands immediate intervention, and any delayinprocessingperformanceorqueryresolutioncan resultinsevereconsequences,includingpermanentloss of business opportunities or reduced organizational competitiveness.
This project, titled AI Based Smart Data Analyst, aims to leverage the power of modern AI technologies, specifically Python, Groq API, and LLaMA 3.3 70B with a conversational agent framework, to build an efficient and reliable system for the early interaction and conversion of natural language queries into executable dataanalysiscode.Byanalyzingmoderndesignpatterns suchas modular agent-based architecture and multiturn conversational memory,theproposedsystemcan identify opportunities for analytical patterns and optimizations that may go unnoticed by traditional, code-dependentdataanalysismethods.
II OVERVIEW
The AI Based Smart Data Analyst is an intelligent, conversational data analysis agent that allows users to analyze complex datasets using plain English no programming knowledge required. The system leverages the Groq API with LLaMA 3.3 70B to convert natural language queries into executable Python code, whichisthensafelyexecutedwithinanisolatedsandbox environment and results are displayed as interactive charts and tables. Built with Python, Streamlit, Pandas, and Plotly, the application supports CSV, Excel, PDF, Image, and Audio file formats and is deployed publicly onStreamlitCloud.
AgenticAI frameworkssuch asLangChain andAutoGPT further advanced the field by introducing multistep reasoning, tool use, and memory management capabilities. These frameworks inspired the design of the DataMind AI agent, which incorporates a rolling conversation history, automated error recovery, and a secure code execution sandbox to deliver reliable analyticaloutputs.
TECHNOLOGY STACK
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072
III. SYSTEM ARCHITECTURE AND METHODOLOGY
A. Modular Agent-Based Architecture
The proposed system follows a Modular Agent-Based Architecture (MABA) comprising four distinct processinglayers:theInput Layer,theProcessingLayer, the AI Core Layer, and the Output Layer. This layered design ensures strict separation of concerns, reduces inter-component coupling, and significantly improves maintainability and scalability across the entire analyticalpipeline.
B. File Ingestion Pipeline
The File Ingestion Module accepts five distinct file formats through an automatic format detection engine thatroutesuploadedfilestotheappropriateprocessing pipeline based on file extension. CSV and Excel files are loadedintopandasDataFramesusingtheread_csv()and read_excel() functions respectively. Image files are processed through Pillow and Pytesseract OCR or the LLaMA Vision AI model. Audio files are transcribed using OpenAI Whisper. PDF documents are processed usingpdfplumberforpageleveltextextraction.
C. Groq LLM Inference Engine
The Agent Core Module communicates with the Groq API using the LLaMA 3.3 70B Versatile model. A structuredsystem promptisconstructedcontainingthe dataset's column names, data types, sample rows, and descriptivestatistics.TheLLMgeneratesPythoncodein response totheuser'snatural languagequestion, which isthenextractedusingaregex-basedcodeextractorand executedwithinanisolatedexec()sandbox.
D. Secure Code Execution Sandbox
All LLM-generated Python code is validated against a comprehensive keyword-level security blocklist before execution. Dangerous operations including os.system, subprocess, shutil.rmtree, socket, and dynamic import statementsareblockedatthevalidationstage.Approved code is executed within an isolated variable scope containing only the pandas DataFrame and standard analytical libraries, preventing unauthorized access to thehostsystem.
E. Development Methodology
The development followed a structured, iterative Agile methodology. The process began with a detailed RequirementAnalysisphase,followedbytheDesignand Planning Phase involving system architecture diagrams and data flow prototypes. The Implementation Phase integrated the Groq API, Streamlit frontend, and modular Python backend. The Testing Phase validated the system against the New Zealand Annual Enterprise Survey 2024 dataset comprising 55,620 rows and 10 columns. Finally, the Deployment Phase published the applicationonStreamlitCloudforpublicaccessibility.
IV. KEY FUNCTIONAL FEATURES
The AI Based Smart Data Analyst incorporates the followingkeyfunctionalfeatures:
• Natural Language to Python Code Generation usingLLaMA3.370BviaGroqAPIwithaverage responsetimeunder3.0seconds.
• Multi-Format File Support covering CSV, Excel, PDF, PNG, JPG, MP3, and WAV formats through dedicatedprocessingpipelines.
• Interactive Data Visualization using Plotly Express with automatic chart type selection includingbar,line,pie,andscattercharts.
• Conversational Multi-Turn Memory using Streamlit session state for context-aware follow-upqueryhandling.
• Automated Error Recovery with a single-cycle retry mechanism that resubmits failed code to theLLMwitherrorcontext.
• Secure Code Execution Sandbox with keywordlevel threat scanning and isolated exec() variablescope.
• Vision AI Image Analysis using the LLaMA 4 Scout Vision model for direct semantic understandingofuploadedimages.
• Audio Transcriptionand Analysisusing OpenAI WhisperforMP3andWAVfileprocessing.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072
• Real-Time Dataset Statistics displaying row count,columncount,andnullvaluecountinthe sidebaruponupload.
TransparentCodeDisplaywithacollapsiblesyntaxhighlightedcodeviewerforeveryanalyticalresponse.
V. RESULTS AND DISCUSSION
The experimental results of the proposed AI Based Smart Data Analyst framework demonstrate that combining conversational natural language processing with a secure agent-based execution pipeline significantly improves analytical accessibility and performance compared to traditional code-dependent tools.
The system was evaluated on the New Zealand Annual EnterpriseSurvey2024datasetcomprising55,620rows and 10 columns with zero missing values. Performance benchmarksweremeasuredacrossmultiplequerytypes onstandardconsumerhardware.
Table I: SystemPerformanceBenchmarks
The automated error recovery mechanism successfully resolved 92% of initial code execution failures within a single retry cycle. The keyword-level security scanner blocked 100% of test inputs containing dangerous operation keywords including os.system, subprocess, andsocketcallsbeforeexecution.
The conversational multi-turn memory system maintained accurate context across an average of 6.4
follow-up queries per session, demonstrating strong context retention capabilities. The system successfully generatedinteractivePlotlyExpresschartsfor 100% of eligible analytical queries, including bar, line, pie, and scatter plot types, without requiring any explicit chart typespecificationfromtheuser.
VI. CONCLUSION
In this project, we successfully developed a highperformance, intelligent AI Based Smart Data Analyst achieving all technical and functional objectives established at the outset. The implementation directly addressed the severe limitations found in existing data analysis tools, primarily concerning poor accessibility for non-technical users, absence of natural language support, and lack of multi-format data processing capabilities.
The mandatory adoption of a Modular Agent-Based Architecture using Python and Streamlit proved instrumental in creating a highly maintainable and extensible codebase. The integration of the Groq API with LLaMA 3.3 70B delivered sub-3-second response times consistently, making the analytical experience genuinelyinteractiveandpracticalforreal-worlduse.
The successful deployment on Streamlit Cloud demonstrates that the system is production-ready and publiclyaccessiblewithoutanylocalinstallation.Future enhancements include DuckDB integration for largescale datasets exceeding 100MB, multi-dataset join analysis for enterprise BI use cases, voice-based query inputviatheWebSpeechAPI,andscheduledautomated reportgenerationviaemailnotification.
This project successfully demonstrates a refined application of modern AI engineering principles to deliver a fast, secure, and user-centric solution that makes advanced data analysis accessible to everyone, regardlessofprogrammingbackground.
REFERENCES
[1] T. B. Brown et al., "Language models are few-shot learners,"inProc.NeurIPS,2020,pp.1877–1901.
[2] M. Chen et al., "Evaluating large language models trainedoncode,"arXivpreprintarXiv:2107.03374, 2021.
[3] A. Radford et al., "Robust speech recognition via large-scaleweaksupervision,"inProc.ICML,2023.
[4] W. McKinney, "Data Structures for Statistical Computing in Python," in Proc. 9th Python in ScienceConf.,2010,pp.56–61.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072
[5] A. Vaswani et al., "Attention is all you need," in Proc.NeurIPS,2017,pp.5998–6008.
[6] Meta AI, "LLaMA: Open and efficient foundation language models," arXiv preprint arXiv:2302.13971,2023.
[7] Groq Inc., "Groq LPU Inference Engine," Online. Available:https://groq.com,2024.
[8] Streamlit Inc., "Streamlit: The fastest way to build data apps," Online. Available: https://streamlit.io, 2023.
[9] H. Smith, "pdfplumber: Plumb a PDF for detailed information about each text character," GitHub Repository,2022.
[10] S. An et al., "Evaluating code generation with language models for data analysis tasks," in Proc. ACL,2023.