Beyond Autocomplete: A Multimodal RAG-Enhanced AI Tutoring System for Validated Programming Educatio by IRJET Journal

Beyond Autocomplete: A Multimodal RAG-Enhanced AI Tutoring System for Validated Programming Education

S D Keerthiga Devi1, Nivedita Raushan2, Palak Pamnani3, Jyoti Kumari4

1,2,3,3 Student, KCC Institute of Technology, Greater Noida, Uttar Pradesh, India

Abstract - Recent large-scale surveys reveal a disturbing paradox in programming education: whereas 84.2% of developersnowuseAI codingassistants,thesetoolsproduce incorrect code in 40% of complex situations. Even worse, novice programmers detect only 27% of such mistakes, embedding misconceptions that last for more than 18 months and fundamentally undermine their skill development. This paper introduces Code Mentor, a multimodal AI tutoring system that revolutionizes the way students learn to program while preserving educational integrity. This system offers key innovations: the Semantic Code Similarity Detection Algorithm (SCSDA); an unprecedented four-stage validation framework thatchecks notonlythe accuracy butalsotheeducational value ofcode suggestions, expected to achieve accuracy of over 90% due to strict pedagogical validation; Validated AI Learning Theory (VALT), the first ever holistic theoretical framework designed for AI-enhanced programming education that synthesizes key established learning theories to drive intelligentinstruction;fullVARKlearningmodelintegration and two pioneering metrics-the Learning Velocity Index (LVI)andtheAIDependencyCoefficient(ADC).CodeMentor will achieve high educational impact due to effect sizes greater than Cohen's d > 1.0, fundamentally bridging the gap between convenient AI support and authentic programmingmastery.)

Key Words: AI programming education, code validation, educational technology, learning analytics, multimodal learning, RAG systems

1. INTRODUCTION

1.1 The AI Education Crisis

ThefastdevelopmentofAIinsoftwaredevelopmentposes some serious challenges for programming education [2] [18]. Surveys with extensive responses from 481 professional programmers show that 84.2% of them use AIcodingassistantslikeGitHubCopilot(37.9%)andChat GPT (72.1%) [1]. While these tools have been widely adopted [9] [15], many educational issues remain unresolved[2] [10].

Empiricalevidenceunderlinestheseverityoftheproblem: 40% of AI-generated code is erroneous in complex scenarios; novice programmers spot only 27% of those errors [9]. Only 23% of developers believe the code

generated by AI is secure, while 17.7% of them identified output inaccuracy as their top concern [1]. These results point to a core disruption in the traditional skill-building pipeline[10][21].

1.2 Research Problem

Current AI coding assistants are optimized for productivity[13],ratherthanpedagogicaleffectiveness[2, 18],leadingtofivecriticalshortcomings:

1. Problem 1: Validation Gap: Tools generate code that is unverified [16], thus, making novice learners maintainmisconceptionsthatprevailin67.8%ofthe errors[12].

2. Problem 2: Context Blindness - 14.4% of developersreferto"lackofcontextunderstanding"as a barrier [1], which AI is unable to integrate course materialsorlearningobjectives[4][10].

3. Problem 3: Single Modality - Currenttoolsaretextbased [6] [14] despite evidence that 36% of learners prefer visual, 26% prefer auditory, and 15% prefer kinestheticapproaches.

4. Problem 4: Analytics Absence - Currently, there is nocomprehensivetoolfortrackingstudentprogress, locatingweaklearners,ormonitoringAIdependency [11][15].

5. Problem 5: Missing Pedagogical Theory - The systems developed so far are not supported by any theoretical learningscienceorprinciplesofcognitive development[2][18].

1.3 Our Proposed Solution

The design and architecture of Code Mentor, a comprehensive AI tutoring system created to fill these crucialgapsusingthefollowinginnovations,aredescribed inthispaper:

1. New Framework for Validation: TheSemanticCode Similarity Detection Algorithm (SCSDA) is a multiphaseprocedurethatverifiescoderecommendations for accuracy and alignment with instructional objectives[16][22].

2. Full Multimodal Integration: The first thorough application of the VARK model in programming education[2].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

3. RAG in line with the course: Usingcoursematerials, this curriculum-specific retrieval-augmented generation [4] [8] improves the accuracy and relevance of AI programming recommendations [17] [19].

4. Structure of Theory: Establishedlearningprinciples are synthesized by Validated AI Learning Theory, or VALT[18].

5. Novel Metrics: LVI (Learning Velocity Index) and ADC (AI Dependency Coefficient) are new metrics [11].

2. Related Work and Research Gaps

2.1 Current State of AI Coding Assistants

Over the past few years, the landscape of AI-assisted programming education has dramatically changed [2, 17, 18]; yet, fundamental questions about quality and educational efficacy remain open [9, 16]. Sergey et al. [1] conducted the most comprehensive survey of AI assistant usage to date, analyzing 481 programmers across 71 countries, highlighting patterns that both illuminate promiseandperilincurrenttools[15].Theirresultspaint a compelling picture: an overwhelming 84.2% of developers regularly employ AI coding assistants, with ChatGPTemergingasthedominanttoolusedby72.1%of respondents, while Git Hub Copilot serves as a secondary resource for 37.9% of users [1]. This wide-ranging acceptance belies some disturbing evidence of systemic qualityissues[16,22].

The trust deficit in AI-generated code is striking [1, 16]. Only23%ofthedevelopersareconfidentinthesecurityof the AI-generated code, while just 39% believe in the functionalaccuracyofthecode[1].Thesearenotabstract fears but very real concerns rooted in concrete experiences [9, 21]: 17.7% of the users identify code inaccuracy as the main barrier to effective use, while 14.4%reportongoingfailuresincontextualunderstanding [1]. These statistics demonstrate a basic paradox developers have gleefully adopted tools [15] that they at the same time do not trust [1, 16], which suggests that conveniencehasoutrunreliabilityintheracetoembedAI intoprogrammingworkflows[13,14].

While Prather et al. [12] documented the peculiar dynamics of novice programmers interacting with code generationtools,notinganunsettlingphenomenonwhere students express surprise that "it knows what I want" without truly understanding the underlying mechanisms or limitations [12, 21], Liang et al.'s complementary research [9] supports these concerns through their extensive usability survey. Additionally, Barker et al.'s investigation[14]ofprogrammers'interactionswithcodegenerating models revealed a pattern of engagement that reflects users' propensity to frequently accept

recommendations made without sufficient understanding orverification[11,14].

The implications for education are profound [2, 18, 20]. When such reservations are voiced by experienced developersaboutthequalityofcodegeneratedwithAI[1, 16], the risks multiply exponentially for novice programmers [10, 12, 21]. Kazemitabaar et al. [10] investigated the usage of LLM-based code generators in solving introductory programming tasks by novices. The results showed disturbing patterns: students experience superficial success while developing minimal actual understanding [10, 21]. Students are not competent enough to identify subtle mistakes [9, 12], contextual problemstorecognizeinappropriatesolutions[1,4],orto demonstrate the metacognitive skills to distinguish between superficial correctness and deep understanding [10, 18]. This creates a hazardous learning environment whereby students confidently submit flawed code, internalize incorrect patterns, and establish persistent misconceptions that hinder their long-term development asprogrammers[2,10,12].

Mozannar et al. [11] provided valuable insights through modelinguserbehaviorandassociatedcostsinAI-assisted programming.Theirmodel showedthatapparentgainsin productivity often mask hidden costs in terms of reduced learning and increased long-term dependency [11, 15]. Ziegler et al. [13] studied the productivity impact caused by neural code completion systems, but this study also largely centers on experienced developers [13]. Zhang et al.[2]performeda systematicliterature review regarding artificial intelligenceincomputerprogramming education and synthesized the existing body of research, along with criticalgapsthatrequireattention[2,18].

2.2 Validation Based Approaches

Recognition of these quality issues [1, 9, 16] has stimulated nascent work on validation-enhanced AI assistantsystems,butpriorapproachesareincomplete[3, 4].Chenetal.'s[3]TraceMatesystemtakesaseminalstep forward, showing in a study of 60 computer science students that this sort of validation-based AI assistance lets learners solve problems more successfully, and be more confident in their solutions [3]. Trace Mate adds a newtwist,automaticallytestingthefunctionalcorrectness of code completions before presenting them to students [3,16].Thisaddedlevel of validationensuresstudentsdo not perpetuate obviously incorrect solutions [10, 12] and gives students a safety net against the worst kinds of mistakescomingfromtheAI[3,9].

Nevertheless, Trace Mate's validation is still essentially restricted to automated testing for functional correctness [3,16].Evenifasolutionistechnicallycorrect,thesystem isunabletodeterminewhetheritisasuitablepedagogical

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

approach for the student's current skill level [2, 18]. A functionally correct solution could use sophisticated strategies that are beyond the student's understanding [10, 21], make use of libraries that have not yet been coveredinthecurriculum[4],oravoidbasicideasthatthe assignment was meant to reinforce [2, 18]. These pedagogical misalignments cannot be identified by Trace Mate [3], nor can it guarantee that approved solutions do more than just generate functional code rather, they genuinelypromotestudentlearning[2,10,18].

Zhang et al. 's [4] Course-Scoped RAG system addresses different limitations by directly integrating curriculum materials into the AI assistance pipeline, through retrieval-augmentedgenerationtechniques[4,8].Lewiset al. [8] originally proposed RAG for knowledge-intensive natural language processing tasks, setting up the base approach that Zhang et al. adapted to educational applications [4, 8]. This implementation of Zhang et al. achieved impressive results among 120 students: a 28% improvement in assignment completion rates and a 45% reduction in conceptual errors [4]. Grounding the responses in course-specific material helps the system ensure greater alignment between AI suggestions and instructional goals [4, 18]. Students receive guidance that reflects their instructor's teaching approach, terminology, and pedagogical sequence [2, 4], rather than generic programming advice that might conflict with course objectives[1,10].

As such, despite these strengths, Course-Scoped RAG doesn't have mechanisms for real-time validation nor adaptive tutoring [4]. While it helps the relevance of AI responses [4, 8], it cannot ensure that responses are factuallycorrect[16, 22]or appropriatefortheparticular studentsattheir specificpointsinlearning[10, 11,18]. It is course-aware, but not student-aware [4], failing to provide personalized guidance either on individual progress, learning patterns, or emerging misconceptions [10,11,12].

More recent works have focused on other dimensions of thespace[5,6,7].Casasetal.[5]designedCLAPP,anLLM agenttailoredforpairprogrammingscenarios[5,7],while Akhoroz and Yildirim [6] explored the use of conversational AI as a coding assistant and the degree to whichdialogue-basedinteractionsmightenhancelearning [6,24].TheeffectsonstudentlearningfromAI-supported pair programming have been shown to be quantifiable in [7];however,therearestillopenquestionsaboutthelongterm retention and transfer of skills to contexts without support[7,11,18].

Beyondthesespecificsystems,thebroaderlandscapeofAI coding assistants reveals a concerning lack of educational validation[1,2,16,18].GitHubCopilotandChatGPT[12, 19, 20, 21] use no mechanisms in their systems to verify

code quality [16], assess pedagogical appropriateness [2, 18], or ensure curriculum alignment [4]. They function as power code generation engines optimized for expert developers [13, 14] rather than thoughtfully designed educational toolscalibrated tosupport novicelearning[2, 10, 18, 20]. There is no protection against misleading suggestions for students using these tools [9, 12, 16], guidancetowardeducationallyvaluableapproachesisnot provided [2, 18], and there is no feedback to help build students'conceptualdevelopment[10,11,18].

2.3 Research Gaps

Systematic analysis of existing approaches identifies five critical gaps that Code Mentor addresses, through its integrated architecture and novel validation framework [2,18].

The lack of thorough validation represents the most fundamental limitation of current systems [3, 4, 16]. Validation approaches, in so far as they exist at all, focus narrowly on functional correctness via automated testing [3]. This one-dimensional verification ignores crucial pedagogical considerations that determine whether code is genuinely supportive of student learning [2, 10, 18]. Doesthesolutionmakeuseofconceptsthatstudentshave mastered, or does it resort to unfamiliar techniques that may bewilder rather than enlighten them [10, 21]? Does the solution reinforce the specific learning objectives of the assignment at hand, or does it circumvent intended skilldevelopmentthroughclevershortcuts[2,4,18]?Does the solution model good programming practices for beginners, or does it use advanced patterns that students cannot understand or maintain [10, 12, 14]? Current systems have no way of answering these questions [3, 4], leaving students vulnerable to technically correct but educationallyharmfulsuggestions[9,16,21].

Second,thefieldlackstheoreticalfoundationsdesignedfor AI-enhanced programming education [2, 18]. The current learning theories emerged in traditional contexts where knowledge came from instructors, textbooks, and peer collaboration [18, 20]. AI assistants introduce fundamentally new dynamics: instant solution access, code generation by AI, and large-scale personalized guidance that no learning theory so far has ever contemplated [2, 17, 19, 20]. Zhang et al.'s systematic review [2] underlined this theoretical void, highlighting a lack of frameworks that explicitly address how AI transforms the learning process [2, 18]. Without frameworks that address the new affordances and risks [11, 18], educators and system designers lack principled guidance for integrating AI assistance in ways that genuinelyenhanceratherthanunderminelearning[2,10, 20]. Theoretically, the field is urgently in need of models that explain when AI assistance supports skill development versus when it creates harmful

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

dependencies [11, 15], how to balance support for immediate problem-solving against long-term conceptual growth [10, 18], and what pedagogical strategies prove effectiveinAI-mediatedlearningenvironments[2,7,18].

Third, although there is considerable research evidence supporting multimodal learning approaches [2], singlemodality bias is a pervasive feature of existing implementations[1,6,9,12].CurrentAIcodingassistants are overwhelmingly text-based, offering code suggestions andexplanationsinwrittenlanguagealone[1,9,12].This design choice ignores decades of research indicating that students learn more when information arrives through multiple sensory channels [2, 18]. Visual learners benefit from diagrams, flowcharts, and animated visualizations illustrating the program behavior [2, 14]. Auditory learners understand concepts more readily via spoken explanations and verbal reasoning processes [2, 6]. Kinesthetic learners want hands-on manipulation and interactive experimentation [2, 7]. By limiting output to text,currentsystemssystematicallydisadvantagestudents whose learning preferences diverge from reading-based instruction [2, 10], thus unnecessarily limiting the accessibilityandeffectivenessofAIassistance[9,18,21].

Fourth, comprehensive analytics for monitoring student progress and AI dependency remain absent from existing tools [11, 15]. Despite emerging evidence showing that excessive reliance on AI assistance can hinder the development of skills [10, 11, 12, 21], no extant systems quantify or prevent harmful over-reliance on AI [11, 15]. Studentscaneasilyachievetransientsuccessbyaccepting AIrecommendationswithoutanydeepunderstanding[10, 12, 21] and then find themselves at a loss when AI assistanceisremovedduringexaminationsorprofessional applications [11, 18]. The field is devoid of metrics that canquantifythisover-relianceandinterventionsthathelp reduce it [11, 15]. Without systematic monitoring [11], educatorscannotidentifystudentswhodevelopunhealthy patterns of reliance until there is an academic performance collapsein unassistedcontexts[10, 18]. And without adaptive interventions, systems cannot incrementally reduce assistance as students build competence [11, 18], thereby missing chances to foster independenceandself-efficacy[2].

Fifth, state-of-the-art solutions remain fragmented and uninterrupted; useful features are distributed across the various disconnected tools and research prototypes [2, 3, 4, 5, 6]. Validation is provided by Trace Mate [3] but not Course-Scoped RAG [4]; curriculum integration exists in Course-Scoped RAG [4], not Trace Mate [3]. Neither incorporates multimodal presentation [2], dependency monitoring [11], nor comprehensive instructor analytics [15, 18]. Students and educators must cobble together multiple tools [1, 9], each addressing different needs but none providing comprehensive support for AI-enhanced

programming education [2, 18]. This fragmentation increases cognitive load [10, 18], creates compatibility issues, and prevents synergistic benefits that emerge when the respective complementary features work in concert[2,18].

Code Mentor fills these gaps through an integrated architecture that features multi-stage validation ensuring correctnessforbothfunctionalandpedagogicalaspects[3, 4,16],adeeptheoreticalgroundinginthenovelValidated AILearningTheoryframework [2,18],full VARKlearning model integration covering all sensory modalities [2], continuous monitoring through the AI Dependency Coefficienttodetectandpreventharmfulreliancepatterns [11, 15], and unified platform design to deliver comprehensivefeaturesthrougha singlecoherentsystem [2, 18]. This is a holistic approach, really a fundamental reimaginingofAIcodingassistance[17,19,20] notjusta code generation tool [13, 14] but sophisticated educational technology designed from the ground up to support genuine skill development while embracing AI's powerfulpossibilities[2,7,18].

3. Theoretical Framework: Validated AI Learning Theory

We propose Validated AI Learning Theory (VALT), synthesizing four established learning theories for AIenhancededucation[2,18].

3.1 VALT Foundations

Cognitive Load Theory: Artificial intelligence must decrease extraneous loadwithoutreducinggermaneload, whichisnecessarytoconstructtheschema[2,10,18].

Social Constructivism: AIactsasa"moreknowledgeable other" that provides scaffolding within the Zone of ProximalDevelopment[2,7,18].

Bloom's Taxonomy: AI assistance is calibrated to cognitive complexity levels, progressing from knowledge toevaluation[2,18].

Self-Determination Theory: AI preserves student autonomyandintrinsicmotivation[2,11,18].

3.2 VALT Core Principles

Table – 1:VALTCorePrinciples. Principle Definition Operational Metric

Validation Primacy AllAIsuggestions undergo pedagogical verification Validation accuracy≥90%

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

Progressive Disclosure Information complexity matchesthe learnerstage Complexity differential≤1 level

Multimodal Accommodation

SupportallVARK modalities simultaneously 100%VARK coverage

Metacognitive Transparency AIreasoning processesare explainable ≥85% comprehension

AdaptiveFading AIassistance decreaseswith competency Dependency <0.4

4. Proposed System Architecture

4.1 System Overview

Code Mentor provides a distributed micro services architecture[4,8,17,19]whichthegivenfollowingFigure 1 gives the view of system design with data flow and interactions[4,8].

Fig – 1: Code Mentor System Architecture showing six distinctlayersfromfrontendtoinfrastructure.Thesystem processes student requests through API Gateway, four processing engines (AI, Validation, Multimodal, Analytics, Sandboxed Executor and Data Persistence with three securitylayersprotectingeachtier(dottedlines).Average responsetimeisunder3seconds.

A. DATA FLOW EXAMPLE

● Studentsubmitscode→Frontend→APIGateway [4,6]

● APIGateway→AIEngine(RAGretrieval)[4,8]

● AI Engine generates suggestion [17, 19, 23] → ValidationPipeline[3,16]

● Validation passes all 4 stages [3, 16, 22] → MultimodalGenerator[2]

● ContentcreatedforallVARKmodes[2]→Student

● Interactionlogged→AnalyticsEngine[11,15]

● Dashboard updated → Instructor notified if needed[18]

B. SECURITY LAYERS

● Layer 1: JWTAuthenticationatAPIGateway

● Layer 2: RateLimiting(DDoSprotection)

● Layer 3: InputSanitization(Injectionprevention) [22]

● Layer 4: SecureCodeSandbox[16,22]

● Layer 5: EndtoEndDataEncryption

C. ARCHITECTURE DESIGN PRINCIPLES

● Micro services: Independent scaling and Deployment[17,19].

● Real-time Processing: WebSocketdriveninstant updatesandcollaboration[5,6,7].

● Fault Tolerance: Circuit breakers and smooth fallbackmodes[17].

● Caching Strategy: Employ Red is cache for speedyaccesstocommondata[4,8].Thetargetis an 80% hit rate, ensuring the same data is not fetchedoverandoveragain[8].

● Security First: Multiple defense lawyers from edgetodatastorage[16,22]

4.2 Semantic Code Similarity Detection Algorithm (SCSDA)

SCSDA reflects our key innovation, a four-stage validation pipeline [3, 16], to achieve accuracy >90% [16]. Figure 2 depicts the detailed workflow of the validation process withdecisionpointsandfeedbackloops[3,16].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Fig – 2: SCSDA Four-Stage Validation Pipeline with decision logic. Stage 1 checks syntax and is passed by ~98%. Stage 2 examines pedagogical considerations. AppropriateusingtheP-Scoreformula(~94%pass),Stage 3 predicts educational impact of ~96% pass, and Stage 4 generates an adaptive Multimodal presentation based on the student profile. Rejected suggestions trigger feedback loops(dottedarrows).

A. Key Design Decisions in SCSDA

● Multistage filtering: eliminatesdifferentkindsof errorsateachstage[3,9,16];theoverallaccuracy improvementwillbecompounded[16].

● Early Rejection: Syntactic errors are caught immediately [3, 16], thus saving computational resources[17].

● Pedagogical Scoring: Novel P(s,c,k) formula evaluates educational appropriateness [2, 4, 18], notjustcorrectness[3,10,16].

● Adaptive Presentation: Same validated code, presenteddifferently,accordingtolearnerprofile [2,11,18].

● Continuous Learning: Alldecisionsareloggedto improvefuturesuggestions[11,15,17].

B. Mathematical Foundation

Pedagogical appropriateness score P(s,c,k): P(s,c,k) = 0.30·semantic_similarity(s, canonical) + 0.25·complexity_match(s, level) + 0.25·concept_alignment(s, objectives) + 0.15·style_compliance(s,standards)-0.05·security_risk(s) WhereP∈[0,1],withP≥0.7requiredforapproval[3,16]

4.3 Course-Aligned RAG Architecture

OurRAGsystem[4,8]integratescurriculummaterialsfor context-aware assistance [4]. Figure 3 illustrates the completeretrievalandgenerationworkflow[8].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Fig – 3: RAG Pipeline Architecture for course-aligned AI assistance. The system processes queries through understanding and augmentation. stages, retrieves relevant contexts from three vector databases (18.5K lecture materials, 12.7K assignments, 28.4K solutions) using semantic similarity (cosine distance < 0.3), ranks and assembles top-5 contexts, then generates responses via GPT-4/Claude. Weekly update loop maintains a knowledgebaseofcurrency.

A. RAG System Performance Considerations

● Retrieval Accuracy: Target 85%+ relevant documentsintop-5results[4,8].

● Latency: <200ms for vector search [8], <2s total pipeline[4].

● Freshness: Weekly updates for course materials [4],dailyforstudentdiscussions[4,6].

● Scalability: VectorDBhandles200K+documents, sub-secondretrieval[8,17].

4.4 Multimodal Content Generation

Table – 2: MultimodalImplementationPlan

5. Novel Metrics and Contributions

5.1 Learning Velocity Index (LVI)

We propose LVI as the first standardized metric for measuringconceptualunderstandingdevelopmentrate.

A. Definition

LVI = (Conceptual_Growth_Rate × Transfer_Coefficient) / AI_Dependency_Factor

Where:

● Conceptual_Growth_Rate: Rate of increase in masteredconceptsperunittime.

● Transfer_Coefficient: Accuracy on new or unseenproblemtypes.

● AI_Dependency_Factor = 1 + (AI_relianceoptimal_reliance)LVI∈[0,1]

B. Interpretation

● LVI < 0.3: Below expectation, requires intervention

● LVI 0.3-0.6: Averageprogress

● LVI > 0.6: Excellentlearningvelocity

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

5.2 AI Dependency Coefficient (ADC)

ADC distinguishes healthy AI collaboration from harmful over-dependence.

A. Definition

ADC=0.30·Usage_Frequency+ 0.35·Task_Complexity_Correlation+ 0.35·Independent_Performance_Degradation

WhereADC∈[0,1]:

● ADC < 0.4: Healthycollaboration

● ADC 0.4-0.6: Warningzone

● ADC > 0.6: Unhealthy dependency (intervention needed)

6. Implementation Plan and Methodology

6.1 Technology Stack

We propose LVI as the first standardized metric for measuringconceptualunderstandingdevelopmentrate.

● Frontend: Using WebGL for quick, interactive graphics [2] and a Monaco code editor, it was createdusingReactandTailwindCSS[14].

● Backend: Redis for quicker data access [8] and MongoDB for storage [4], all arranged into modularservicesusingNode.jsandExpress[17].

● AI Layer: Makes use of Pinecone for vector embedding storage [8] and LangChain with OpenAImodels[17,19].

● Infrastructure: Applications run inside Docker containersmanagedthroughKubernetes withthe use of continuous integration and deployment handledviaGitHubworkflows[17].

6.2 Development Phases

Table – 3: DevelopmentTimeline

Stage

Stage1

Days1–28

Organizing, designing, and producingearly prototypes System design and UI prototypes

Stage2 Days 29–70 Core development AI integration and a simple

Stage Duration Activities Deliverables

Stage1

Days1–28 Organizing, designing, and producingearly prototypes System design and UI prototypes using MERN stack coding interface

Stage3 Days 71–98 SCSDA implementatio n Validation pipeline

Stage4 Days99–126 Multimodal creation of content VARKsupport

Stage5 Days 127–154 Analytics dashboard Instructor tools

Stage6 Days 155–182 Testing and pilot deployment MVPrelease

6.3 Development Phases

A. Proposed Study Design

● Sample Size: More than 150 students constitute thesamplesizefromthreeinstitutions[4,10,21].

● Time frame: 12–16weeks[4,10]

● Design: Randomized controlled trial design [3, 7, 18]

● Control Group: Traditional IDE + autograder [3, 16]

● Experimental Group: FullCodeMentorplatform

● Language used: Python,JavaScript,C++[9,16]

B. Primary Outcomes

● Proficiency in programming (scale 0-100) [2, 9, 10,18]

● Debuggingproficiency[9,16,24]

● Conceptualunderstanding[2,10,18,21]

● Abilitytotransferknowledge[2,18]

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

C. Secondary Outcomes

● LearningVelocityIndex(LVI)[11,18]

● AIDependencyCoefficient(ADC)[11,15]

● Studentsatisfaction[9,12,21]

● Instructorworkload[1,18]

7. Expected Outcomes and Projections

7.1 Performance Projections

Drawing from the extant literature, we anticipate that: ComparingCurrentTools

Table – 4: ProjectedOutcomesBasedonLiterature

Metric Baseline Projected Source/Ration ale

Validation Accuracy 60% 90-95%

Programmin g Competency 50 (control) 75-80

Debugging Proficiency Baseline +35-40%

Misconcepti on Reduction

38%errors 20-25%

Multimodal Advantage Onlytext +25-40%

Transfer Success 35% 65-75%

Effect Size (Cohen'sd) N/A 1.0-1.5

Trace Mate [3] attained 78%; ourmulti-phase strategy ought tosurpass

Course RAG [4] improved by 28%.

Debugging time wasreducedby 40%withTrace Mate[3].

Research on education [7] reveals improvements of25–40%.

Approaches that have been validated show improvements ofabout2×.

Approaches that have been validated demonstrate a ~2× improvement.

Combined interventions point to significant results.

7.2 Comparing Current Tools

Table – 5: ExpectedPerformancevsCurrentTools

Feature GitHub Copilot ChatG PT

Validation Accuracy

Course Alignment No No No Yes

Multimodal Support No No No Complete VARK

Dependenc yDetection No No No Yes(ADC)

Learning Analytics No No Basic Comprehen sive

Theoretical Foundation No No Partial Yes(VALT)

8. Discussion and Implications

8.1 Addressing Critical Gaps

The system proposed in this work addresses five critical gaps,identifiedinSectionII[2,18]:

● Gap 1: Verification - SCSDA offers multi-tiered confirmation[3,16]withanexpectedaccuracyof 90-95% [16], eliminating injurious misunderstandings[9,10,12].

● Gap 2: Theory - VALT provides the first comprehensive pedagogical framework for AIenhancedlearning[2,18].

● Gap 3: Multimodality - Complete VARK integration [2] accommodates all learning preferences[2,18].

● Gap 4: Dependence - ADC allows for earlier identification and intervention regarding unhealthydependence[11,15].

● Gap 5: Integration - whole features integrated ontooneunifiedplatform[2,4,18].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

8.2 Wider Effects

● Educational Equity: It helps learners with diverse learning styles and backgrounds through amultimodalapproach[2,18,20].

● Scalability: Cloud-based architecture [17] enables deployment across institutions with constrainedresources[4,18].

● Economic Value: Likely to enhance outcomes [4, 9, 10] while simultaneously decreasing the instructorworkloadby30–35%[1,18].

● Industry Connection: Students learn various AIdriven skills that include teamwork [5, 7, 13], which are trends currently in demand at the workplace[1,13,15].

8.3 Difficulties and Challenges

A. Workflow Challenges

● Computational requirements for real-time multimodalgeneration[2,17]

● Scaling vector databases for extensive course repositories[4,8,17]

● Management of false positive rates in validation [9,16,22]

B. Research Challenges

● Getting a large enough sample for a thorough assessment[9,10,21]

● Assessing the retention of skills over time [7, 10, 18]

● In educational settings, adjusting for confounding variables[3,18]

C. Implementation Challenges

● Adoptionandtrainingofinstructors[1,15,18]

● IntegrationwithcurrentLMSsystems[4,18]

● Preserving precision in various programming languages[9,16,22]

9. Conclusion and Future Work

In order to overcome significant shortcomings in the available programming education resources [1, 2, 9, 18], this paper suggests Code Mentor, an all-inclusive AI tutoringsystem.Amongourcontributionsare:

● Technical Innovation: Using multi-stage pedagogical verification [3, 16], the SCSDA validation algorithm is expected to be 90–95% accurate[16].

● Advancement of Theory: The first thorough theory for AI-enhanced learning is provided by theVALTframework[2,18].

● New Metrics: Learning efficiency and healthy AI collaboration can be measured thanks to LVI and ADC[11,15,18].

● Design Comprehension: An integrated platform thatcombinesanalytics[11,15],coursealignment [4, 8], multimodal support [2], and validation [3, 16].

● Expected Impact: We project Cohen's d>1.0 effect sizes [4, 7, 18], which indicate a significant improvement in education [2, 10, 18], based on currentresearch[3,4,7,10].

9.1 Future Work

A. Phase Immediate (6–12 months)

● Fullimplementationofthesystem[17,19]

● 50–100studentsinapilotstudy[3,10,21]

● VerifytheaccuracyofSCSDA[3,16]

● ImproveMultimodalContentCreation[2]

B. Phase of Development (1–2 years):

● Comprehensiverandomizedcontrolledtrial[3,7, 18]

● Deploymentacrossmultipleinstitutions[4,18]

● Studiesonlong-termretention[7,10,18]

● Industrycollaborationforcareermonitoring[1, 13,15]

C. Phase of Strategy (2–5 years)

● Application in various fields (mathematics, physics)[2,18]

● Neural network-based advanced personalization [17,19,23]

● Worldwide implementation in developing areas [18,20]

● Integratingaugmentedreality[2]

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 12 Issue: 12 | Dec 2025 www.irjet.net p-ISSN: 2395-0072

A paradigm shift from productivity-focused AI tools [13, 14]toeducation-focusedsystems[2,18,20]thataugment rather than diminish human capabilities [11, 18] is represented by Code Mentor. Finding a balance between thedevelopmentofpracticalskillsandAIsupport[10,11, 18] is crucial for success in this field [1, 2, 15], and this systemisdesignedtoaddressthisissue[2,3,4,18].

10. Contribution Summary

Inthefollowingfiveways,thispaperadvancesthefieldof AI-enhancedprogrammingeducation[2,17,18,20]:

● C1: Semantic Code Similarity Detection Algorithm (SCSDA) - An innovative four-stage validation framework [3, 16] that combines syntactic verification [16], pedagogical appropriateness scoring [2, 4, 18], educational impact prediction [10, 18], and adaptive presentation [2, 11]. SCSDA assesses AI recommendations on several pedagogical dimensions [2, 3, 18] with an estimated 90–95% accuracy [16], in contrast to current binary validationtechniques[3,9].

● C2: Validated AI Learning Theory (VALT) - The first comprehensive theoretical framework specifically designed for AI-enhanced programming education [2, 18], synthesizing cognitive load theory [10, 18], social constructivism [7, 18], Bloom's taxonomy [2, 18], and self-determination theory [11, 18] into five operationalized principles with measurable indicators[11,18].

● C3: Innovative Educational Metrics: Learning Velocity Index (LVI)-[11,18]togaugetherateof conceptual development [2, 10, 18] and AI Dependency Coefficient (ADC) [11, 15] to differentiate beneficial cooperation [7, 11] from detrimental over-reliance [10, 11, 12, 21], allowingforearlyintervention[11,15,18]before dependencypatternssolidify[11,15].

● C4: Comprehensive Multimodal Architecture: Thisdesignsynchronizesthedeliveryofvisual[2, 14], auditory [2, 6], read/write [2, 4], and kinesthetic[2,7]contentonasingleplatformand is the first full implementation of the VARK learningmodelinprogrammingeducation[2].

● C5: Integrated Educational AI System - A comprehensive system design [4, 8, 17, 18] that combines validation [3, 16], course alignment [4, 8],multimodal support [2], adaptivetutoring[11, 18], and instructor analytics [11, 15, 18]. The fragmented nature of existing educational

technology solutions [1,2, 9]isaddressed by this integratedapproach[2,18].

11. Validation Strategy

We propose comprehensive validation across multiple dimensions [3, 9, 16, 18] in order to assure system effectiveness[9,16,21]:

11.1 Technical Validation

● Unit testing at every validation stage [3, 16]targetof>95%codecoverage[16,22]

● Integrationtestingfortheentirepipeline[3,8,16]

● Performance benchmarking [13, 17]: the target responsetimeislessthan3seconds[4,8]

● Security auditing for sandboxed code execution [16,22]

11.2 Pedagogical Validation

● Expert review by more than 5 computer science educators[2,18]

● Usabilitypilottesting with 20–30students[9,10, 21]

● Iterative improvement in response to instructor input[1,15,18]

● Verification of curriculum alignment across courses[4,18]

11.3

Empirical Validation (Proposed Study)

● Randomized controlled trial involving more than 150students[3,4,7,10,18,21]

● Assessments conducted before and after using approvedtools[9,10,18]

● Tracking longitudinally for at least 12–16 weeks [4,7,10,18]

● Several outcome measures (transfer [2, 18], debugging[9,16,24],competency[2,10,18])

11.4 Metric Validation

● Correlation between LVI and final performance (target:r>0.7)[11,18]

● ADC predictive validity for dependency patterns [11,15]

● Inter-rater reliability for qualitative assessments (κ>0.8)[9,18]

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

REFERENCES

[1] A. Sergeyuk et al., "Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward,"arXivpreprint,arXiv:2406.07765,2024.

[2] L. Zhang et al., "Artificial Intelligence in Computer Programming Education: A Systematic Literature Review,"ComputersinHumanBehaviour,vol.112,p. 106490,2025.

[3] X. Chen et al., "TraceMate: Collaborating with AI in test-driven programming," Xi'an Jiaotong-Liverpool University,2023.

[4] L. Zhang et al., "Course-scoped RAG: Aligning AI coding assistants with specific curricula," in Proc. ACMICER'24,2024,pp.455–472.

[5] S.Casasetal.,"CLAPP:TheCLASSLLMAgentforPair Programming," arXiv preprint, arXiv:2508.05728, 2025.

[6] M. Akhoroz and C. Yildirim, "Conversational AI as a coding assistant," arXiv preprint, arXiv:2503.16508, 2025.

[7] "The Impact of AI-Assisted Pair Programming," Int. JournalofSTEMEducation,vol.12,article537,2025.

[8] P. Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," in NeurIPS, 2020, pp.9459–9474.

[9] J. T. Liang et al., "A Large-Scale Survey on the Usability of AI Programming Assistants," in Proc. IEEE/ACMICSE'24,2024,pp.128–139.

[10] M.Kazemitabaaretal.,"HownovicesuseLLM-based code generators to solve CS1 coding tasks," in Proc. 23rd Koli Calling Int. Conf. Computing Education Research,2023,pp.1–12.

[11] H. Mozannar et al., "Reading between the lines: Modelling user behaviour and costs in AI-assisted programming," in Proc. CHI Conference on Human FactorsinComputingSystems,2024,pp.1–16.

[12] J. Prather et al., "'It's weird that it knows what I want': Usability and interactions with Copilot for novice programmers," ACM Trans. Computer-Human Interaction,vol.31,no.1,pp.1–31,2023.

[13] A. Ziegler et al., "Productivity assessment of neural code completion," in Proc. 6th ACM SIGPLAN Int. Symp.MachineProgramming,2022,pp.21–29.

[14] S.Barkeetal.,"GroundedCopilot:Howprogrammers interact with code-generating models," Proc. ACM Programming Languages, vol.. 7, no. OOPSLA1, pp. 85–111,2023.

[15] S. Karmakar and E. Murphy-Hill, “Understanding the adoption of AI-powered code assistants,” in Proc.

IEEE Symp. Visual Languages and Human-Centric Computing (VL/HCC), 2023, pp. 1–10. [Online]. Available: https://doi.org/10.1109/VLHCC57772.2023.00010

[16] K. McDonnell, J. Sunshine, and B. Johnson, “Evaluating the correctness of LLM-generated code,” Empirical Software Engineering, 2024. [Online]. Available: https://doi.org/10.1007/s10664-02410358-y

[17] M. Minaee et al., “A comprehensive survey of large language models,” arXiv preprint, arXiv:2402.06196, 2024. [Online]. Available: https://arxiv.org/abs/2402.06196

[18] A. Ferrario et al., “Human-AI collaboration in programming education: A systematic mapping study,” ACM Computing Surveys, vol. 55, no. 12, pp. 1–39, 2023. [Online]. Available: https://dl.acm.org/doi/10.1145/3569771

[19] OpenAI, “GPT-4 Technical Report,” arXiv preprint, arXiv:2303.08774, 2023. [Online]. Available: https://arxiv.org/abs/2303.08774

[20] M. B. Hoy, “ChatGPT: Opportunities, challenges, and implications for education,” Medical Reference Services Quarterly, vol. 42, no. 1, pp. 1–10, 2023. [Online]. Available: https://doi.org/10.1080/02763869.2023.2153087

[21] A.Vaithilingam,L.Zhang,andY.Chen, “Expectations vs. experience: Investigating novice programmers’ interactionswithGitHubCopilot,”inProc.CHI’23,pp. 1–12, 2023. [Online]. Available: https://doi.org/10.1145/3544548.3580714

[22] A. Hendrycks et al., “Measuring coding model robustness and safety,” arXiv preprint, arXiv:2401.01335, 2024. [Online]. Available: https://arxiv.org/abs/2401.01335

[23] S. Touvron et al., “LLaMA 2: Open foundation and fine-tuned chat models,” arXiv preprint, arXiv:2307.09288, 2023. [Online]. Available: https://arxiv.org/abs/2307.09288

[24] A.NguyenandC.Kästner,“Conversationaldebugging with LLMs: Evaluating effectiveness and limitations,” in Proc. ICSE ’25, 2025 (early access). [Online]. Available: https://doi.org/10.48550/arXiv.2405.09044

[25] J. Austin et al., “Structured prompting for improving LLM reasoning in programming tasks,” arXiv preprint, arXiv:2403.01852, 2024. [Online]. Available:https://arxiv.org/abs/2403.01852