International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 11 Issue: 12 | Dec 2024
p-ISSN: 2395-0072
www.irjet.net
Leveraging the Complexity and Criticality Matrix as a Governance Framework for Compliance in Generative AI Systems Saiprapul R Thotapally Global Payments, Georgia, USA ---------------------------------------------------------------------***--------------------------------------------------------------------to serious repercussions, including legal penalties, Abstract - Generative AI systems, particularly those financial losses, and reputational damage [5], [6].
powered by Large Language Models (LLMs), are increasingly integrated into compliance-heavy enterprise workflows, including financial audits, healthcare compliance checks, and data privacy reviews. While these models excel at producing contextually rich and fluent responses, their reliance on probabilistic patterns rather than semantic understanding poses significant challenges. Errors such as hallucinations, biases, and factual inaccuracies risk severe legal, financial, and reputational consequences.
Motivation High-level governance frameworks (e.g., NIST AI RMF [11], OECD AI Principles [15], ISO/IEC standards [12]) emphasize transparency, accountability, and fairness. However, they offer limited guidance on turning these abstract principles into daily operational practices. Enterprises struggle with questions like: When can LLM outputs be trusted as-is? When is it necessary to bolster responses with RAG for factual correctness? When should a human reviewer intervene for compliance-critical decisions?
This thesis introduces the Complexity and Criticality Matrix, a governance framework designed to guide responsible AI deployment in regulated environments. By classifying tasks according to complexity (domain specificity) and criticality (compliance impact), the framework prescribes oversight strategies: direct LLM responses for low-risk tasks, Retrieval-Augmented Generation (RAG) for improved factual grounding where needed, and Human-in-the-Loop (HITL) interventions for high-stakes, compliance-critical scenarios. This structured approach bridges the gap between high-level AI governance principles and practical operationalization, enabling organizations to integrate oversight seamlessly within DevOps/MLOps pipelines.
Contribution This thesis addresses these gaps by introducing the Complexity and Criticality Matrix, a governance framework that classifies tasks according to their complexity (domain-specific knowledge required) and criticality (severity of errors). Each quadrant of the matrix is associated with a different oversight strategy:
While this work is primarily conceptual, it lays the groundwork for future empirical validation and scalability. By providing a method to balance automation with accuracy and regulatory adherence, this thesis aims to empower enterprises to confidently leverage generative AI in even the most stringent compliance contexts. Key Words: Generative AI, Compliance, RetrievalAugmented Generation (RAG), Human-in-the-Loop (HITL), DevOps, MLOps, AI Governance
1.INTRODUCTION
|
Impact Factor value: 8.315
Low Complexity/Low Criticality: Direct LLM outputs for efficiency.
●
High Complexity/Low Criticality: Selective RAG to ensure factual grounding.
●
Low Complexity/High Criticality: Prompt templates with minimal HITL checks for compliance accuracy.
●
High Complexity/High Criticality: Combined RAG and HITL interventions plus auditing for top-tier regulatory adherence.
This structured approach operationalizes governance principles, enabling organizations to integrate compliance checks into DevOps/MLOps pipelines [7], [8]. While no empirical results are presented here, the framework sets a foundation for future validations, including heuristic or zero-shot classification approaches for automating task categorization.
Large Language Models (LLMs) such as GPT-4 and BERT have demonstrated remarkable capabilities in producing contextually relevant and fluent responses [1], [2]. As these systems find use in regulated sectors— financial auditing, healthcare compliance, and data privacy requests—organizations must ensure not only efficiency but also strict adherence to legal and ethical standards [3], [4]. Misinterpretations, hallucinations, and biases can lead
© 2024, IRJET
●
|
ISO 9001:2008 Certified Journal
|
Page 687