International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 08 | Aug 2025
p-ISSN: 2395-0072
www.irjet.net
Bridging Performance and Interpretability in Reinforcement Learning: The eXActor-Critic Framework Adegoke Oluyemi Borisade 1, Taiwo Fele 2 1Centre for Energy Research and Development, Obafemi Awolowo University, Ile Ife, Nigeria 2Computer Science Department, The Federal Polytechnic, Ado Ekiti, Nigeria
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Reinforcement learning (RL) is powerful for
avoidance and autonomous navigation in UAVs [7] and surface vehicles [8], transparent decision-making in autonomous driving and human-in-the-loop systems [9],[10], optimized debt recovery models [11], multi-intersection signal control to reduce traffic congestion [12], flood risk mitigation in urban drainage systems [13], adversarial attack detection for secure planetary landings [14], and sustainable manufacturing through real-time demand response [15]. These applications highlight RL's versatility in addressing complex, real-world challenges. Despite its successes, RL lacks explainability and interpretability features due to the opaque decision-making processes of its complex deep neural networks, which obscure how actions are derived from states and rewards [16]. This lack of transparency limits trust and hinders deployment in some critical applications like healthcare, robotics, and autonomous systems, where safety and reliability are paramount [16,17]. Additionally, achieving efficient interpretability without compromising computational performance remains a challenge, particularly in real-time systems [18]. The complexity of policy representations in high-dimensional or continuous action spaces further complicates understanding the relationships between states, actions, and rewards [17]. These challenges underscore the need for enhancements to make RL systems more transparent, interpretable, and efficient for trustworthy decision-making. Current approaches to achieving interpretability in reinforcement learning exhibit several key limitations that hinder real-world deployment. Selective Particle Attention [19] enables feature selection but struggles with highdimensional spaces and may miss critical features in dynamic environments. Dynamic state representations [20] show promise for automated driving yet depend on predefined structures that limit adaptability to novel scenarios. Model-based approaches [21] offer interpretability but demand substantial computational resources, making them impractical for real-time applications. Layer-wise Relevance Propagation [17] provides explanations but is noise-sensitive and lacks temporal coherence. Policy distillation methods [18] risk oversimplifying complex policies during compression. Blockchain-oriented RL [22] handles parameter configuration well but struggles with multi-agent dynamics and environment complexity, while photovoltaic frameworks [23] require extensive datasets and shows less
complex control tasks but often lacks transparency, limiting its use in high-stakes real-world applications. To bridge this gap, we propose eXActor-Critic, an explainable RL framework based on the Actor-Critic architecture, balancing performance and interpretability. The framework features a Dual-Path GRU Network, combining Bidirectional GRUs (BiGRUs) for longterm dependencies and Unidirectional GRUs (UniGRUs) for short-term adaptability, with Dynamic Mode Switching to optimize reward-based learning in non-stationary environments. For interpretability, eXActor-Critic employs attention-based saliency maps to identify critical state variables and PCA/t-SNE visualizations to reveal hidden-state dynamics. Tested on a modified CartPole-v1 environment with controlled non-stationarity, eXActor-Critic outperformed a traditional Actor-Critic baseline by 154.4% in mean reward (28.409 vs. 11.167), with statistical significance confirmed via t-Test (p ≈ 3.03E-159) and Mann-Whitney U Test (p ≈ 8.83E193). Stability mechanisms like Layer Normalization, Automatic Mixed Precision, and Z-score Advantage Normalization ensured robust training. Key contributions include a transparent RL framework with real-time explainability, a dual-path GRU architecture for adaptive learning, and visual interpretability tools (saliency maps, dimensionality reduction). eXActor-Critic advances trustworthy RL, with applications in robotics, healthcare, and finance, while future work focuses on scaling to larger environments and optimizing RNN architectures. Key Words: Explainable Reinforcement Learning, ActorCritic, Dual-Path GRU, Interpretable AI, Saliency Maps, NonStationary Environments.
1. INTRODUCTION Reinforcement learning (RL), a machine learning paradigm where agents learn optimal control actions by interacting with their environments, has achieved impactful results across a wide range of fields. It enables early fault detection for predictive maintenance and scheduling in industrial systems [1], dynamic pruning and inferencing for efficient healthcare IoT systems [2], energy-efficient and damage-recovery locomotion in adaptive robotics [3], systemic risk reduction in interbank networks [4], personalized medicine through optimized treatments including pharmacological anemia management [5], resilient energy grids through network reconfiguration [6], collision
© 2025, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 249