International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 10 | Oct 2025
p-ISSN: 2395-0072
www.irjet.net
Reinforcement Learning for Adaptive Optimization in Personalized Federated Learning Systems Binay Kumar Sah1, Md Sarazul Ali2, Abbas Mehdi3 1Packaged App Development Associate, Accenture, India 2Senior Associate Technical Consultant, Ahead DB, India 3Analyst, Deloitte, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Federated Learning (FL) enables multiple clients
To address these limitations, Personalized Federated Learning focuses on fitting models with clients’ local characteristics. However, most of the current PFL algorithms are based on fixed aggregation or heuristic adjustment strategies, which lacks adaptability and responsiveness.
to collaboratively train a shared model without exchanging raw data, preserving privacy across decentralized systems. However, heterogeneity among clients — in terms of data distribution, computational resources, and communication capabilities — often degrades convergence and generalization. To address these challenges, this paper proposes a novel framework integrating Reinforcement Learning (RL) for adaptive optimization in personalized federated learning (PFL). The RL agent dynamically adjusts hyperparameters such as learning rates, aggregation weights, and local update frequency to achieve faster convergence and personalized performance. Our approach models the optimization process as a Markov Decision Process (MDP), where the RL agent learns an adaptive policy that minimizes global loss while maximizing personalization rewards. Experimental results on benchmark datasets demonstrate significant improvements in convergence stability, accuracy, and fairness across clients compared to conventional FL algorithms. The suggested framework creates a basis for intelligent, self-tuning federated systems by bridging the gap between personalized optimization and adaptive learning..
Reinforcement Learning (RL), with its ability to learn dynamic policies from interaction, offers a promising avenue for adaptive optimization. By integrating RL into FL, the system can autonomously adjust training configurations based on real-time feedback, optimizing both global and personalized performance metrics. An RL agent continuously learns to modify parameters like learning rates, aggregation weights, and communication intervals in this paper's RL-driven adaptive optimization framework for PFL. In federated settings, the suggested approach seeks to increase convergence speed, fairness, and personalization.
2. LITERATURE REVIEW / RELATED WORK 2.1 Federated Learning (FL)
Key words: Deep Q-Network (DQN), Adaptive Optimization, Federated Learning (FL), Personalized Federated Learning (PFL), Reinforcement Learning (RL), Proximal Policy Optimization (PPO), Communication Efficiency, and Non-IID Data.
Federated Learning is a decentralized machine learning paradigm in which multiple clients, such as mobile devices, edge servers or organizations learn a global model together without storing and exchanging their raw data in a central repository. FL ensures that each client runs a local training process on its own private data pool, and only updates the model that is periodically sent to a coordinating server. The server aggregates these local changes using an algorithm known as Federated Averaging, creating a new global model, which once again redistributes updated versions to individual clients, etc.
1. INTRODUCTION Federated Learning (FL), a decentralized learning paradigm that enables clients to cooperatively train global models without sharing local data, has emerged as a result of the exponential growth of distributed data across mobile devices, IoT systems, and edge platforms. FL has significant obstacles despite protecting privacy, most notably statistical heterogeneity (non-IID data), system heterogeneity, and ineffective communication. Conventional FL techniques, such as FedAvg, perform poorly in heterogeneous environments because they assume uniform participation and identical data distributions. In addition, global optimization easily ignores the needs of particular clients; as a result, the developed models make poor generalization to all participants.
© 2025, IRJET
|
Impact Factor value: 8.315
This method keeps the data private and uses rules like the GDPR. For this reason, FL is extremely useful in areas where privacy is important, including healthcare, finance, and smart devices. But FL also has many difficulties to overcome – things like statistical heterogeneity and system heterogeneity and communication bottlenecks. These things make the model take more time to converge and work less well. Some recent advances in FL attack these
|
ISO 9001:2008 Certified Journal
|
Page 820