Survey on Optimization of IoT Routing Based On Machine Learning Techniques

Survey on Optimization of IoT Routing Based On Machine Learning Techniques

1Research Scholar, Department of Computer Science with Data Analytics, Vellalar College for Women, Thindal, Erode, Tamil Nadu,India.

2Assistant Professor, Department of Computer Science with Data Analytics, Vellalar College for Women, Thindal, Erode, Tamil Nadu,India. ***

Abstract: Internet of Things (IoT) is a new paradigm that is supplying enormous offerings for the inventive technology achievements by the researchers. However, thereisstillalotofroomtodealwithvariouschallenges and utilize IoT in different sectors to maximize automation. IoT smart portable devices such as smart phones, home appliances, healthcare gadgets, smart automotivedevices,automationindustrydevices,andso onarelinked.ThoughtheuseofIoTenableddeviceshas increased in many fields, the manufacturing sector still faces many connectivity issues due to factors such as device mobility; limited processing power and resource availability, which includes energy, bandwidth constraints, routing cost, and end-to-end delay; communication between nodes via intermediate mobile nodes towards destination may also fail links often, affecting the industry. IoT is rapidly making the world smarter by connecting the physical and digital worlds, and it is anticipated that more than 20 billion devices will be connected by 2024. It gives opportunity, but it alsocomesavarietyofrisks.Theissueathandishowto safeguard billions of gadgets, as well as the networks that support they operate. In this paper, we covered numerous IoT-related studies and how routing algorithmsareusedinmethodsofmachinelearning.

Keywords: Internet of Things, Routing, machine learning, Deep Learning, convolution neural network, support vectormachine

I. INTRODUCTION

IoTisanetworkofinterconnecteddevices,eachwithits ownuniqueidentification,thatautomaticallygatherand exchange data across a network. Connected gadgets collect data and, in some cases, act on it using built-in sensors. The purpose of IoT is to have devices that selfreport in real time, increasing efficiency and bringing crucial information to the surface faster. The Internet of Things has the potential to change a wide range of industries.Ithasalargeareathatisconnectedwithwebenabled devices and is utilised in a variety of applications ranging from road transportation to intelligent households, health, retail, and supply

networks. Kevin Ashton coined the phrase "Internet of Things"in1999.

The Internet continues to be changing. The internet continues to grow rapidly. Every thing is becoming connected, and each will have its own distinct identity [1]. The current static nature of the internet will be transformedintotheextremelydynamicFutureInternet, frequently referred to as the Internet of Things. The same way that individuals are linked to the internet, devices are linked to the internet of things. The devices rely on humans to generate all of the information. Humans lack resources such as time, precision, and attention. There is a high demand for machines to behave independently [2,3]. It will significantly reduce waste, costs, and resource loss. The Internet of Things (IoT) has gained prominence in the current context and has as a result. It will significantly reduce waste, costs, and resource loss. The Internet of Things (IoT) has become common in today's environment and has as a massive scientific revolt on workstation in the present digital age that creates an energy to the human life in everyfield.

EachdeviceintheInternetofThingscanberegardedas a route, and a routing algorithm is used when data is shared and transmitted throughout routes. Routing is essentialbecausenodesinanIoTnetworkserveashosts and routers, supplying data to gateways. Many routing protocols for networks of sensors have been proposed and have been implemented in IoTs. The energy consumption of transmitting nodes is influenced by the routing of data from source to goal. Stochastic methods are a natural way of analysing the energy consumption of each node and the overall network due to the network'srandomactivity.

Furthermore, routing typically incorporates nodes, as evidenced by a significant amount of overhead in the form of beaconing to destinations. During beacons, the source nodes look for flood path/ping-messages from neighbours, that are re-diffused before packets arrive. Thedestinationmeetstherequirements,and thepathis determined. Furthermore, variables such as beacon

interval influence the velocity of beacon propagation. Theenergyandperformanceeffectsofroutingmustalso be estimated, as well as the related control and information packet overheads. The Internet of Things is currently seeing an upsurge in threats and security vulnerabilities. Current security approaches could be utilized to counter specific IoT attacks. Traditional tactics, however, may be ineffective in the face of technology improvements, as well as a diversity of assaulttypesandintensitylevels.Thus,itisfundamental and critical to connect IoT and Machine Learning (ML) technologiesinordertoimprovetheircollaborationina varietyofways.

As a result, enabling ML in IoT for learning and analyzing the behaviors of IoT devices/objects and systems based on prior information and experiences may help the IoT ecosystem to successfully manage the unanticipated deterioration induced by anomalous conditions. As a result, ML approaches have witnessed substantial technological advancement, bringing up a plethoraofnewresearchavenuestoanswerpresentand futurechallengesinavarietyoffields[4][5].IOTexhibit unpredictable dynamics and behaviors, and ML algorithms, in keeping with the nature of IOT, do not require human interaction [6]. However, there are two major difficulties to ML in IOT: node resources and computational restrictions, and the requirement for huge data sets for learning. In terms of IoT network security, one of the most significant issues that ML algorithms confront is the difficulty in applying them to theintegrityandconfidentialityofsecuritystandards.As a result, machine learning algorithms can aid in the enhancement of security in IoT networks. In the next section, we reviewed numerous studies related to machinelearningalgorithmsforroutinginIOT.

II.LITERATURE REVIEW

Robert Basomingera et al.,[7] proposed host and cluster-baseddetectionsystemsthatuseroutecachesto detect malicious routes caused by routing misbehavior attacks on control packets. The detection system can be supervised (using labeled data or an exploration network) or unsupervised (using unlabeled data). Although the three techniques perform differently, their use cannot be determined solely by their performance. In contrast, the choice of consumption should be made relatively based on the network's state. If the network's priorstateisknown,supervisedwithlabeleddatacanbe utilizedtogenerateatrainingdataset.Whenthenetwork lasts longer and can be monitored, such a scenario is morepossible.Whensomenetworkdetailsareknownor can be anticipated, an exploration network may be employed. Unsupervised is appropriate when the network is completely unpredictable, unmonitored, or short-lived. The proposed methods' complexity is quite

lowbecausetwoofthethreerecommendedwaysdonot involve training the model on the real test network. Unsupervised learning does not require training, and supervised learning with an exploratory network trains outside of the actual test network. The third method, supervised bylabelleddata, trainsinlessthana second, because to the short quantity of the dataset. The simulationusestheDSRreactiveroutingprotocol,butit sheds light on building a more universal technique to adapt to different routing systems. After the cluster head is chosen, the suggested cluster-based detection system is confined to the detection system. The proposed detection methodology is intended to be suitable for any method of selecting a cluster head. However, it is worth mentioning that the cluster head selection procedure causes network issues. The act of picking the cluster head adds overhead to the network (which can be significant if done extensively on a large network). As a result, future expanded study will investigatetheimpactofclusterheadselectionoverhead onthedetectionsystem,aswellasquantifythedetection scheme's energy and computation resources. An examination of various network variables such as throughput, delays, or latency will allow the systems to beappliedtotrendingadhocnetworkapplicationssuch asvehiclenetworks,flyingnetworks,andevenIoT-based networks.

Murtadha M et al.,[8] extend method has significantly improved MANET lifetimes compared to existing methods, while maintaining similar packet delivery ratios (PDR) and route generation times. Establishing communications during a disaster or its aftermath is a critical action for conclusion and rescuing survivors. However, during such disasters, the infrastructure requiredtoestablishwirelesscommunications,B.Mobile communicationsaffectedby thedisaster.Inthisstudy,a new routing method is proposed to efficiently connect nodes in MANETs to operational BSs to establish communicationwithsurvivorsduringorafteradisaster. The proposed method aims to extend the network lifetimeby balancingnodeloadandavoiding exhausting nodes with limited remaining power. This goal is achieved using RL. In RL, a neural network predicts routesfromnodestoBSsgiventhenetwork state.Using the proposed method can extend the life of the device, greatly increasing the chances of saving a life. Besides better robustness, the proposed method was able to achieve similar performance in terms of PDR and route discovery time compared to existing routing protocols. Also, since the nodes in wireless networks move at relativelyhighspeeds,theproposedmethodcanprovide efficient communication for search and rescue teams duringsearchandrecoveryofsurvivors.Infuturework, the ability to directly generate routes based on MANET input states using generative adversarial networks (GANs) depends on the ability of these neural networks

to generate complex multidimensional outputs. The rootsusedfortrainingtheneuralnetworkaregenerated using the method proposed in this work. Therefore, the sameroutecanbegeneratedwhilesignificantlyreducing theroutegenerationtime.

Saeed Kaviani et al.,[9] presents DeepCQ+ routing integrates emerging multi agent deep reinforcement learning (MADRL) techniques into existing Q-learningbased routing protocols and their variants in a novel way,achievingpersistentlyhigher performanceacross a widerangeofMANETconfigurationswhiletrainingona limited set of network parameters and conditions. DeepCQ+ routinely outperforms its Q-learning-based competitors in terms of end-to-end throughput and overhead, with an overall improvement in efficiency. In terms of network sizes, mobility circumstances, and traffic dynamics, DeepCQ+ maintains impressively similar performance benefits in numerous cases for which it was not trained. This study presented a successful and practical hybrid approach combining MADRL and CQ+ routing approaches for designing a robust,reliable,efficient,andscalablepolicyfordynamic wireless communication networks, including numerous instances for which the algorithm was not trained. Our MADRLarchitecture,inconjunctionwiththeexplainable CQ+ structure, is uniquely built for scalability, allowing us to train and test routing policies for varying network sizes, data rates, and mobility dynamics while maintaining consistently excellent performance across scenariosthatwerenotpreviouslytrainedfor.DeepCQ+ routing, a DNN-based resilient routing policy for dynamic networks, is based on CQ-routing but also monitors network statistics to improve broadcast/unicast decisions. DeepCQ+ routing is proved tobefarmoreefficientthantypicalCQ+routingsystems, with significantly lower normalised overhead (number of transmissions per number of successfully delivered packets). Furthermore, the policy is scalable and employsparametersharingforallnodesduringtraining, allowing it to reuse the same taught policy in scenarios withvaryingmobilitydynamics,datarates,andnetwork sizes. It should be noted that sharing parameters across all nodesisnotrequiredduring execution.Inthefuture, weintendto broaden theDeepCQ+ routing action space toincludenext-hopselectionfortheunicastmode.Other interesting extensions include extending DeepCQ+ routing to accommodate heterogeneous wireless networks with multiple radio interfaces per node, expanding ACK-based information sharing to include additional context, and accommodating different performance metrics such as end-to-end delay minimization, overhead minimization, and goodput rate maximisation. A further extension of the hybrid DeepCQ+ routing paradigm continues to maintain scalable and robust routing policies, while prioritizing andbalancingnetworkmetricstobestmeettheneedsof

almost any MANET environment, including heterogeneousMANETs.That'sit.

Hossam Farag et al.,[10] proposed method adopts Qlearning at each node to learn the best parent selection policybasedondynamicnetwork conditions.Eachnode maintains the routing information of neighboring nodes as Q values. It expresses the combined routing cost as a function of congestion level, link quality, and hop distance. The Q value is continuously updated using existing RPL signaling mechanisms. The performance of the proposed approach is evaluated through extensive simulations and compared with existing work to demonstrate its effectiveness. The results show that the proposed method significantly improves network performance in terms of packet delivery and average delaywithasmallincreaseinsignalingfrequency.Inthis study, an RL-based routing approach was proposed to reduce node congestion and improve routing performance in RPL networks. Each node applies Qlearning to select a preferred parent based on a composite feedback function. The congestion level of each node is incorporated into the RANK metric and distributed using a modified trickle timer strategy Performance evaluations were conducted to prove the effectiveness of the proposed method to achieve load balancingandimprovenetworkperformanceintermsof packetdeliveryandaveragedelay.

Pengjun Wang et al.,[11] analyzed to the Deep learning-basedroutingoptimizationofawirelesssensor network is described, as is the neural network. This studyintroducesthesubjectofrouteoptimization,which is based on wireless sensor network dynamic programming, and then elaborates on its concept and related techniques, as well as develops and analysesthe caseofwirelesssensornetworkoptimization.According to the comparative analysis of the five algorithms in computer simulation, even if the average network delay performance of DPER reached 0.47 s, it could successfully extend the network's life cycle. The DPER method not only increases network life, but it also increases network energy utilization rate, shortens network average path length, and minimizes the standard deviation of the node's remaining energy. Wireless sensor networks installed on site or on each floor of a large building operate in a passive environmentforalongtime,whichlimitsthetimethata wirelesssensor network canoperate,iethelifetimeof a H. wireless sensor network. Therefore, it is very important to use existing science and technology to extendthelifespanofwirelesssensornetworksasmuch as possible so that wireless sensor networks can continue to function without stopping due to power issues. While routing in traditional networks is largely irrelevant to node power sharing, power efficiency of routing algorithms in wireless sensor networks is often

more important than finding the shortest path. This study mainly focused on node residual energy and energy uniformity, and proposed an energy routing algorithm for wireless sensor networks based on dynamic programming.In order to extend the life of the network, we use the idea of dynamic programming to generalizethedataasmuchaspossibleandfindefficient waystoensurethebalanceofenergyconsumptionofall network nodes. Due to limitations in study time, study conditions, and academic level, this study inevitably has some shortcomings, and further improvements are expected.

Jothikumar et al.,[12] proposed system an Optimal Cluster-Based Routing (Optimal-CBR), Power efficiency andnetworkresilienceareimprovedwithahierarchical routing approach for IoT applications in 5G environments and beyond. The Optimal-CBR protocol usesthek-meansalgorithmtoclusternodesandamultihop approach to chain routing. The clustering phase is called until two-thirds of the nodes are down, after whichtheconsolidationphaseiscalledfortherestofthe datatransfer.Nodesareclusteredusingabasick-means algorithm during the clustering phase, and the highest energy of the node closest to the centroid is chosen as the cluster head (CH). The CH gathers packets from its members and transmits them to the base station (BS). Because two-thirds of the nodes are dead and the leftover energy is insufficient for clustering during the chaining phase, the surviving nodes attempt multihop routingtoestablishchaininguntilthedataistransferred to the BS. This improves energy efficiency and network lifespan, as demonstrated by theoretical and simulation investigations. The k-means algorithm is used to form clusters in the Optimal Cluster-Based Routing (OptimalCBR) protocol for organising Internet of Things-based WirelessSensorNetworks.Achainingphaseisutilisedto establish a routing channel when the residual energy of the CH is less than the threshold energy. The cluster head is chosen based on the Euclidean distance and the residual energy of the node. Because the results of the simulation plots show that Optimal-CBR has a smaller vitality variance within CH and the total remaining entangled vitality is insignificant compared to the kmeans, CHIRON, and LEACH-C protocols. , the node will live longer. Expanded. Therefore, current schemes have the potential to evenly distribute power to all network nodes,maximizetransmissionrounds,andreducepower consumption in 5G setting. Therefore, the proposed system improves the efficiency of sensor nodes by minimizing the power consumption of the nodes and extendsthenetworklifetime.Thesystemdoesnotfocus on security aspects when sharing data via a multi-hop routingapproach.

Qianao Ding et al.,[13] produced a theoretical hypothetic model formulation of ML as a viable way for

constructing a power-efficient green routing model that can overcome the constraints of previous green routing methods. Furthermore, the study presents an overview of previous, current, and future progress in green routingsystemsinWSNs.Thisstudy'sfindingswillbeof interesttoawidespectrumofpeoplewhoareinterested in ML-based WSNs. This study expanded on traditional and ML approaches to designing green routing algorithms in WSNs. The study developed a mathematical hypothesis model of an ML-based routing algorithm for increasing the lifetime of WSNs based on comprehensive comparisons and analysis. Furthermore, essential principles and characteristics of various routing algorithms in WSNs are investigated in this study. The research also discusses the advantages and disadvantages of the many strategies that may be utilised to improve the performance of routing algorithms in WSN. Finally, this study discusses the problems of using ML for routing algorithm creation in WSNs, as well as future research objectives that should be addressed and explored with ML. This debate will be ofinteresttoawiderangeofpeopleinterestedinMLand WSNs. Futureresearchworkmayincludethefollowing: Due to the arithmetic bottleneck and energy consumption limitation of WSNs, ML algorithms cannot bedeployedatscaleinsensorswithsmallcomputational powerandlimitedenergy.However,distributedlearning methods require less computational capacity, energy consumption, and smaller memory footprints than centralizedlearningalgorithms(i.e.,theydo not need to consider the entire network information). Distributed cooperative learning overcomes the arithmetic bottleneck, resulting in ML-based green routing with lower energy usage, which is ideal for WSNs. ML techniques, on the other hand, require a significant amount of computation and energy for effective parameter learning during the training learning phase, making their implementation in WSNs highly difficult. Furthermore, nodes have extremely diverse computational capabilities (for example, the sink has a large computational power, whilst the rest of the nodes arepoor),whichgivesusideasforhowtoapplytransfer learning in WSNs. Sink nodes can train the model in a distributed manner with other sensor nodes because they have more power and computational capacity. The trainedmodelparameterscanthenbetransmittedtothe sensor nodes using sink nodes. After receiving model parameters from various nodes, the sensor nodes multiplytheseparametersbythecorrespondingweights and perform a weighted average to obtain their final model parameters, which reduces sensor node training consumption and improves model accuracy in WSNs. Meanwhile,dealing with QoS-aware routingisa difficult task. Satisfying delay restrictions, bandwidth limits, and applying machine learning techniques to design a routing protocol is an intriguing area of research,

particularly when applied to WSNs using hybrid ML techniques.

Sapna Chaudhary et al.,[14] established a wireless networkalgorithmcalledOptimizedRoutinginWireless Networks Using Machine Learning (ORuML). Network type of source and destination nodes. ML models are trained using real-time collected node features such as: B. Battery power usage, available internal storage, node IP addresses and ranges. Intuitively, MLR should outperformANNandSVMintermsofaccuracyandarea under the ROC curve (AUC). The proposed algorithm determines whether the source and destination nodes areco-located,andalsodeterminesthebestneighboring hops for efficient routing. In this research paper, a new HisORuMLalgorithmforwirelessnetworksisproposed. Machine learning techniques, namely ANN, MLR and SVM, were applied to the NNCF dataset to predict the network type of nodes. Simulations highlight the fact that MLR outperforms KNN and SVM in terms of accuracy and AUC. make a prediction. In addition, the selection of optimal neighbors for efficient routing in MANET or DTN was simulated. As a future extension of current research work, it is proposed that machine learning algorithms such as decision trees and random forests can be applied to specific datasets with network node (NNCF) characteristics. Apart from that, we can apply deep learning to specific datasets and extend the same work to cellular networks. Machine learning can also be used to route messages in opportunistic networks.

Fang Wang et al., [15] Q-Routing is a highly random network environment, and overestimation of values leads to poor performance. To solve this problem, this research proposes an algorithm called Delayed QRouting(DQ-Routing).Itusestwosetsofvaluefunctions to perform random delayed updates to reduce value function overestimation and improve the convergence rate. Experiments have shown that the DQ routing algorithm works well for some problems. Traditional routing algorithms cannot dynamically adapt their routing strategies to network fluctuations and are not applicable to increasingly complex network environments. Q-routing algorithms can dynamically adjust their strategy, but overestimation of the value function leads to poor performance in high-random networks.Inthisstudy,weproposedaraggedQ-routing algorithm that avoids overestimation of the value function using a ragged estimator. Experiments show that DQ routing not only avoids overestimation of the valuefunction,butalsoimprovesthelearningrate.

Mahima.v et al.,[16] considerations for Algorithm

Development RL-EM outperforms existing algorithms with 40% fewer sleeping nodes and 60% fewer energy overflows using the LEACH protocol. The proposed RL-

EM algorithm outperforms LEACH, SEP-M and ACO algorithms in terms of throughput, sleep nodes, power spill and load balancing. Node planning in this research work is done by the RL approach and compared to biologically-inspired algorithms such as ACO and classical probability-based algorithms such as LEACH and SEP-M approaches. The proposed work provides better power management with 40% fewer sleeping nodes and 60% fewer network power overflows comparedtotheLEACHbenchmarkprotocol.Theresults of the proposed RL-EM algorithm show even load distribution.Theinvestigativeeffortalsousesmaximum resourcestoachievethegoalofsuccessfullytransferring data to the sink. RL-EM avoids energy holes and HOTSPOT problems in networks that other comparison algorithms cannot solve. This algorithm appears to be a new solution to the power management problem in wirelesssensornetworks.

Maitreyi Ponguwala et al.,[17] MANET-based trust computation techniques were introduced. However, incorrect trust value computation lowers mitigation schemeperformance.Thus,theprimarygoalofthiswork is to create a novel security method to secure the MANET-IoT from various threats. In this study, a novel group-based routing algorithm with suggestion filtering was suggested, which was backed by security monitors (SMs). The unsupervised machine learning approach is applied for network recommendation filtering. The Secure Certificate-based Group Formation (SCGF) mechanism initially groups the whole network. To compute trust in each group, the Recommendation Filtering by K-means technique (RF-K means) is used. A hybridoptimisationtechniquethatcombinestheGenetic technique and the Fire Fly Algorithm (GA-FFA) is developed for secure and optimal route selection. Data transmission is protected by a new hashed message authentication code using the High Encryption Standard (HMAC-AES) algorithm, where the hash function is integratedwiththeencryptionfunction.TheMANET-IoT network is secured by a grouping and trust filtering approach. In addition, built-in encryption functions ensure data security and hashing functions ensure data integrity.

Vially Kazadi Mutombo et al.,[18] establishedEER-RL, an energy-efficient routing protocol based on reinforcement learning. Reinforcement learning (RL) enables devices to adapt to network changes, such as mobilityandpowerlevels,toimproveroutingdecisions. The performance of the proposed protocol is compared withotherexistinglow-powerroutingprotocolsandthe results show that the proposed protocol is superior in terms of power efficiency, network robustness and scalability.Inthisstudy, a cluster-based energy-efficient routing protocol for IoT using reinforcement learning called EER-RL was proposed. The aim of this work was

tooptimizeenergyconsumptionandextendnetworklife by finding the best routes for data transmission. Research has produced two versions of the same algorithm. One is cluster-based (EER-RL) and the other is flat-based (FlatEER-RL). More scalable than flat base. However, for small networks it is recommended to use the flat version of the proposed work. EER-RL was designed in three phases, including network construction and CH election. In this study, considering thehopcountfactorandinitialenergy,wecalculatedthe initial Q value used for CH selection at this stage. The second step was to form clusters. Each CH sent an invitation to all devices within its coverage area, and eachdevicefarfromthebasestationjoinedtheclusterto which the CH was closest. Finally, the data transmission phase, characterized by learning, considered both the device'sresidualenergyandthenumberofhopstomake routing decisions, providing energy-efficient routing. Additionally, an energy threshold for CH substitution was specified. Simulation results showed that EERRL achieved better power consumption and network robustness than his LEACH and PEGASIS. In this study, we used lightweight RL to reduce protocol latency and minimize power consumption. In the future, we would like to explore additional parameters for more optimal routingprotocols.

Yuvaraj Natarajan et al.,[19] upgradable cross-layer routingprotocolbasedonCR-IoTwasdescribedinorder to improve routing efficiency and optimise data transmissionina reconfigurablenetwork. Thesystemis buildingadistributedcontrollerinthiscontext,whichis created with many activities such as load balancing, neighbourhood sensing, and machine-learning path development. On an average of 2 bps/Hz/W, the proposedtechniqueisbasedonnetworktrafficandload, as well as several other network parameters such as energy efficiency, network capacity, and interference. The testing are carried out using conventional models, demonstrating the reconfigurable CR-IoT's residual energy and resource scalability and robustness. In this study,amachinelearningassistedcrosslayerroutingina reconfigurable CR IoT application was built.The distributedcontrollersensestheentireenvironmentand generatesCR-IoTdatatomakethebestchoiceforcluster member selection, cluster head selection, and routing path selection. Such ideal selection permits actual data transferbytakingintoaccountthefullnatureofCR-IoTs, i.e., its properties. The use of three phases with the assistanceofMLimprovesthisdecisionbyreconfiguring clusters, thresholds, and CR-IoT based on network requirements. The advantages include making the best use of the CR-IoT network with a controller to enable optimal data transfer with enhanced network level heterogeneity. The cross-layered technique allows for layer cooperation with periodic reconfiguration during the routing process. During the settling phase, the

distributed controllers with mutual coordination optimise the processes and therefore the routing paths are deployed. This cross-layer network routing operation strategy improves energy efficiency, network stability, and resource utilisation. The research is also expanded to include the routing operation with channel imperfection effects in heterogeneous cooperative CRIoTs. Furthermore, the ability to add cloud storage for routing operations would no longer be considered a constraintwhenchangingtheroutingtable.

Mohamed Mira et al.,[20] research focuses on the effectiveness evaluation of the previously indicated selection path mechanisms. In addition, as a considered factor for this evaluation, research will manage two important parameters: power consumption and packet delivery ratio. Different scenarios will be used to simulate what research can find in real-world applications. Finally, research will suggest a strategy based on machine learning contributions to assist us in node deployment according to the specifications of our airport application that research wishes to develop. Several simulations were performed in this study to demonstrate a clear impact on RPL performance by adjusting the transmitting intervals. The machine learning approach used in the previous part validated our findings. However, research should be conducted to develop a prediction model that aids in the deployment ofnodeswiththebestobjectivefunctionbasedoninput values such as NN, TX, and SI. Furthermore, the predictive model develops an academic technique that canbeappliedandusedinreal-worldapplications,such asourusecaseofcontrollingpassengerflowatairports. The researchersbelievethattheML methodestablished in this study is an excellent starting point for constructing an automated procedure that could be included in any IOT application's data processing section. Future work will focus on how to construct a customer algorithm that collects all of the limitations that can occur in the everyday operation of an IOT applicationanddetermineswhichOFismoresuitable.

E. Laxmi Lydia et al.,[21] determined a novel green energy efficient routing approach for IoT applications using DL-based anomaly detection (GEER-DLAD). The provided architecture enables IoT devices to effectively use energy in order to expand network span. To reduce the amount of data transfer across the network, the GEER-DLADapproachemploysErrorLossyCompression (ELC). In addition, the moth flame swarm optimisation (MSO) algorithm is used to optimise network path selection. Furthermore, the DLAD method detects anomalies in IoT communication networks using the recurrent neural network-long short term memory (RNN-LSTM)model.Athoroughexperimental validation procedure is followed, with the findings ensuring that the GEER-DLAD model improves in terms of energy

efficiency and detection performance. In This study created an effective GEER-DLAD approach for IoT applications. The provided methodology enables IoT devices to optimally exploit energy in order to increase network span. Initially, IoT devices continuously collect data. They are then compressed using the ELC process. Following that, the IoT devices used the MSO algorithm to execute the routing approach and determine the best path to the destination. The compressed data will be forwardedtowardsthetimeaftertheroutesare chosen. The DLADapproachisused during data transmission to detect the appearance of anomalies in IoT communication networks. A thorough experimental validation process is carried out, with the findings ensuring the improvement of the GEER-DLAD model intersofenergyefficiency.

Anjaneya Tripathia et al.,[22] using a neural network and real-time measurements, the Optimal Routing with Node Prediction (ORNC) algorithm has been developed topredictthebestrouteforpackets.Theneuralnetwork is trained on node characteristics such as internal storage availability, IP address, battery consumption, and node range. For MANET or DTN network types, an optimal routing algorithm is implemented after categorization. The effectiveness of the approach is comparedtothatofK-nearestneighbour(KNN),support vector machine (SVM), and multinomial logistic regression(MLR).TheORNCalgorithm'smajorgoalisto classify networks as MANET, DTN, or Bluetooth using machinelearningtechniquesasKNN,SVM,MLR,andNN. Thisclassificationaidsindeterminingwhetherornotto use collocation and thus the optimal routing algorithm. The node class type and trust factor are then used to calculate its fitness value, which determines how well it is suited to serve as an intermediate node. The neural network model is determined to be superior among the four ML techniques used in classifying networks as MANET, DTN, or Bluetooth. With this higher accuracy, theoptimalhopselectionmaytakeplacewithincreased efficacy.Thesemachinelearningmethodscanbeutilised to route messages in an opportunistic network as a futureexpansionofthestudywork.

Davi Ribeiro Militani et al.,[23] e-RLRP is a research enhanced routing system based on RL that reduces overhead. To compensate for the overhead incurred by the use of RL, a dynamic modification in the Hello message interval is introduced. Various network scenarios with varying numbers of nodes, routes, traffic flows,anddegreesofmobilityareimplementedinorder to acquire network characteristics such as packet loss, latency, throughput, and overhead. A Voice-over-IP (VoIP) communication scenario is also implemented, with the E-model method used to forecast connection quality. The OLSR, BATMAN, and RLRP protocols are usedtocompareperformance.Experimentresultsreveal

that e-RLRP minimises network overhead compared to RLRP and, in most circumstances, outperforms all of these protocols when network parameters and VoIP are considered.

The experimental results in this study show that a routing protocol based on RL outperforms existing protocolssuchasBATMANandOLSR,notablyinPpland throughputmetrics.These network performance results demonstrate the utility of RL-based routing algorithms for improving computer and ad-hoc networks. The RL approach, on the other hand, adds considerable overhead. As a result, the suggested and developed adjustment method reduced network overhead by reducing the amount of control messages. In terms of throughputand Ppl, the e-RLRPperformed betterin the majority of the test situations employed in this work, particularly with regard to the Ppl parameter. As a result, it has been proved that the proposed technique minimizes overhead while simultaneously improving network conditions. Reducing network overhead in traditional protocols is an essential strategy since it improves performance. This method is even more essentialwhennovelroutingalgorithms, suchasRL,are utilised to increase network performance while generating more overhead. As a result, an essential contribution of this study is to show how the proposed dynamic modification function can be used to eliminate additional overhead. It is worth noting that several network topologies and settings were used in our experimental tests, including varying numbers of nodes and their drops, as well as different numbers of traffic flows. As a result, it is possible to conclude that RLbasedroutingalgorithmshaveaconsiderablefavourable impact on the QoE of users in real-time communication services. As a general conclusion, this study emphasises the importance of embedding machine learning algorithmsintoroutingprotocols,particularlyforadhoc networks that frequently experience node drops. RLbased routing protocols can help to enhance network conditions, which in turn improves many communication applications. Only the VoIP service is analysed in this study; however, video communication services will be tested in future studies. Furthermore, the established dynamic adjustment method in the sending of Hello messages improved network performance, mostly by lowering overhead, which is an important approach to use in RL-based routing protocols.

Mohammad Shoab et al.,[24] DRL is highly suited to solving optimisation problems in distributed systems in general,andnetwork routinginparticular.Asa result,a novel query routing strategy termed Deep Reinforcement Learning based Route Selection (DRLRS) based on a Deep Q-Learning algorithm is suggested for unstructured P2P networks. The major goal of this

strategy is to improve retrieval efficacy while lowering searchcostsbyreducingthenumberofconnectedpeers, messagessent,andtimespentsearching.Inthisstudy,a novelstrategytoefficientlyroutingthequerytodiscover the relevant resource was provided. In this context, the query routing problem has been presented as a deep reinforcementlearningproblem, with a fullydistributed method devised to solve it. As a result, the Deep QLearning-based query routing algorithm DRLRS is introduced to intelligently select neighbours in order to increase the performance of the P2P network. The results show that the DRLRS learns from past queries andefficientlyidentifiesthebestneighboursholdingthe relevant resource for the current query. The suggested approach consistently improves retrieval efficacy and search cost, outperforming the k-Random Walker and DirectedBFS.

Meisam Maleki et al.,[25] presented a bi-objective intelligent routing protocol with the goal of lowering an estimated long-run cost function comprised of end-toend delay and path energy cost. The routing problem is formulatedasaMarkovdecisionprocess,whichcaptures both the link state dynamics caused by node mobility and the energy state dynamics caused by node rechargeable energy sources. This research proposed adaptive perturbation strategy are a multi-agent reinforcement learning based algorithm to approximate the optimal routing policy in the absence of a priori knowledge of the system statistics. The suggested algorithm is based on model-based RL concepts. More specifically, develop a cost function for each node by deriving an equation for the expected value of end-toend expenses. The transition probabilities are also calculated live using a tabular maximum likelihood technique. According to simulation results, our modelbased approach outperforms its model-free version and behaves similarly to standard value-iteration, which assumesperfectstatistics.Thebi-objectiveissueofdelay andenergyefficientroutinginenergycollectingMANETs was tackled in this study. Research shows how nodes choosenext-hoprelaysinthepresenceoflinkandpower dynamics to optimize both end-to-end latency and network lifetime in the long run. To explicitly consider thestochasticdynamicsofthenetworkenvironment,the studymodeledtheroutingproblemasaMarkovdecision process (MDP). In this work, we also proposed an algorithmbasedonmodel-basedreinforcementlearning (RL) to approximate the optimal routing policy of the formulated MDP. Specifically, the researchers modeled thecostfunctionofeachnodebyderivinganexpression for the expected end-to-end cost. Also, transition probabilitiesareestimatedonline,soanodecanperform multiple updates after his single interaction with the system. Moreover, the multi-agent base of the proposed RL method enables global system optimization. As proved by simulations, the proposed scheme converges

muchfasterandconvergestobettervaluesofthesystem goalcomparedtothemodel-freesolution.

Wenjing Guo et al.,[26] proposed the RLLO intelligent routingalgorithm.RLLOemploysreinforcementlearning (RL) to define the reward function while taking into account remaining energy and hop count. It attempts to distribute energy more equally and enhance parcel delivery at no extra expense. This proposed algorithm was evaluated in comparison to Energy Aware Routing (EAR) and Enhanced EAR (I-EAR). The simulation results reveal that RLLO outperforms these two algorithms in terms of network resilience and packet delivery. To extend network lifetime, an intelligent routing algorithm RLLO for WSNs was proposed in this study. Network lifespan is a critical factor for determining the performance of network algorithms in WSN. RLLO takes advantage of the benefits of RL to accomplish global optimisation at no extra expense. To balance energy usage and improve package delivery, define a round reward function. NS2 was used to validatethisapproach.Intermsofthetimethefirstnode losespower,thetimethenetworkcannotexecutepacket delivery, and the number of packet deliveries, RLLO outperformsEARandI-EAR.Theinherentadvantagesof RL algorithms make them suited for dealing with dispersed challenges. A RL-based algorithm was proposed in this work to provide fresh ideas and incentives for solving routing challenges in WSNs. However, there are still concerns that must be addressed. To begin, numerous RL-based WSN routing algorithms have been developed to increase specific performance. The objectives of these RL-based algorithms differ. As a result, it is difficult to compare these methods explicitly. Second, in simulation trials, RLLOhasdemonstratedtobesuperior.Futureworkwill be done to evaluate this method in real-world WSN contextsincludingtestbedsanddeployments.

Dr. Joy Iong Zong Chen et al.,[27] for enhancing the convergence speed and ant searchability, research has devised termination requirements for the algorithm as wellasanadaptiveperturbationtechnique.Thispermits the discovery of the global best solution. The suggested routing design methodology improves network performance, network life cycle, energy distribution, node equilibrium, network delay, and network energy consumption. The Internet of Things networks, which include wireless sensors and controllers as well as IoT gateways, provide incredibly high functionality. However, little attention is made to optimising the energyconsumptionofthesenodesandenablinglossless networks.Withtheadvancementofartificialintelligence and the popularisation of machine learning, wireless sensor networks and their applications have gradually industrialised and scaled up. Due to the high energy consumption difficulties associated with the routing

method, the algorithm protocol achieves the uneven network node energy consumption and local optimum. For providing energy balanced routing at needed locations,thesmartantcolonyoptimisationtechniqueis applied. Based on the smart ant colony optimisation method, a neighbour selection technique is proposed by merging wireless sensor network nodes and energy variables. The development of smart devices and technology has resulted in remarkable growth in the field of IoT. This can be avoided by employing a clever optimisation technique, such as the ACO, and doing dynamic optimisation. The network's energy consumption is optimised, and a global optimal solution is obtained by separating the areas based on delay and energy utilising smart ant colony optimisation and a routing protocol. Machine learning is utilised to investigate energy patterns and reduce energy consumption in these networks. The jump probability is combined based on the node location and the node transmission region, which is divided in order to investigatepotentialnodeswhiletakingintoaccountthe delayandenergyofthedividedarea.Theglobaloptimal solution is generated by combining the method's termination criteria, adaptive perturbation technique, neighbour selection strategy, and wireless sensor network with the smart ant colony optimisation algorithm. PSO, ABC, SRT, LRT, and ACO extension algorithms are compared with the ACO for network performancemeasuresandenergydelaymetricanalysis. The efficientimplementationof thisalgorithm resultsin balanced energy consumption as well as network life cycle extension. The sensor network's delay and energy consumption are taken into account, and the network node's energy consumption is balanced. Even in large sizenetworks,severalmobileagentsovercomethedelay andenergydifficulties.

Y. Liu et al.,[28] developed a distributed as well as energy-efficient reinforcement learning (RL) based routing system for the wide area wireless mesh IoT networks. They compare the failure rate, spectrum, and powerefficienciesoftheproposedalgorithmtothatofa random routing with loop-detection algorithm and a centralised pre-programmed routing algorithm, which represents the ideal-scenario, using simulations that resemble long-range IoT networks. They also show progressiveresearchtodemonstratehowthelearningin the algorithm reduces the overall network's power usage. Significant increases in power efficiency, failure rate, and spectrum efficiency have been achieved by utilising RL for routing in IoT/M2M energy sensitive mesh networks. With increasing network scale, the advantage over routing algorithms without learning capability becomes increasingly substantial. Furthermore, the method's significant gain in performance in comparison to the slight addition of complexityclearlyillustratesitspotential.

III.CONCLUSION

The IoT consists of many different devices that are interconnected and transmit vast amounts of data. The solution starts by collecting data from the nodes and preprocessing it to create a dataset that can be used during the training phase. During the training phase, a decision model is created using machine learning algorithms. This study covers various research papers relatedtomachinelearningalgorithmsforroutinginIOT using deep learning algorithms, neural network algorithms, deep learning algorithms, deep reinforcement learning algorithms, convolutional neural networks, and support vector machine algorithms was discussed. In this routing, each node sends its routing decisions to a specific server for all routing operations performed using conventional routing. The data sent includes the decision made and properties such as date and time, source node, destination node, and previous node.Givenenoughtime,theservercanusethecollected datatobuildadecisionmodelforeachnodeandmakea large number of routing decisions without using traditional routing. We believe such a framework can reduce the resources used for routing and avoid its use entirely.

IV.REFERENCES

[1]. Ashton, K. (2009), “That ‘internet of things’ thing”,RFiDJournal,Vol.22,No.7,pp.97-114.

[2]. Rawat,P.,Singh,K.D.,Chaouchi,H.,andBonnin, J. M. (2014), “Wireless sensor networks: a survey on recent developments and potential synergies”, the Journal of supercomputing, Vol. 68,No.1,pp.1-48.

[3]. Bello, O., and Zeadally, S. (2016), “Intelligent device-to-device communication in the internet ofthings”, IEEE,Vol.10,No.3,pp.1172-1182.

[4]. L. Xiao, X. Wan, X. Lu, Y. Zhang and D. Wu, "IoTSecurity Techniques Based on Machine Learning:HowDoIoTDevicesUseAItoEnhance Security?," in IEEE SignalProcessing Magazine, vol.35,no.5,pp.41-49,Sept.2018.

[5]. M. Injadat, A. Moubayed and A. Shami, "Detecting BotnetAttacks in IoT Environments: An Optimized MachineLearning Approach," 2020 32nd International Conferenceon Microelectronics(ICM),2020,pp.1-4.

[6]. Cui,L.;Yang,S.;Chen,F.;Ming,Z.;Lu,N.;Qin,J.A survey on application of machine learning for Internet of Things. Int. J. Mach. Learn. Cybern. 2018,9,1399–1417.[CrossRef]

[7]. Robert Basomingera and Young-June Choi ”Learning from Routing Information for Detecting Routing Misbehavior in Ad Hoc Networks:, Sensors 2020, 20, 6275; doi:10.3390/s20216275

[8]. Murtadha M.A.Alkadhmi, OsmanN.Uçan,and Muhammad Ilyas,”An Efficient and Reliable Routing Method for Hybrid Mobile Ad Hoc Networks Using DeepReinforcementLearning”, Hindawi Applied Bionics and Biomechanics Volume2020,ArticleID8888904,13pages

[9]. Saeed Kaviani, Bo Ryu1, Ejaz Ahmed, Kevin Larson, Anh Le, Alex Yahja, and Jae H. Kim,”Robust and Scalable Routing with MultiAgent Deep Reinforcement Learning for MANETs”,

[10]. Hossam Farag and Cˇ edomir Stefanovic, ”Congestion-Aware Routing in Dynamic IoT Networks:AReinforcementLearningApproach”

[11]. Pengjun Wang , Jiahao Qin, Jiucheng Li, Meng Wu, Shan Zhou, and Le Feng,”Dynamic Optimization Method of Wireless Network Routing Based on Deep Learning Strategy”, Hindawi Mobile Information Systems Volume 2022,ArticleID4964672,11pages

[12]. C. Jothikumar, Kadiyala Ramana , V. Deeban Chakravarthy ,Saurabh Singh,”An Efficient Routing Approach to Maximize the Lifetime of IoT-Based Wireless Sensor Networks in 5G and Beyond, Hindawi Mobile Information Systems Volume2021,ArticleID9160516,11pages

[13]. Qianao Ding, Rongbo Zhu, Hao Liu, Maode Ma ,”An Overview of Machine Learning-Based Energy-EfficientRoutingAlgorithmsin Wireless SensorNetworks”,

[14]. Sapna Chaudhary, Rahul Johari,”ORuML: Optimized Routing in wireless networks using MachineLearning”,

[15]. Fang Wang, Renjun Feng and Haiyan Chen,”Dynamic Routing Algorithm with Qlearning for Internet of things with Delayed Estimator”, IOP Conference Series: Earth and Environmental Science, 234 (2019) 012048 IOP Publishing

[16]. V Mahima, A. Chitra ,” Reinforcement Learning Based Energy Management (Rl-EM) Algorithm For Green Wireless Sensor Network, ICCAP 2021,December07-08,Chennai,IndiaCopyright

© 2021 EAI DOI 10.4108/eai.7-122021.2314541

[17]. Maitreyi Ponguwala, DR.Sreenivasa Rao ,” SecureGroup basedRoutingandFlawlessTrust Formulation in MANET using Unsupervised Machine Learning Approach for IoT Applications, EAI Endorsed Transactions on EnergyWeb,2019,

[18]. Vially Kazadi Mutombo , Seungyeon Lee, Jusuk Lee, and Jiman Hong,”EER-RL: Energy Efficient Routing Based on Reinforcement Learning”, EER-RL: Energy-Efficient Routing Based on Reinforcement Learning, Hindawi Mobile Information Systems Volume 2021, Article ID 5589145,12pages

[19]. Yuvaraj Natarajan,”An IoT and machine learning-based routing protocol for reconfigurable engineering application”, 2021, IETCommunications

[20]. MohamedMira,MohamedHechmiJeridi,Djamal Djabour and Tahar Ezzedine,”Optimization of IoT Routing Based on Machine Learning Techniques. Case Study of Passenger Flow Control in Airport 3.0”, 978-1-5386-91311/18/$31.00©2018IEEE

[21]. E.LaxmiLydia,A.ArokiarajJovith”GreenEnergy Efficient Routing with Deep Learning Based Anomaly Detection for Internet of Things (IoT) Communications”, Mathematics 2021, 9, 500. https://doi.org/10.3390/math9050500

[22]. Anjaneya Tripathia, Vamsi Kiran Mekathotia, Isha Waghuldea, Khushali Patela,”Neural Network aided Optimal Routing with Node Classification for Adhoc Wireless Network”, Workshop on Computer Networks & Communications, May 01, 2021, Chennai, India, 2021

[23]. Davi Ribeiro Militani 1 , Hermes Pimenta de Moraes 1,”Enhanced Routing Algorithm Based on Reinforcement Machine Learning A Case of VoIPService”,2021,Sensors2021,21,504.

[24]. Mohammad Shoab and Abdullah Shawan Alotaibi*,”Deep Q-Learning Based Optimal Query Routing Approach for Unstructured P2P Network”,Computers,Materials&ContinuaTech SciencePressDOI:10.32604/cmc.2022.021941

[25]. MeisamMaleki,VesalHakami,MehdiDehghan,”A Model-BasedReinforcementLearningAlgorithm

forRoutinginEnergyHarvestingMobileAd-Hoc Networks”, Wireless Pers Commun (2017) 95:3119–3139

[26]. Wenjing Guo, Cairong Yan , Yanglan Gan,”An IntelligentRoutingAlgorithminWirelessSensor Networks based on Reinforcement Learning”, Applied Mechanics and Materials Vol. 678 (2014)pp487-493

[27]. Dr. Joy Iong Zong Chen, Kong-Long Lai,” MachineLearning basedEnergyManagementat Internet of Things Network Nodes, Journal of trends in Computer Science and Smart technology (TCSST) (2020) Vol.02/ No. 03 Pages:127-133

[28]. Y. Liu, K.-F. Tong and K.-K. Wong,”Reinforcement learning based routing for energy sensitive wireless mesh IoT networks”, CORE Metadata, citation and similar papersatcore.ac.uk