Malicious Webpage Detection Using Machine Learning: A Comprehensive Review by IRJET Journal

Malicious Webpage Detection Using Machine Learning: A Comprehensive Review

Mayank Singh, Md Absarul Haque, Dr. InderPreet Kaur

Sharda School of Engineering and

Technology,

Sharda University, Greater Noida, India

Abstract- Traditional, signature-based methods for detecting malicious webpages, such as blacklisting, are increasingly ineffective against the dynamic and sophisticated nature of modern web threats. These methods struggle to identify newly generated or obfuscated malicious URLs, creating a significant security gap. This review paper provides a comprehensive analysis of machine learning (ML) based approaches, which have emerged as a powerful alternative for proactive and accurate malicious webpage detection. We explore the evolution of these systems, from classical algorithms like Random Forest and Support Vector Machines to advanced deep learning models that provide robust, real-time classification. This paper surveys the critical role of feature engineering, categorizing features into lexical, content-based, behavioral, and network-related types, and examines their impact on model performance. A significant portion of this review is dedicated to the critical challenges of model generalizability across different datasets, class imbalance, and the growing threat of adversarial attacks designed to evade ML-based detectors. We synthesize findings from a broad range of studies to identify the current state-of-the-art, pinpoint existing research gaps, and suggest future directions for developing more effective, scalable, and adversarially robust detection systems. A conceptual hybrid framework is presented as a next-generation solution that addresses many of the limitations of existing single-model systems.

Keywords: malicious webpage detection, machine learning, phishing detection, URL classification, feature engineering, deep learning, adversarial attacks, random forest, support vector machine, cybersecurity, web security, convolutional neural network, LSTM, ensemble methods, blacklist evasion

I. INTRODUCTION

The exponential growth of the internet has made it an indispensable tool for communication, commerce, and information access.However,thisproliferationhasbeenaccompaniedbyasurgeinweb-basedcyberattacks,includingphishing,malware distribution, and drive-by-downloads [1]. Malicious webpages serve as the primary vector for these attacks, posing a significantthreattousersecurityandprivacy.

Historically, the primary defense mechanism has been the use of blacklists curated lists of known malicious URLs. While simple to implement, blacklist-based methods are inherently reactive. They are limited in their ability to detect zero-day threats, newly generated malicious URLs, or pages using cloaking techniques [2]. This limitation has driven the research communitytoseekmoredynamicandintelligentsolutions.

Theapplicationofmachinelearning(ML)hasmarkedaparadigmshiftinthisdomain[3].MLmodelscanlearntodistinguish betweenbenignandmaliciouswebpagesbyanalyzingawidearrayoffeatures,enablingthedetectionofnovelthreatswithout priorknowledge[4].EarlysystemsutilizedclassicalMLalgorithms,whichprovedsignificantlymoreeffectivethantraditional methods.Morerecently,deeplearningandensembletechniqueshavefurtheradvancedthefield,achievingexceptionallyhigh accuracy rates [5]. These advanced systems leverage complex feature sets ranging from URL lexical patterns to webpage contentandnetworktrafficdata tobuildrobustandadaptabledetectionmodels[6].

Thisreviewpaperaimsto providea comprehensiveoverviewofthefieldofML-basedmaliciouswebpagedetection. Wewill examine the key methodologies, feature engineering techniques, and prevalent challenges, positioning a proposed hybrid frameworkwithinthisevolvinglandscape.

II. METHODOLOGY FOR LITERATURE SEARCH

A systematic literature search was conducted to identify relevant studies on machine learning for malicious webpage detection. The search was performed across several academic databases, including IEEE Xplore, ACM Digital Library, ArXiv,

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

and Google Scholar, supplemented by the Consensus search engine which covers sources like Semantic Scholar and PubMed [7].

The search terms included, but were not limited to: "malicious webpage detection," "phishing detection using machine learning," "malicious URL detection," "feature engineering for web security," "deep learning for cybersecurity," and "adversarialattacksonwebclassifiers."

Thefollowinginclusioncriteriawereappliedtoscreenthearticlesforrelevance:

• ThestudyhadtobepublishedinEnglish.

• ThestudyhadtofocusondetectingmaliciouswebpagesorURLsusinganMLordeeplearningcomponent.

• Thestudyhadtoaddressoneormoreofthefollowing:featureengineering,modelarchitecture,comparativeanalysis, real-worlddeployment,oradversarialrobustness.

Thereferencesofselectedarticleswerealsoscannedtoidentifyadditionalrelevantstudies.Thisiterativeprocessresultedina comprehensivecollectionofover60papersthatformthebasisofthisreview.Thefinalselectionwasthencategorizedbased onprimaryfocustofacilitateastructuredanalysis.

III. COMPARATIVE ANALYSIS OF FEATURE ENGINEERING TECHNIQUES

The performance of any ML-based detection system is critically dependent on the quality and relevance of the features extractedfromthewebpageoritsURL[8].Thesefeaturescanbebroadlycategorizedintofourtypes.

• LexicalFeatures:ThesefeaturesareextracteddirectlyfromtheURLstringwithoutaccessingthewebpage'scontent. They are computationally inexpensive and effective for real-time analysis. Common lexical features include URL length, number of dots, presence of special characters ('@', '-'), use of IP addresses in the domain name, and characteristicsofthehostnameandpath[9].

• Content-Based Features: These features are derived from the raw HTML, JavaScript, and text of a webpage. They provide deep insight into the page's purpose and functionality. Examples include the use of suspicious HTML tags (e.g.,<iframe>),obfuscatedJavaScriptcode,keywordfrequencyanalysisusingTF-IDForWord2Vec,andthenumber ofexternallinks[10].Whilepowerful,thesefeaturesaremoreresource-intensivetoextract.

• Behavioral/Network Features: These features relate to the hosting environment and network interactions of the webpage. They include domain registration information (e.g., domain age from WHOIS records), IP address reputation, geographical location of the server, DNS records, and HTTPS certificate details [11]. These features are highlyeffectiveatidentifyingtransientornewlyset-upmaliciousdomains.

• Visual Features: A more recent approach involves using computer vision to analyze screenshots of webpages. Techniques like perceptual hashing or CNN-based feature extraction can identify visual similarities between a suspiciouspageandalegitimateone,whichisparticularlyeffectivefordetectingphishingattacksthatclonepopular websites.

Recentstudiesconsistentlydemonstratethatcombiningmultiplefeaturetypesleadstothemostrobustandaccuratemodels, asitprovidesamoreholisticviewofthepotentialthreat[12].

IV. MACHINE LEARNING MODELS FOR MALICIOUS WEBPAGE DETECTION

A. Classical Machine Learning Models

A wide range of classical supervised algorithms have been successfully applied to this problem. Commonly used models include Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (k-NN), Decision Trees, Logistic Regression,andNaïveBayes[13].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Comparative studies have repeatedly shown that ensemble methods, particularly Random Forest, consistently achieve high accuracyandarerobusttonoisydata[14].Theirabilitytohandlehigh-dimensionalfeaturespacesmakesthemwell-suitedfor thistask.

B. Deep Learning Models

With the availability of large datasets, deep learning has emerged as a state-of-the-art approach. Deep Neural Networks (DNNs) can automatically learn complex patterns and feature interactions from raw data, often outperforming classical models[15].

Specific architectures have been adapted for different feature types: Convolutional Neural Networks (CNNs) are used for analyzing webpage screenshots (visual features), while Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM)networksareeffectiveforprocessingsequentialdatalikeURLstringsandHTMLcode.

C. Ensemble and Multi-Modal Approaches

To further enhance performance, researchers have focused on ensemble and multi-modal frameworks. Ensemble methods combine the predictions of multiple individual models to improve overall accuracy and reduce variance [16]. Multi-modal deeplearningintegratesdifferenttypesoffeatures(e.g.,lexical,content,andvisual)intoaunifiedmodel,allowingittocapture a more comprehensive representation of the webpage and achieve superior detection rates [17]. These sophisticated approachesoftenreportaccuraciesexceeding98%onbenchmarkdatasets[18].

V. CHALLENGES AND ADVERSARIAL THREATS

Despitehighreportedaccuracies,severalsignificantchallengespersistinthefield.

• Model Generalizability: A major issue is the poor generalizability of models across different datasets [19]. A model trained on a specific data distribution may underperform significantly when deployed in a new environment with differentwebtrafficpatternsorattacktypes[20].Thishighlightstheneedformorediverseandstandardized public datasetsforbenchmarking[21].

• Class Imbalance: In the real world, benign webpages vastly outnumber malicious ones. This severe class imbalance canbiasMLmodelstowardsthemajorityclass,leadingtoahighnumberoffalsenegatives(missedthreats),whichisa critical security failure [22]. Techniques like oversampling (e.g., SMOTE) or undersampling are required to mitigate thisissue.

• Adversarial Attacks: As ML-based detectors become more common, attackers are actively developing techniques to evade them. Adversarial attacks involve creating carefully crafted, malicious webpages that are misclassified as benignbythemodel[23].These"adversarialexamples" ofteninvolvesubtlemodificationstothewebpage'sfeatures thatareimperceptibletohumansbutsufficienttofooltheclassifier[24].Researchintoadversarialtrainingandmore robustmodelarchitecturesiscrucialtocounterthisevolvingthreat.

• Concept Drift: The web landscape is constantly changing, with new attack vectors and obfuscation techniques emerging daily. This phenomenon, known as "concept drift," can render static ML models obsolete over time. Continuousmonitoringandperiodicretrainingarenecessarytomaintainhighdetectionaccuracy.

VI. REAL-WORLD DEPLOYMENT AND PERFORMANCE METRICS

Deploying ML models in a real-world setting introduces practical challenges related to scalability and performance. For systems that analyze webpage content, the latency of feature extraction and classification is a major concern, as it directly impacts the user's browsing experience [25]. Therefore, many practical solutions, such as browser extensions or DNS firewalls, use a tiered approach where lightweight lexical models are used for initial, real-time screening, followed by more intensivecontentanalysisforsuspiciouscases[26].

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Key performance metrics go beyond simple accuracy. The False Positive Rate (FPR) is critical, as incorrectly blocking legitimatewebsitescanbehighlydisruptive.TheFalseNegativeRate(FNR)representsmissedthreatsandis adirectmeasure of the system's security effectiveness. A balance between these metrics, often captured by the F1-Score, is essential for a practicalandreliabledetectionsystem.

VII. A PROPOSED HYBRID DETECTION FRAMEWORK

Toaddressthelimitationsofexistingsystems,weproposeaconceptualhybridframeworkdesignedforrobustness,scalability, andadaptability.

A. System Architecture

Theproposedsystemhasamulti-layeredarchitecture:

• DataCollectionLayer:ThislayeruseswebcrawlerstogatherURLsfromvarioussources,includingemailspamtraps, socialmediafeeds,andwebtrafficlogs.

• Feature Extraction Layer: This layer operates in parallel to extract lexical, content-based, and network features. A lightweight module extracts lexical features in real-time, while a more comprehensive module performs deeper contentandnetworkanalysisasynchronously.

• Hybrid Detection Layer: This is the core of the framework. It employs a two-stage detection process. Stage 1 (Rapid Filtering):AhighlyoptimizedRandomForestmodeluseslexicalfeaturestoquicklyclassifythemajorityofincoming URLs.URLsclassifiedasclearlybenignormaliciousarefinalized.Stage2(DeepAnalysis):URLsflaggedassuspicious oruncertainarepassedtoamulti-modaldeeplearningmodelthatintegratescontent,network,andvisualfeaturesfor amoredefinitiveclassification.Thisensembleapproachimprovesaccuracywhilemanagingcomputationalload[27].

• AdaptiveLearningLayer:Theframeworkincorporatesanonlinelearning module.Itperiodicallyretrainsthemodels onnewlylabeleddataandincludesadversarialtrainingtechniquestoenhancerobustnessagainstevasionattacks.

VIII. CHALLENGES AND FUTURE DIRECTIONS

Despitesignificantprogress,severalchallengesremain.Theprimaryareasforfutureresearchare:

• Adversarial Robustness: Developing intrinsically robust models against adversarial attacks is the most pressing challenge. Future work should focus on adversarial training, certified defenses, and anomaly detection to identify manipulatedinputs[28].

• Cross-Dataset Generalization: There is a critical need for techniques that improve model generalizability. Transfer learninganddomainadaptationcouldhelpmodelsadapttonewwebenvironmentswithminimalretraining[29].

• ExplainableAI(XAI):MostadvancedMLmodelsoperateas"blackboxes."IncorporatingXAIcanprovideinsightsinto whyawebpageisflaggedasmalicious,whichisvaluableforsecurityanalystsandforreducingmodelbias.

• Standardized Benchmarking: The field lacks large-scale, diverse, and publicly available datasets for standardized benchmarking [30]. The creation of such resources would significantly accelerate research and allow for more meaningfulcomparisonofdifferentapproaches.

IX. CONCLUSION

Machine learning has fundamentally transformed the approach to malicious webpage detection, offering a dynamic and proactivedefensemechanismthatfarsurpassestraditionalblacklist-basedmethods[31].High-performingsystemsbasedon deeplearningandensemblemodels,leveragingarichfusionoffeatures,canachieveoutstandingaccuracy.However,thepath

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

to deploying truly effective real-world solutions is fraught with challenges, most notably the lack of generalizability and the persistentthreatofadversarialevasion[32].

ThefutureofwebsecuritywilldependonthedevelopmentofthenextgenerationofMLsystemsthatarenotonlyaccuratebut also robust, adaptable, and scalable. By addressing the open research questions surrounding adversarial defense, generalization, and explainability, the research community can build a more resilient and secure web infrastructure for all users.

REFERENCES

[1] Aljabri, M., et al. (2022). Detecting Malicious URLs Using MachineLearning Techniques: Review and Research Directions. IEEEAccess.

[2]Sahoo,D.,Liu,C.,&Hoi,S.(2017).MaliciousURLDetectionusingMachineLearning:ASurvey.ArXiv.

[3]Reyes-Dorta,N.,etal.(2024).DetectionofmaliciousURLsusingmachinelearning.WirelessNetworks.

[4]Liaquathali,S.,&Kadirvelu,V.(2025).Integrationofnaturallanguageprocessingmethodsandmachinelearningmodelfor maliciouswebpagedetection.IAESInternationalJournalofRoboticsandAutomation.

[5]Vanhoenshoven,F.,etal.(2016).DetectingmaliciousURLsusingmachinelearningtechniques.IEEESymposiumSerieson ComputationalIntelligence(SSCI).

[6]Kazemian,H.,&Ahmed,S.(2015).Comparisonsofmachinelearningtechniquesfordetectingmaliciouswebpages.Expert SystemswithApplications.

[7] Shin, S., Ji, S., & Hong, S. (2022). A Heterogeneous Machine Learning Ensemble Framework for Malicious Webpage Detection.AppliedSciences.

[8]Hani,R.,etal.(2024).MaliciousURLDetectionUsingMachineLearning.15thInternationalConferenceonInformationand CommunicationSystems(ICICS).

[9]Hou,Y.,etal.(2010).Maliciouswebcontentdetectionbymachinelearning.ExpertSystemswithApplications.

[10]Oshingbesan,A.,etal.(2022).DetectionofMaliciousWebsitesUsingMachineLearningTechniques.ArXiv.

[11] Tabassum, T., et al. (2023). A Review on Malicious URLs Detection Using Machine Learning Methods. Journal of EngineeringResearchandReports.

[12] Nguyen, L. A. T., et al. (2020). PSI-rooted subgraph: A novel feature for IoT botnet detection using classifier algorithms. IEEEAccess.

[13]Buber,E.,Diri,B.,&Sahingoz,O.K.(2017).NLPbasedphishingattackdetectionfromURLs.InternationalConferenceon ComputerScienceandEngineering(UBMK).

[14]Breiman,L.(2001).Randomforests.MachineLearning,45(1),5–32.

[15]Lecun,Y.,Bengio,Y.,&Hinton,G.(2015).Deeplearning.Nature,521,436–444.

[16]Sagi,O.,&Rokach,L.(2018).Ensemble learning:Asurvey.WileyInterdisciplinaryReviews:DataMiningandKnowledge Discovery.

[17]Tian,K.,Jan,S.,Hu,H.,Yao,D.,&Wang,G.(2018).Needleinahaystack:Trackingdownelitephishingdomainsinthe wild. ACMInternetMeasurementConference.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

[18]Rao,R.S.,etal.(2019).Jail-Phish:Animprovedsearchenginebasedphishingdetectionsystem.Computers&Security.

[19]Marchal,S.,etal.(2016).Knowyourphish:Noveltechniquesfordetectingphishingsitesandtheirtargets.IEEEICDCS

[20]Liang,G.,etal.(2016).Crackingclassifiersforevasion:AcasestudyontheGoogle'sphishingpagesfilter.WWW2016

[21]Ma,J.,etal.(2009).IdentifyingsuspiciousURLs:anapplicationoflarge-scaleonlinelearning.ICML.

[22] Chawla, N. V., et al. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research,16,321–357.

[23]Goodfellow,I.J.,etal.(2015).Explainingandharnessingadversarialexamples.ICLR.

[24]Corona,I.,etal.(2017).DeltaPhish:Detectingphishingwebpagesincompromisedwebsites.ESORICS.

[25]Stringhini,G.,etal.(2013).Shadypaths:Leveragingsurfingcrowdstodetectmaliciouswebpages.ACMCCS.

[26] Cova, M., Kruegel, C., & Vigna, G. (2010). Detection and analysis of drive-by-download attacks and malicious JavaScript code.WWW2010.

[27]Opitz,D.,&Maclin,R.(1999).Popularensemblemethods:Anempirical study.Journal ofArtificial IntelligenceResearch, 11,169–198.

[28]Madry,A.,etal.(2018).Towardsdeeplearningmodelsresistanttoadversarialattacks.ICLR.

[29]Pan,S.J.,&Yang,Q.(2010).Asurveyontransferlearning.IEEETransactionsonKnowledgeandDataEngineering.

[30]Patil, D.R., & Patil,J. B. (2018).Maliciouswebpagesdetectionusing feature selection techniques and machinelearning. InternationalJournalofInformationTechnology.

[31]Sahingoz,O.K.,etal.(2019).MachinelearningbasedphishingdetectionfromURLs.ExpertSystemswithApplications.

[32]Montazer,G.A.,&ArabYarmohammadi,S.(2015).DetectionofphishingattacksinIraniane-bankingusingafuzzy–rough hybridsystem.AppliedSoftComputing.