International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 05 | May 2025
p-ISSN: 2395-0072
www.irjet.net
Graph Neural Networks Vs Transformers: A Comparative Study on Word Sense Disambiguation Ms. Margi Patel1, Dr. Ajaysinh Rathod2, Mrs. Nidhi Shah3 1 Student, Department of Computer Engineering, SVIT, Vasad, India
2 HoD CSD, Department of Computer Engineering, C.U. Shah University, Wadhwan, India 3 Assistant Professor, Department of Computer Engineering, Dharmsinh Desai University, Nadiad, India ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - This paper presents a comprehensive
The advent of deep learning marked a new era for WSD. Pretrained contextual language models like BERT (Bidirectional Encoder Representations from Transformers) revolutionized the field by generating dynamic word embedding’s that adapt to context. Yet, despite their accuracy, such models are computationally expensive and suffer from limited interpretability, especially in minimal-pair disambiguation scenarios like those posed by the WiC (Word-in-Context) dataset (Apostol, Truică, & Paschke, 2024). In parallel, graph-based learning approaches have gained traction. Graph Neural Networks (GNNs)—and specifically Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs)—introduce a relational reasoning paradigm by modelling text as graphs. Nodes in such graphs represent words, and edges reflect syntactic or semantic relations. These models aggregate information from a word’s neighbourhood, offering an alternative lens to interpret language structure (Apostol, Coban, & Truică, 2024).This paper explores the comparative strengths of GATConv-based GNNs and Transformer-based architectures like BERT in solving WSD. It aims to assess their performance across several dimensions: predictive accuracy, computational efficiency, and interpretability, using the WiC dataset as the experimental benchmark.
comparative analysis of Graph Neural Networks (GNNs), with a specific focus on Graph Attention Networks (GATConv), and Transformer-based models, particularly BERT, for the task of Word Sense Disambiguation (WSD). Utilizing the Word-inContext (WiC) dataset as a benchmark, we evaluate both paradigms on three key metrics: predictive accuracy, computational efficiency, and interpretability. Our results reveal that graph-based models, particularly GATConv, outperform BERT in minimal-context WSD tasks. In addition to superior accuracy, GNNs offer significant advantages in terms of runtime and memory consumption, demonstrating their potential for practical deployment in resourceconstrained environments. This study highlights the growing utility of graph-based reasoning in advancing semantic understanding in Natural Language Processing (NLP), paving the way for more efficient and interpretable solutions to complex linguistic tasks. Key Words: Graph Neural Networks, Word Sense Disambiguation, Graph Attention Networks, Transformer models, Computational efficiency.
1. INTRODUCTION
2. LITERATURE REVIEW
Natural language is inherently ambiguous. A single word may carry multiple meanings, and the correct interpretation often depends on subtle contextual clues. This phenomenon, known as polysemy, presents a critical challenge for Natural Language Processing (NLP). The process of resolving such ambiguity is known as Word Sense Disambiguation (WSD). WSD is central to a wide range of NLP applications, including machine translation, information retrieval, semantic search, and question answering. An incorrect sense interpretation can compromise the integrity of downstream tasks, making robust WSD essential. Historically, WSD approaches have evolved from simple rule-based models to complex deep learning architectures. Early methods such as the Lesk algorithm relied on dictionary definitions and gloss overlaps, while later supervised models introduced statistical learning techniques, including Naïve Bayes, SVMs, and Decision Trees, often combined with hand-crafted features. However, these models struggled with scalability and generalization, especially when applied to unseen domains or languages (Buturoiu, Corbu, & Boțan, 2023).
© 2025, IRJET
|
Impact Factor value: 8.315
2.1 Evolution of Word Sense Disambiguation Techniques The development of Word Sense Disambiguation (WSD) has undergone several distinct stages, each characterized by new paradigms in linguistic representation and computational modeling. At the foundation lie symbolic and dictionarybased methods, followed by supervised statistical learning, neural embedding techniques, and most recently, deep contextual models and graph neural architectures. These evolving techniques reflect broader trends in natural language understanding, as researchers continually refine models to cope with the increasing complexity of language phenomena and online information environments (Giachanou, Zhang, Barrón-Cedeño, Koltsova, & Rosso, 2022).
|
ISO 9001:2008 Certified Journal
|
Page 24