A REVIEW OF SELF-SUPERVISED DEEP ENCODER–DECODER ARCHITECTURE WITH DYNAMIC MASKED RECONSTRUCTION FOR

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 02 | Feb 2026

p-ISSN: 2395-0072

www.irjet.net

A REVIEW OF SELF-SUPERVISED DEEP ENCODER–DECODER ARCHITECTURE WITH DYNAMIC MASKED RECONSTRUCTION FOR STRUCTURED DATA REPRESENTATION LEARNING Amit Gupta1, Mrs. Arifa Khan2 1Master of Technology, Computer Science and Engineering, Lucknow Institute of Technology, Lucknow, India 2Assistant Professor, Department of Computer Science and Engineering, Lucknow Institute of Technology,

Lucknow, India ---------------------------------------------------------------------***--------------------------------------------------------------------1. INTRODUCTION Abstract - Self-supervised learning (SSL) has emerged as a transformative approach in deep learning, enabling models to learn meaningful representations from unlabeled data. Among various SSL paradigms, encoder–decoder architectures with masked reconstruction have demonstrated significant potential in structured data representation learning, such as tabular, time-series, and graph-based datasets. Dynamic masked reconstruction, a recent advancement over fixed masking strategies, adaptively selects portions of input data to mask during training, improving the model’s ability to generalize and capture underlying data structures. This review paper systematically examines the development, methodologies, and applications of self-supervised encoder– decoder architectures with dynamic masked reconstruction, emphasizing their effectiveness in structured data representation. The literature survey covers traditional autoencoders, transformer-based models, masked autoencoders, and hybrid approaches, highlighting performance metrics, strengths, and limitations reported in recent studies. Additionally, the review evaluates Python-based implementations, frameworks, and tools commonly used to build and experiment with these models, providing practical insights for researchers and practitioners. Challenges, such as dataset complexity, computational costs, and reproducibility issues, are discussed, alongside emerging trends and potential future directions, including hybrid SSL models, adaptive masking strategies, and standardized benchmarking for structured datasets. By synthesizing existing research, this paper aims to offer a comprehensive perspective on current methodologies and guide future work in efficient and scalable representation learning for structured data in Python. The review demonstrates that dynamic masked reconstruction, combined with encoder–decoder architectures, represents a promising avenue for advancing self-supervised learning in structured domains.

1.1 Background 1.1.1 Overview for Representation Learning Representation learning is a critical aspect of modern machine learning that focuses on automatically extracting meaningful and compact features from raw data. Traditional machine learning methods often rely on handcrafted features, which are time-consuming and domain-specific, limiting their generalizability (Bengio, Courville & Vincent, 2013). Deep learning approaches, particularly those leveraging neural networks, have revolutionized this domain by learning hierarchical representations that capture complex patterns and latent structures inherent in the data (LeCun, Bengio & Hinton, 2015). Efficient representation learning is essential for improving the performance of downstream tasks such as classification, regression, clustering, and anomaly detection. 1.1.2 Importance of Structured Data Structured data, including tabular datasets, time-series records, and graphs, constitute a significant portion of realworld information across various domains, such as finance, healthcare, and network analytics (Guo et al., 2021). Unlike unstructured data (e.g., images or text), structured data has clearly defined attributes, often with semantic meaning, making its effective representation critical for decisionmaking. Learning robust representations from structured data helps in uncovering relationships between features, reducing dimensionality, and enhancing predictive accuracy in downstream applications. 1.1.3 Challenges in Labeled Data Scarcity

Key Words: Self-Supervised Learning, Encoder–Decoder Architecture, Dynamic Masked Reconstruction, Structured Data Representation, Deep Learning, Python Implementation

© 2026, IRJET

|

Impact Factor value: 8.315

A persistent challenge in supervised learning is the dependency on large amounts of labeled data, which is often expensive, time-consuming, or even infeasible to obtain (Goodfellow, Bengio & Courville, 2016). Structured datasets frequently suffer from label sparsity, missing values, or noisy annotations, which can severely degrade model performance. These limitations have motivated research into self-supervised learning, where models leverage intrinsic

|

ISO 9001:2008 Certified Journal

|

Page 757