International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 06 | Jun 2025
p-ISSN: 2395-0072
www.irjet.net
INTELLIGENT INFORMATION AGGREGATION AND CONDENSATION SYSTEM JVR Vinayak1, K Sai Varshith2, L Lokesh Nayak3, B Sushmitha4 123Student(B.Tech), Department of Computer Science and Engineering, AVN Institute of Engineering and
Technology, Hyderabad, India
4Assistant Professor, Department of Computer Science and Engineering, AVN Institute of Engineering and
Technology, Hyderabad, India -------------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - This paper introduces an Intelligent Information Aggregation and Condensation System developed to generate high-quality summaries from large volumes of unstructured text using a hybrid summarization strategy. The system integrates extractive and abstractive techniques, where key sentences are first identified through syntactic parsing using SpaCy, and then refined via transformer-based models such as DistilBART, BART Large, and T5 Base from Hugging Face Transformers for fluent and coherent output. The application accepts multiple document formats, including PDF, DOCX, and TXT, and employs automatic chunking for processing content that exceeds model token limits. It also features TF-IDF-based keyword extraction and provides download options in TXT, DOCX, and PDF formats. Built using Python and deployed via Streamlit, the system offers a responsive web interface with integrated CSS for enhanced usability. Designed with modularity and extensibility in mind, this solution applies to fields requiring efficient text comprehension such as healthcare, law, academia, and media, and aims to reduce cognitive load while improving access to essential information.
content summarization. Traditional summarization methods, including rule-based and statistical approaches, often fall short of capturing semantic depth and linguistic nuance. With advancements in deep learning and natural language understanding, transformer-based architectures have emerged as stateof-the-art in various natural language processing tasks. This paper presents an integrated framework designed not only to summarize lengthy and unstructured textual content but also to enhance usability through a real-time, web-accessible interface. The system is architected to support real-world use cases, enabling both technical and non-technical users to derive concise and meaningful summaries with minimal computational overhead. Notably, the framework is extensible, capable of supporting additional language models or preprocessing layers and designed for portability across operating environments. The system leverages tokenization, sentence boundary detection, semantic filtering, and modelagnostic preprocessing pipelines to ensure both precision and contextual preservation during summarization. Furthermore, the interface is designed to maintain accessibility and scalability, promoting adaptability in enterprise and research-grade workflows.
Keywords: Natural Language Processing (NLP); Text Summarization; Extractive Summarization; Abstractive Summarization; Bidirectional Encoder Representations from Transformers (BERT); TextTo-Text Transfer Transformer (T5); Generative Pretrained Transformer (GPT); BART (Bidirectional and Auto-Regressive Transformers); SpaCy NLP Library; Hugging Face Transformers; Term Frequency–Inverse Document Frequency (TF-IDF); Keyword Extraction; Document Processing; Streamlit Web Framework; Human-Centered Artificial Intelligence (AI).
2. LITERATURE REVIEW 2.1Evolution of Summarization Techniques The field of text summarization has undergone a significant transformation, beginning with early rulebased systems that used handcrafted linguistic rules to identify sentence importance. These methods, while transparent, lacked scalability and adaptability. The introduction of statistical techniques—such as term frequency-inverse document frequency (TF-IDF)— enabled content selection based on word distribution but often failed to capture contextual relationships. The advent of machine learning models improved summary relevance by learning patterns from annotated corpora, yet they were constrained by domain-specific dependencies and limited language comprehension. The progression to neural network-based approaches
1. INTRODUCTION In the digital age, the volume of textual data generated across domains such as healthcare, law, journalism, and scientific research is expanding at an unprecedented rate. The increasing demand for rapid assimilation of large-scale documents has intensified the need for intelligent systems capable of automating
© 2025, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 808