International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 05 | May 2025
p-ISSN: 2395-0072
www.irjet.net
Vision AI: An Interactive Perception System Bhaumik Shyam Birje1, Swaraj Prakash Patil2, Hardik Ganesh Patil3, Atharva Bhimrao Sawant4,Anil. S. Londhe5, Sanjay M. Patil6 1 2 3 4 5 6
Artificial Intelligence S Data Science, Datta Meghe College of Engineering, Mumbai, India bhaumikbirje@gmail.com, swarajpatil1913@gmail.com, hardikpatil091@gmail.com, sawantatharva63@gmail.com, anil.londhe@dmce.ac.in, sanjay.patil@dmce.ac.in ---------------------------------------------------------------------***--------------------------------------------------------------------This crucial design choice bypasses the need for constant cloud connectivity, drastically cutting down latency and keeping sensitive data local, which significantly enhances privacy. We’ve built in several clever optimizations, like making the AI models smaller and using multiple parts of the computer’s brain at once (multi-threading), to ensure Vision AI performs well even on this limited hardware. By weaving together real-time video analysis with voicedriven queries, Vision AI offers a remarkably intuitive way for users to interact with technology. You simply speak to the system, and it provides detailed, contextaware responses about your environment. This opens up exciting possibilities across many fields. Think of it empowering people with visual impairments by describing their surroundings, making factories smarter and safer through automated monitoring, creating more responsive and helpful smart homes, or even serving as an engaging, interactive learning tool in education. Our core focus is on creating a perception system that is not just intelligent, but also more accessible and user-friendly for a wider audience by bringing sophisticated AI capabilities to edge devices. This paper will take you through the journey of building Vision AI. We’ll start in Section II by looking at the current state of similar technologies and identifying where Vision AI fits in. Section III will unveil the system’s architecture, showing how its different parts work together. Section IV will dive into the technical details of how we built it, including the software tools and techniques we used. Section V will present the results of our experiments, showing how well Vision AI performed under various tests. Section VI will share the advanced implementation details and the significant challenges we faced and overcame. Section VII will explore real-world examples and where Vision AI could shine. Section VIII will openly discuss the important ethical questions our work raises. Finally, Section IX will wrap everything up and look ahead to where we see Vision AI going next.
Abstract—Vision AI introduces a novel approach to interactive perception by seamlessly integrating realtime object recognition with natural language processing (NLP) on compact, low-resource platforms like the Raspberry Pi. Our system creates a bridge between what a machine sees and how it can converse about it, allowing users to simply speak commands and receive intelligent, context-aware responses about their physical environment. This paper provides a comprehensive look at Vision AI, detailing its complete system architecture, specific implementation choices, extensive experimental evaluations, the challenges we navigated, and the ethical considerations inherent in such technology. The results showcase solid object detection accuracy, reliable speech recognition, and practical processing latency, confirming Vision AI’s potential for a wide array of real-world applications. Built with modularity and scalability in mind, the system can be easily adapted to different hardware setups and diverse application needs. We also discuss plans for future improvements, including incorporating more advanced NLP for deeper understanding and building in feedback mechanisms to continuously enhance performance. Vision AI stands as a step towards more accessible, privacy-conscious intelligent systems operating right where the action is. Index Terms—Key Words: Vision AI, Interactive Perception, Object Detection, NLP, Voice Interaction, Raspberry Pi, Real-Time Systems, LLM Integration, HumanComputer Interaction. Edge Computing, Embedded Systems.
I.
INTRODUCTION
Imagine a world where you can simply ask your surroundings questions, and they intelligently respond based on what they see and understand. While machines have gained incredible abilities to perceive the visual world and process human language separately, truly natural interaction requires blending these capabilities. Current virtual assistants, like those found in smart speakers, are great listeners but remain largely blind to their immediate physical environment. They struggle to answer questions tied directly to what’s in front of them in real-time. Furthermore, their reliance on sending all requests to distant cloud servers introduces frustrating delays and raises serious privacy concerns, making them unsuitable for many sensitive or time-critical applications. Vision AI steps in to address these very challenges. It cleverly combines efficient object detection – the ability to ”see” and identify things – with natural language processing into a single, cohesive system designed to run entirely on a small, affordable computer like the Raspberry Pi.
© 2025, IRJET
|
Impact Factor value: 8.315
II.
RELATED WORK
The world of AI has seen incredible progress in letting computers understand speech and recognize objects, but bringing these two senses together seamlessly remains a significant area of research. Today’s popular virtual assistants, like Amazon Alexa [9] and Google Assistant
|
ISO 9001:2008 Certified Journal
|
Page 561