International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 12 | Dec 2025
p-ISSN: 2395-0072
www.irjet.net
GESTURE CONTROL VIRTUAL MOUSE AND VOICE ASSISTANT Prof. Arpita Patil1, Srusti Laxani2, Sejal Patil3, Shifa Mulla4, Vaishnavi Lokapur5 1Assistant Professor, SG Balekundri Institute of Technology, Belagavi, Karnataka, India 2,3,4,5, Student, SG Balekundri Institute of Technology, Belagavi, Karnataka, India
---------------------------------------------------------------------------***----------------------------------------------------------------------------------
Abstract In this project, we combined hand-gesture control with voice commands to create a completely touch-free way of
using a computer. Instead of relying on a mouse or keyboard, the system uses a camera to follow the movement of your hand. Simple actions—like moving the cursor, clicking, or scrolling—happen automatically based on how you position or move your hand in front of the camera. This makes it possible to control the computer without ever touching any device. Along with gestures, a built-in voice assistant adds another layer of convenience. By speaking naturally, users can open apps, perform searches, adjust settings, or carry out small tasks without lifting a finger. The two features complement each other well: when gestures feel easier, you can rely on your hands, and when speaking is faster, you can use your voice. During our testing, both systems performed consistently and didn’t require much adjustment. The gesture tracking responded accurately, and the voice commands were understood most of the time without needing to repeat them. Because of this, the overall setup feels smooth and practical for everyday use. What makes this system especially useful is how adaptable it is. It can help people who need simpler or more accessible ways to operate a computer, such as users with limited mobility. It also works well in environments where touch-free interaction is preferred—like laboratories, healthcare settings, or situations where keeping devices clean matters. Overall, the combination of gestures and voice control creates a comfortable, flexible, and modern way to interact with technology.
Key Words: Blaze Pose, Media Pipe Framework, XG Boost, Open CV, Python Programming, Real-time Video Processing; Gesture-Controlled Virtual Mouse; Voice Assistant Integration, Touch-Free Human-Computer Interaction, Hand-Gesture Recognition.
1. INTRODUCTION Computers play a major role in our everyday activities, but using a traditional mouse and keyboard isn’t always practical— especially for people who can’t use their hands comfortably or need a touch-free way to interact with a device. To solve this, our system brings together two natural forms of interaction: hand gestures and voice control. With the gesture feature, the camera follows the movement of your hands, allowing you to move the pointer, click, or select items just by shifting your fingers or palm. At the same time, the voice assistant listens for spoken commands, making it possible to open applications, search the internet, adjust settings, or perform routine tasks simply by talking. By combining both features, the system offers a smooth, intuitive, and completely hands-free way of using a computer. This makes everyday interactions easier, reduces physical effort, and supports users who need simpler or more accessible alternatives to traditional input devices. It also works well in environments where touching equipment isn’t ideal, such as laboratories, hospitals, or public spaces. Overall, the blended use of gestures and voice commands creates a more natural and flexible computing experience.
2. LITERATURE REVIEW People who know a lot about computers have found new ways to use them without a mouse or keyboard. By using hand movements, users can control the pointer and select items just by moving their hands. In addition, speaking to the computer allows users to give commands using their voice. By combining these two methods makes computer interaction easier and hands-free. This interaction these techniques are highly effective for improving usability, enabling multitasking, and working in environments where physical contact is not possible. In the paper “Hand Gesture Recognition-Based Virtual Mouse Using Media Pipe” by Pavithra et al., the authors introduced un touch-free way to control computer by tracking hand movements through a webcam. Using media pipe, the system identifies a finger positions and converts them into action like moving cursor or clicking. While the approach is simple and effective, especially for basic computer interactions it is performance drops in poor lighting, and offers only limited gesture options. The system also does not include other helpful input modes—such as voice control—which would make it more accessible in different environments.
1. In this paper “Virtual Mouse is using Hand Gestures” by Matlani et al., developed a system that allows the user to operate the mouse through basic hand gestures movements captured through webcam. Using OpenCV and Media Pipe, the system detects fingertips to perform tasks such as controlling the cursor, clicking, dragging. It offers a touch free way to interact with a computer, but its performance drops in poor lighting or busy backgrounds, and it relies only a gesture without additional input options.
© 2025, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 513