International Research Journal of Engineering and Technology (IRJET) Volume: 11 Issue: 12 | Dec 2024
www.irjet.net
e-ISSN: 2395-0056
p-ISSN: 2395-0072
CONTROLLING SOCIAL MEDIA APPLICATION USING COMPUTER VISION Mohan H G1, Abhishek S2, Aishwarya S G3, Arpitha V4, Bharath C5 1 Asst professor, Department of Computer Science and Engineering, Jawaharlal Nehru New College of Engineering,
Shimoga 2, 3, 4, 5 BE students, Department of Computer Science and Engineering, Jawaharlal Nehru New College of
Engineering, Shimoga ---------------------------------------------------------------------***---------------------------------------------------------------------I. Abstract With advancements in computer vision, the potential for accessibility-focused human-computer interaction (HCI) has expanded. This paper presents an innovative system that uses hand gestures and facial movements for touch-free digital interaction, targeting users with physical limitations and specialized environments requiring hands-free interfaces. By combining OpenCV for image processing, MediaPipe for real-time landmark detection, and PyAutoGUI for action execution, this system offers an accessible, adaptive, and intuitive alternative to traditional input devices. The system achieves realtime performance and demonstrates robust functionality under diverse conditions, making it an ideal solution for accessibility enhancement, professional multitasking, and interaction innovation. Future expansions include adaptive learning algorithms, gesture library enhancement, and privacy safeguards to further enhance usability and security. Keywords: Gesture-based control system, Hands-free interaction, MediaPipe, OpenCV, PyAutoGUI, Real-time processing, Human-computer interaction (HCI).
II.
Introduction
The evolution of HCI technologies has been pivotal in improving accessibility and efficiency across various domains. Traditional input devices such as keyboards and mice, though widely used, pose limitations for individuals with motor disabilities and in environments where direct touch is impractical. The demand for intuitive, hands-free systems has consequently surged. This paper explores a gesture and facial movement-based control system designed to enable seamless interaction with digital devices. The system uses OpenCV to process video frames, MediaPipe to detect hand and facial landmarks, and PyAutoGUI to translate gestures into actionable computer commands. This modular design supports real-time interaction while maintaining adaptability across different environments and user needs. Applications of the system range from accessibility tools for individuals with disabilities to enhanced productivity in sterile or multitasking environments such as healthcare or manufacturing. By addressing existing limitations in gesturerecognition systems and emphasizing usability and robustness, this project contributes significantly to the field of HCI.
III.
Related work
The evolution of gesture-based control systems has seen significant advancements in recent years, with researchers exploring various methods to achieve touch-free human-computer interaction (HCI). Early systems, such as the "Virtual Mouse Controlled by Tracking Eye Movement," used webcams to detect and track pupil movements, allowing users with motor disabilities to control computer cursors. [1] have proposed the approach provided a low-cost solution for accessibility but faced challenges such as sensitivity to lighting and the need for precise calibration. [2] Hand gesture recognition systems, on the other hand, have leveraged tools like OpenCV and Python to enable real-time tracking of hand movements. For example, a "Hand Gesture Controlled System Using OpenCV and Python" mapped specific gestures to predefined actions, showcasing the potential for gesture-based navigation. [3] have proposed promise, such systems often struggled with environmental variability and were limited to simple gestures due to hardware constraints. [4] MediaPipe's on-device real-time hand-tracking framework introduced a significant leap in precision and adaptability. [5] have proposed the studies utilizing MediaPipe for gesture recognition have demonstrated high-quality tracking of hand landmarks, supporting applications in accessibility and AR/VR environments. [6] have proposed challenges such as performance under poor lighting and limitations in detecting complex gestures persisted.
© 2024, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 317