Loading...
Publication
Real-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronization
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 2.77 MB | Adobe PDF |
Advisor(s)
Abstract(s)
Gesture recognition technology has emerged as a transformative solution for
natural and intuitive human–computer interaction (HCI), offering touch-free operation
across diverse fields such as healthcare, gaming, and smart home systems. In mobile
contexts, where hygiene, convenience, and the ability to operate under resource constraints
are critical, hand gesture recognition provides a compelling alternative to traditional touch based interfaces. However, implementing effective gesture recognition in real-world mobile
settings involves challenges such as limited computational power, varying environmen tal conditions, and the requirement for robust offline–online data management. In this
study, we introduce ThumbsUp, which is a gesture-driven system, and employ a partially
systematic literature review approach (inspired by core PRISMA guidelines) to identify
the key research gaps in mobile gesture recognition. By incorporating insights from deep
learning–based methods (e.g., CNNs and Transformers) while focusing on low resource
consumption, we leverage Google’s MediaPipe in our framework for real-time detection
of 21 hand landmarks and adaptive lighting pre-processing, enabling accurate recogni tion of a “thumbs-up” gesture. The system features a secure queue-based offline–cloud
synchronization model, which ensures that the captured images and metadata (encrypted
with AES-GCM) remain consistent and accessible even with intermittent connectivity. Ex perimental results under dynamic lighting, distance variations, and partially cluttered
environments confirm the system’s superior low-light performance and decreased resource
consumption compared to baseline camera applications. Additionally, we highlight the
feasibility of extending ThumbsUp to incorporate AI-driven enhancements for abrupt
lighting changes and, in the future, electromyographic (EMG) signals for users with mo tor impairments. Our comprehensive evaluation demonstrates that ThumbsUp maintains
robust performance on typical mobile hardware, showing resilience to unstable network
conditions and minimal reliance on high-end GPUs. These findings offer new perspectives
for deploying gesture-based interfaces in the broader IoT ecosystem, thus paving the way
toward secure, efficient, and inclusive mobile HCI solutions.
Description
Keywords
gesture recognition hand tracking real-time detection MediaPipe mobile computing computer vision human–computer interaction touchless interfaces
Citation
Marques, P., Váz, P., Silva, J., Martins, P., & Abbasi, M. (2025). Real-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronization. Electronics, 14(4), 704. https://doi.org/10.3390/electronics14040704