Repository logo
 
Publication

Real-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronization

2025-02-12Text Open access
Simple item page
datacite.subject.fosEngenharia e Tecnologia
dc.contributor.authorMarques, Pedro
dc.contributor.authorANTUNES VAZ, PAULO JOAQUIM
dc.contributor.authorSilva, José
dc.contributor.authorMartins, Pedro
dc.contributor.authorAbbasi, Maryam
dc.date.accessioned2025-03-24T13:01:15Z
dc.date.available2025-03-24T13:01:15Z
dc.date.issued2025-02-12
dc.description.abstractGesture recognition technology has emerged as a transformative solution for natural and intuitive human–computer interaction (HCI), offering touch-free operation across diverse fields such as healthcare, gaming, and smart home systems. In mobile contexts, where hygiene, convenience, and the ability to operate under resource constraints are critical, hand gesture recognition provides a compelling alternative to traditional touch based interfaces. However, implementing effective gesture recognition in real-world mobile settings involves challenges such as limited computational power, varying environmen tal conditions, and the requirement for robust offline–online data management. In this study, we introduce ThumbsUp, which is a gesture-driven system, and employ a partially systematic literature review approach (inspired by core PRISMA guidelines) to identify the key research gaps in mobile gesture recognition. By incorporating insights from deep learning–based methods (e.g., CNNs and Transformers) while focusing on low resource consumption, we leverage Google’s MediaPipe in our framework for real-time detection of 21 hand landmarks and adaptive lighting pre-processing, enabling accurate recogni tion of a “thumbs-up” gesture. The system features a secure queue-based offline–cloud synchronization model, which ensures that the captured images and metadata (encrypted with AES-GCM) remain consistent and accessible even with intermittent connectivity. Ex perimental results under dynamic lighting, distance variations, and partially cluttered environments confirm the system’s superior low-light performance and decreased resource consumption compared to baseline camera applications. Additionally, we highlight the feasibility of extending ThumbsUp to incorporate AI-driven enhancements for abrupt lighting changes and, in the future, electromyographic (EMG) signals for users with mo tor impairments. Our comprehensive evaluation demonstrates that ThumbsUp maintains robust performance on typical mobile hardware, showing resilience to unstable network conditions and minimal reliance on high-end GPUs. These findings offer new perspectives for deploying gesture-based interfaces in the broader IoT ecosystem, thus paving the way toward secure, efficient, and inclusive mobile HCI solutions.eng
dc.description.sponsorshipFurthermore, we thank the Research Center in Digital Services (CISeD) and the Instituto Politécnico de Viseu for their support. Maryam Abbasi thanks the national funding by FCT—Foundation for Science and Technology, PI, through the institutional scientific employment program contract (CEECINST/00077/2021).
dc.identifier.citationMarques, P., Váz, P., Silva, J., Martins, P., & Abbasi, M. (2025). Real-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronization. Electronics, 14(4), 704. https://doi.org/10.3390/electronics14040704
dc.identifier.doihttps://doi.org/10.3390/electronics14040704
dc.identifier.eissn2079-9292
dc.identifier.urihttp://hdl.handle.net/10400.19/9295
dc.language.isoeng
dc.peerreviewedyes
dc.publisherMDPI
dc.relationCentre for Research in Digital Services
dc.relation.hasversionhttps://www.mdpi.com/2079-9292/14/4/704
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.subjectgesture recognition
dc.subjecthand tracking
dc.subjectreal-time detection
dc.subjectMediaPipe
dc.subjectmobile computing
dc.subjectcomputer vision
dc.subjecthuman–computer interaction
dc.subjecttouchless interfaces
dc.titleReal-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronizationpor
dc.typetext
dspace.entity.typePublication
oaire.awardTitleCentre for Research in Digital Services
oaire.awardURIinfo:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F05583%2F2020/PT
oaire.citation.issue4
oaire.citation.startPage704
oaire.citation.titleElectronics
oaire.citation.volume14
oaire.fundingStream6817 - DCRRNI ID
oaire.versionhttp://purl.org/coar/version/c_970fb48d4fbd8a85
person.familyNameANTUNES VAZ
person.familyNameSilva
person.givenNamePAULO JOAQUIM
person.givenNameJosé
person.identifier.ciencia-id351C-9899-0EE7
person.identifier.ciencia-id4A14-D3E7-5B32
person.identifier.orcid0000-0002-1745-8937
person.identifier.orcid0000-0001-7285-8282
person.identifier.scopus-author-id55447844100
project.funder.identifierhttp://doi.org/10.13039/501100001871
project.funder.nameFundação para a Ciência e a Tecnologia
relation.isAuthorOfPublication702e79ee-5b0b-47ff-989d-12e6d8ea1e89
relation.isAuthorOfPublicatione9d8719e-af47-4008-b854-817801bb3964
relation.isAuthorOfPublication.latestForDiscovery702e79ee-5b0b-47ff-989d-12e6d8ea1e89
relation.isProjectOfPublicationa2335235-05b4-404c-a71e-cc37ae7fbf2c
relation.isProjectOfPublication.latestForDiscoverya2335235-05b4-404c-a71e-cc37ae7fbf2c

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Real-Time Gesture-Based Hand Landmark Detection for Optimized Mobile Photo Capture and Synchronization.pdf
Size:
2.77 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.79 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

ESTGV - DEMGI - Artigo em revista científica, indexada ao WoS/Scopus
CISeD - Artigo em revista científica, indexada ao WoS/Scopus
ESTGV - DI - Artigo em revista científica, indexada ao WoS/Scopus