Alexandru-Ionuț ȘIEAN

💻 Streamlining interactions with smart devices for accessibility.

📡 Developing radar-based gesture sensing for novel interactions.

💡 Personal drones are more and more present in our lives and acting as “flying cameras” is one of their most prominent applications. In this work, we conduct a synopsis of the scientific literature on human-drone interaction to identify system functions and corresponding commands for controlling drone-based aerial photography and video, from which we compile a dictionary of interactions. We also discuss opportunities for more research at the intersection of drone computing, augmented vision, and personal photography.

💡 WearSkill is our web-based application for personalized input with wearables that acts as a middleware to control connected devices that support Wi-Fi and WebSocket communications, e.g., control the smart lighting system with the smartwatch. WearSkill primarily addresses users with upper-body motor impairments, for whom it enables personalized recommendations for input modalities (touch, motion, and voice) and three categories of wearables (smartwatches, rings, and glasses) based on users' self-reported motor impairments.

💡 We address gesture input for TV control, for which we examine mid-air free-hand interactions that can be detected via radar sensing. We adopt a scenario-based design approach to explore possible locations from the living room where to integrate radar sensors, e.g., in the TV set, the couch armrest, or the user’s smartphone, and we contribute a four-level taxonomy of locations relative to the TV set, the user, personal robot assistants, and the living room environment, respectively. We also present preliminary results about an interactive system using a 15-antenna ultra-wideband 3D radar, for which we implemented a dictionary of six directional swipe gestures for the control of dichotomous TV system functions.

💡 We introduce ChairMX, on-chair input technology for novel interactive media consumption experiences through the chair. ChairMX is the end result of an elaborate four-stage research and development process involving a systematic literature review of the extent in which chairs have been used in interactive systems, involvement of potential end users to elicit their preferences for on-chair input, engineering on-chair gesture input recognition technology with wearables, and exploration of application opportunities. By leveraging a ring-like form factor wearable, ChairMX is a readily deployable technology, working for uninstrumented chairs everywhere and enabling a variety of use case scenarios involving interactive media consumption, from the living room to public environments.

💡 We conduct a systematic literature review on wearable interactions for users with motor impairments and report results from a meta-analysis of 57 scientific articles identified in the ACM DL and IEEE Xplore databases. Our findings show limited research conducted on accessible wearable interactions (e.g., just four papers addressing smartwatch input), a disproportionate interest for hand gestures compared to other input modalities for wearable devices, and low numbers of participants with motor impairments involved in user studies about wearable interactions (a median of 6.0 and average of 8.2 participants per study). We compile an inventory of 92 finger, hand, head, shoulder, eye gaze, and foot gesture commands for smartwatches, smartglasses, headsets, earsets, fitness trackers, data gloves, and armband wearable devices extracted from the scientific literature that we surveyed. Based on our findings, we propose four directions for future research on accessible wearable interactions for users with motor impairments.

💡 We introduce Tap4Light, an interaction technique for controlling smart lighting systems with finger tapping gestures captured with a five-finger augmentation device. Tap4Light employs simple tap input that, when combined across all of the five fingers of the hand, enables a rich gesture vocabulary for controlling applications, systems, and services in the context of Ambient Intelligence environments. We demonstrate Tap4Light with a set of eleven tap gestures, the Tap Strap 2 wearable, and Philips Hue smart lamps.

💡 We show how two recently introduced visual tools, RepliGES and GEStory, can be used conjointly to inform possible replications of Gesture Elicitation Studies (GES) with a case study centered on gestures that can be sensed with radars. Starting from a GES identified in GEStory, we employ the dimensions of the RepliGES space to enumerate eight possible ways to replicate that study towards gaining new insights into end user’s preferences for gesture-based interaction for applications that use radar sensors.

💡 We examine radar-based gesture input for interactive computer systems, a technology that has recently grown in terms of commercial availability, affordability, and popularity among researchers and practitioners, where radar sensors are leveraged to detect user input performed in mid-air, on the body, and around physical objects and digital devices. We analyze forty-five academic papers published on this topic between 2010 and 2021, and report results regarding gesture recognition techniques, application types, and evaluation approaches for radar-based gesture input. Our findings reveal that (1) deep learning techniques, such as Convolutional Neural Networks, have been the most popular approach for radar-based gesture recognition, (2) application opportunities for implementing radar gestures have been diverse, but without any clear contender for a game changer in this area, and (3) the gesture sets employed in prior work have been small with a median of just six gesture types. Based on these findings, we draw ten implications for integrating radar-based gesture sensing in ambient intelligence environments.

💡 We conduct a meta-analysis of scientific contributions in assistive technology, systems, and applications designed for people with motor disabilities and/or limited mobility at the intersection of Ambient Intelligence and Mixed Reality environments. Our findings show that most of the scientific contributions have focused on navigation, rehabilitation, and video games; systems outnumber user studies; and the most frequently implemented attributes of Ambient Intelligence environments have been sensitivity, responsiveness, and adaptation. Our survey also shows that work at the intersection of Ambient Intelligence and Mixed Reality for people with motor disabilities has been scarce despite the opportunities enabled by these two research areas in conjunction to increase access to digital information and services.