Gesture input

💡 We introduce ChairMX, on-chair input technology for novel interactive media consumption experiences through the chair. ChairMX is the end result of an elaborate four-stage research and development process involving a systematic literature review of the extent in which chairs have been used in interactive systems, involvement of potential end users to elicit their preferences for on-chair input, engineering on-chair gesture input recognition technology with wearables, and exploration of application opportunities. By leveraging a ring-like form factor wearable, ChairMX is a readily deployable technology, working for uninstrumented chairs everywhere and enabling a variety of use case scenarios involving interactive media consumption, from the living room to public environments.

💡 We examine radar-based gesture input for interactive computer systems, a technology that has recently grown in terms of commercial availability, affordability, and popularity among researchers and practitioners, where radar sensors are leveraged to detect user input performed in mid-air, on the body, and around physical objects and digital devices. We analyze forty-five academic papers published on this topic between 2010 and 2021, and report results regarding gesture recognition techniques, application types, and evaluation approaches for radar-based gesture input. Our findings reveal that (1) deep learning techniques, such as Convolutional Neural Networks, have been the most popular approach for radar-based gesture recognition, (2) application opportunities for implementing radar gestures have been diverse, but without any clear contender for a game changer in this area, and (3) the gesture sets employed in prior work have been small with a median of just six gesture types. Based on these findings, we draw ten implications for integrating radar-based gesture sensing in ambient intelligence environments.

💡 We address gesture input for TV control, for which we examine mid-air free-hand interactions that can be detected via radar sensing. We adopt a scenario-based design approach to explore possible locations from the living room where to integrate radar sensors, e.g., in the TV set, the couch armrest, or the user’s smartphone, and we contribute a four-level taxonomy of locations relative to the TV set, the user, personal robot assistants, and the living room environment, respectively. We also present preliminary results about an interactive system using a 15-antenna ultra-wideband 3D radar, for which we implemented a dictionary of six directional swipe gestures for the control of dichotomous TV system functions.

💡 Personal drones are more and more present in our lives and acting as “flying cameras” is one of their most prominent applications. In this work, we conduct a synopsis of the scientific literature on human-drone interaction to identify system functions and corresponding commands for controlling drone-based aerial photography and video, from which we compile a dictionary of interactions. We also discuss opportunities for more research at the intersection of drone computing, augmented vision, and personal photography.

💡 We introduce Tap4Light, an interaction technique for controlling smart lighting systems with finger tapping gestures captured with a five-finger augmentation device. Tap4Light employs simple tap input that, when combined across all of the five fingers of the hand, enables a rich gesture vocabulary for controlling applications, systems, and services in the context of Ambient Intelligence environments. We demonstrate Tap4Light with a set of eleven tap gestures, the Tap Strap 2 wearable, and Philips Hue smart lamps.