Mid-air gestures

💡 We examine radar-based gesture input for interactive computer systems, a technology that has recently grown in terms of commercial availability, affordability, and popularity among researchers and practitioners, where radar sensors are leveraged to detect user input performed in mid-air, on the body, and around physical objects and digital devices. We analyze forty-five academic papers published on this topic between 2010 and 2021, and report results regarding gesture recognition techniques, application types, and evaluation approaches for radar-based gesture input. Our findings reveal that (1) deep learning techniques, such as Convolutional Neural Networks, have been the most popular approach for radar-based gesture recognition, (2) application opportunities for implementing radar gestures have been diverse, but without any clear contender for a game changer in this area, and (3) the gesture sets employed in prior work have been small with a median of just six gesture types. Based on these findings, we draw ten implications for integrating radar-based gesture sensing in ambient intelligence environments.

💡 We address gesture input for TV control, for which we examine mid-air free-hand interactions that can be detected via radar sensing. We adopt a scenario-based design approach to explore possible locations from the living room where to integrate radar sensors, e.g., in the TV set, the couch armrest, or the user’s smartphone, and we contribute a four-level taxonomy of locations relative to the TV set, the user, personal robot assistants, and the living room environment, respectively. We also present preliminary results about an interactive system using a 15-antenna ultra-wideband 3D radar, for which we implemented a dictionary of six directional swipe gestures for the control of dichotomous TV system functions.