The development in the field of human-computer interface has put higher demands on humanized and immersive human-machine interfaces (HMIs). New HMIs require not only simple and efficient design, but also intuitive operation. In smart homes, assistive technologies, and smart security, intuitive and multifunctional HMIs are of great importance. As body language that can convey rich intentions, gestures are favored in HMI. Glove-based human-computer interaction interfaces have been reported frequently. On the one hand, previous work in this field only extracted one-dimensional information from gestures, which is not only detrimental to the development of multidimensional HMI, but also increases the complexity of the interactive system, causes higher energy consumption and computational stress of the system, and negatively affects the user experience. Therefore, the multidimensional features contained in gestures need to be extracted and utilized to develop human-computer interaction interfaces with simple structures and various functions.
Recently, the teams of Associate Professor Hongyan Fu and Assistant Professor Wenbo Ding, from Tsinghua-Berkeley Shenzhen Institute (TBSI) at Tsinghua University Shenzhen International Graduate School (Tsinghua SIGS), have made important achievements in the research of novel HMI. They proposed a simple structured sensitive bending angle triboelectric nanogenerator (BA-TENG) and designed an intelligent HMI system from flexible devices to real-time wireless communication modules and graphical interfaces. The bending angle sensor is capable of detecting multidimensional information in hand gestures, including finger bending angles, bending speed, and bending time. In this paper, the multi-dimensional signal extraction of hand gestures is realized by the flexible sensor and the signal processing system.
By extracting and analyzing the aforementioned multidimensional signal features, different dimensions of HMI are realized, including smart home (lighting control, for example), robotic control, and a virtual keyboard that enables user recognition (its recognition accuracy rate can reach up to 93.1%). The unique transimpedance amplifier-based signal conditioning circuit eliminates most of the crosstalk and noise in the real-time transmission of multi-channel signals, enhancing the robustness of the system. This research work provides new ideas for developing multidimensional HMI, which has unlimited potential for future HMI applications.
The teams’ results were recently published as Triboelectric Bending Sensor based Smart Glove towards Intuitive Multi-Dimensional Human-Machine Interfaces in the leading international journal Nano Energy. Their paper was featured on the front cover of the journal, selected among the 160 articles received for the issue. Associate Professor Hongyan Fu and Assistant Professor Wenbo Ding are the co-corresponding authors of this paper; Yang Luo and Zihan Wang are the co-first authors. The paper’s other authors are Jiyu Wang, a postdoctoral fellow at TBSI; Xiao Xiao, a class of 2020 biomedical engineering Ph.D. candidate at UCLA; and Associate Professor Li Qian from Peking University’s School of Electronic and Computer Engineering. Their research was supported by the National Natural Science Foundation of China, the Basic and Applied Basic Research Fund of Guangdong Province, the Key Technologies R & D project of the Science and Technology Innovation Commission of Shenzhen Municipality, the Foshan-Tsinghua Collaborative Innovation Project for Industry-University-Research Cooperation, and the Research Fund of the Tsinghua University Guoqiang Institute.
Link to full article:
https://doi.org/10.1016/j.nanoen.2021.106330
Text/Images: Yang Luo, Zihan Wang
Editing: A.S., Yuan Yang
