Learning the signatures of the human grasp
using a scalable tactile glove

Abstract

Humans can feel, weigh, and grasp diverse objects, and simultaneously infer their material properties while applying the right amount of force – a challenging set of tasks for a modern robot. Mechanoreceptor networks that provide sensory feedback and enable the dexterity seen in the human grasp remain challenging to imitate in manmade robots. While computer vision-based robot grasping strategies have progressed significantly with the abundance of visual data and emerging machine learning tools, there are yet no equivalent sensing platforms and large-scale datasets to utilize tactile information that humans rely on for grasping objects. Importantly, the inability to record and analyse tactile signals currently limits our understanding of the role of tactile information in the human grasp itself – e.g., how tactile maps are used to identify objects and infer their properties is unknown. Here we demonstrate – using a scalable tactile glove (STAG) and deep convolutional neural networks (CNNs) – that sensors uniformly distributed over the hand can be used to identify individual objects, estimate weights and explore the typical tactile patterns that emerge while grasping objects. The sensor array (548 sensors) is assembled on a knitted glove, and consists of a piezoresistive film connected by a network of conductive thread electrodes that are passively probed. Using the low-cost STAG sensor array (~ $10), we record a large-scale tactile dataset with 135,000 frames, each covering the full hand, while interacting with 26 different objects. The collective set of interactions with different objects explain the key correspondences between different regions of the hand while manipulating objects. Insights from the tactile signatures of the human grasp – through the lens of a manmade analogue of the natural mechanoreceptor network – can aid the future design of new prosthetics, robot grasping tools and human-robot interactions.