Data-driven, Vision-based Tactile Sensing: Design, Simulation, and Applications to Robot Control

Abstract:

The human skin is capable of sensing various types of forces with high resolution and accuracy. The development of an artificial sense of touch for robots needs to address these properties, while retaining scalability to large surface areas of arbitrary shape. This talk describes the development of a soft, vision-based tactile sensor that fully exploits the high resolution of modern cameras, while at the same time offering ease of manufacture. In order to overcome the complexity of interpreting raw tactile data, a machine learning strategy is presented to map from the tactile images to the distribution of (pressure and shear) forces applied to the sensor. The strategy is based on data that are entirely generated via finite element simulations and retains high real-time accuracy on real-world data, while generalizing across different sensors and contact objects. Additionally, the talk presents an application of the tactile sensing technique to dexterous manipulation. Reinforcement learning is employed to achieve highly dynamic motions with control policies entirely trained on a faster-than-real-time simulator and successfully deployed to a real-world robotic system without further adaptation.