Simulating the GelSight Sensors: From Physics to Data

In recent years, vision-based high-resolution tactile sensors such as GelSight have been widely used because the rich signal provides useful information regarding the state of the robot and the environment. However, a major barrier for applying tactile sensors like GelSight is the cost. To make the tactile sensors more accessible to a broader community, we propose to build a simulation model for vision-based tactile sensors like GelSight. We explore two modeling methods for making the model: a physically-based method that uses rendering technologies to simulate the sensor's optical design and a data-driven method that uses a small amount of example data to model the sensor quickly. With the help of our simulation model, we can make a robot to collect realistic tactile data in a simulation environment. This will help roboticists test the system design and sensor choice before setting up the real robots and collecting large-scale tactile datasets at a much lower cost. The simulation model will also lead us to have a deeper understanding of the sensor design and improve the sensor design and system design.