Alexander L. Burka
aburka (at) seas (dot) upenn (dot) edu
Haptics Group Research: Surface Texture Perception
| Poster || (Haptics Symposium, Philadelphia, 4/2016) |
| WIP paper || (Haptics Symposium, non-archival) |
| Conference paper #1 || (MFI, Baden-Baden, 9/2016) |
| Workshop paper #1 || (AAAI SSS 2017, Palo Alto, 3/2017) |
| Conference paper #2 || (ICRA 2017, Singapore, 5/2017) |
| Conference paper #3 || (World Haptics 2017, Fürstenfeldbruck, 6/2017) |
The purpose of this research is to explore multimodal surface perception. Most mobile robots have cameras, while relatively few have dedicated haptic sensors. (However, many have sensors such as accelerometers which can be repurposed for haptic perception.) Moreover, while research in computer vision is very well developed with large public datasets available, there are still many open questions as how best to identify objects from haptic sensations, and little data. This project aims to create a multimodal sensing device, use it to collect a large dataset of surface texture percepts, and (in collaboration with Trevor Darrell's lab at UC Berkeley) develop new algorithms for haptic surface classification. We are especially interested in using artificial neural networks to learn correspondences between a surface's visual appearance and haptic feel.
Here are the major components:
- Proton Pack: We have developed a portable sensing device called the Proton Pack, because it looks like the Ghostbusters' energy weapon. Its sensor suite includes two cameras (a high-resolution RGB camera and a low-resolution depth camera), a six-axis force/torque sensor, a digital IMU, high-bandwidth accelerometers, and interchangeable end-effectors including a tooling ball, OptoForce force sensor, and SynTouch BioTac artificial fingertip. These end-effectors are designed to elicit different material properties at the contact point and emulate the varying haptic capabilities of different robots. Using the camera and IMU, the Proton Pack can track its own motion (though in the early experiments we use a Vicon tracker for ground truth data). A backpack contains an onboard computer and battery to make the system fully portable. Experiments are controlled by a simple smartphone interface.
- Analysis: In our proof-of-concept experiments, we have gathered data about the texture of five sample surfaces selected from the Penn Haptic Texture Toolkit, and shown that the Proton Pack's data is sufficient to classify them at over 80% accuracy using an SVM. In time, we will do more advanced analysis using deep neural networks, because deep learning is the solution to all problems.
- Video explainer featuring the first prototype