Grounding Vision through Haptics and Proprioception

To enhance object recognition performance we investigated strategies and methods for the fusion of proprioceptive, visual and tactile sensory data. The goal is to enrich object representations by additional object attributes such as dimension or deformability, which can be explored by combining different sensorial cues. Different objects which visually appear to be similar can be distinguished. Our ongoing research activities in this field focus on the development of a holistic approach for building multisensorial object representations, which not only support the enhancement of recognition performance but also the robot’s manipulation capabilities.