Nonrigid Object Recognition and Tracking
Mathieu Salzmann1, Trevor Darrell and Yangqing Jia
In our everyday life, we manipulate many nonrigid objects, such as clothes. In the context of personal robotics, it would therefore be important to correctly recognize and track these objects for a robot to interact with them. While tracking and recognition of rigid objects has received a lot of attention in the Computer Vision community, similar tasks for deformable ones remain mainly unstudied. The main challenges that need to be addressed arise from the much larger appearance variability of such objects. Furthermore, the wide range of shapes that a piece of clothe can undergo makes 3D reconstruction and tracking very challenging.
In this project, we intend to study machine learning and computer vision methods to solve the following problems: 1) Texture-based classification of the different parts of a single objects, e.g. boundaries vs. interior parts. 2) Instance-level recognition of particular pieces of cloth. 3) Category-level / material recognition, e.g. jeans vs. t-shirts. 4) 3D shape estimation and tracking.
Most of the above-mentioned problems can be tackled in a multi-modal context, where different types of input, such as video or laser, are available. Another subject of interest is the study of a principled way of combining these inputs, in particular when they are asynchronous.
Figure 1: Non-rigid object (frame 1)
Figure 2: Non-rigid object (frame 2)
1Postdoc, EECS & ICSI