Advisor — Master Thesis "Tracking Hand Motion to Infer Object Motion Constraints"

Co-Advised with Xing Li; Supervised and Examined by Oliver Brock & Guillermo Gallego; **Winner of the Rolf-Niedermeier-Award**

Student

Adrian Pfisterer – Now PhD Student with Oliver Brock

Abstract

Perceiving the kinematic structure of small objects is challenging due to their delicate nature and small size, which introduce significant uncertainty in visual inputs. We present a novel probabilistic method that leverages human hand tracking to robustly estimate the kinematic structure of small objects, even under partial occlusion. Our approach utilizes human hand landmarks as perceptual priors to directly address and manage uncertainty within the visual inputs, effectively rejecting outliers and yielding accurate, real-time estimations. We validated our method on a diverse dataset from real-world human manipulations of eight small objects. By modeling uncertainty, our system significantly outperformed two baseline methods on this dataset by 66% and 63%, respectively. An ablation study further underscored the critical role of explicitly considering uncertainty to enhance system performance. The findings demonstrate that integrating human-based priors and uncertainty estimation significantly enhances the quality of articulation models, enabling their direct application in robotic manipulation without the need for extensive fine-tuning.

A. Pfisterer, X. Li, V. Mengers and O. Brock, “A Helping (Human) Hand in Kinematic Structure Estimation,” IEEE International Conference on Robotics and Automation (ICRA), 2025, pp. 11918-11925

Award

This thesis won the Rolf-Niedermeier-Award.

More Information

More information can be found here.