Advisor — Master Thesis "Estimating Robust Affordances from Vision by Combining Multiple Models"
Supervised and Examined by Oliver Brock & Guillermo Gallego
Student
Patrick Lowin – Now PhD Student with Oliver Brock
Abstract
Estimating potential areas for interaction enables agents to explore their environment and perform meaningful manipulations autonomously. However, perceiving these possible actions, also known as affordances, is challenging due to their high ambiguity in input data. In this thesis, we present a probabilistic method that estimates robust affordances for robotic embodiments in real-world environments, even under adverse conditions. Our approach utilizes information about graspable and movable points as cues for robotic interaction, which we estimate with off-the-shelf neural networks. We then recursively estimate beliefs for each cue to manage the inherent uncertainty of our measurement sources. Then, we apply interconnected recursive estimation to fuse these estimators to resolve ambiguities further and estimate robust robotic affordances. We evaluate our method on a dataset of real-world manipulations involving 14 objects in varying environmental conditions. By fusing beliefs, our approach outperforms baseline methods for affordance estimation by 196% and 309% in well-lit and uncluttered scenes, respectively. Additional experiments highlight the robustness of our approach under adverse conditions, such as dark and cluttered environments. Our findings suggest that interconnected recursive estimation enhances performance and robustness for estimating interaction points without fine-tuning any models.
Related Publication
Accepted at ICRA 26.
More Information
More information can be found here.