Learning How Objects Function via Co-Analysis of Interactions

ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016)

Ruizhen Hu¹        Oliver van Kaick²        Bojian Wu³        Hui Huang^1,3*        Ariel Shamir⁴        Hao Zhang⁵

^{1Shenzhen University        2Carleton University        3SIAT        4The Interdisciplinary Center        5Simon Fraser University}   

Figure 1: We learn how a category of 3D objects function. This may lead us to discover that the geometry of a chair could allow it to function as a desk or handcart (top). One could produce functional hybrids (bottom), where different functionalities discovered from two objects are integrated into a multi-functional product.

Abstract 

We introduce a co-analysis method which learns a functionality model for an object category, e.g., strollers or backpacks. Like previous works on functionality, we analyze object-to-object interactions and intra-object properties and relations. Differently from previous works, our model goes beyond providing a functionality-oriented descriptor for a single object; it prototypes the functionality of a category of 3D objects by co-analyzing typical interactions involving objects from the category. Furthermore, our co-analysis localizes the studied properties to the specific locations, or surface patches, that support specific functionalities, and then integrates the patch-level properties into a category functionality model. Thus our model focuses on the how, via common interactions, and where, via patch localization, of functionality analysis.

Given a collection of 3D objects belonging to the same category, with each object provided within a scene context, our co-analysis yields a set of proto-patches, each of which is a patch prototype supporting a specific type of interaction, e.g., stroller handle held by hand. The learned category functionality model is composed of proto-patches, along with their pairwise relations, which together summarize the functional properties of all the patches that appear in the input object category. With the learned functionality models for various object categories serving as a knowledge base, we are able to form a functional understanding of an individual 3D object, without a scene context. With patch localization in the model, functionality-aware modeling, e.g., functional object enhancement and the creation of functional object hybrids, is made possible.

Download and Reference  

To reference our algorithm, code or data in a publication, please include the bibtex below and a link to this website.

Acknowledgments

We thank all the reviewers for their comments and suggestions. This work was supported in part by grants from NSFC (61522213, 61528208, 61379090), 973 Program (2014CB360503, 2015CB352501), Guangdong Science and Technology Program (2015A030312015, 2014B050502009, 2014TX01X033), Shenzhen Innovation Program (JCYJ20151015151249564), NSERC (611370 and 2015-05407) and ISF-NSFC (2216/15).

BibTex

@article{HU2016,

     title = {Learning How Objects Function via Co-Analysis of Interactions },

     author = {Ruizhen Hu and Oliver van Kaick and Bojian Wu and Hui Huang and Ariel Shamir and Hao Zhang},

     journal = {ACM Transactions on Graphics (Proc. SIGGRAPH)},

     volume = {35},

     number = {4},

     year = {2016},

     pages = {47:1--47:13},   

}

Downloads