3D Object Reconstruction from
Hand-Object Interactions

Dimitrios Tzionas        Juergen Gall



3D Object Reconstruction from Hand-Object Interactions


Abstract

Recent advances have enabled a plethora of 3d object reconstruction approaches using a single off-the-shelf RGB-D camera. Although these approaches are successful for a wide range of object classes, they are based on the assumption of objects with a wealth of stable, distinctive geometrical and/or textural features. However, a lot of objects (household, decorative, toys, mechanical parts, etc) are characterized by a simple, minimalistic shape and high symmetry (e.g. cylindrical or spherical objects) or even lack of texture. Existing in-hand scanning systems and 3d reconstruction techniques fail for such symmetric objects in the absence of highly distinctive features. In this work we show that 3d reconstruction based on low-level features can be facilitated by higher level ones. Although existing in-hand scanning systems simply ignore information originating from the hand, we show that 3D hand motion capture can provide strong and reliable features, effectively facilitating the reconstruction of even featureless, highly symmetrical objects. This work presents the first system that fuses the rich additional information of hands into a 3D reconstruction pipeline, significantly contributing to the state-of-the-art of in-hand scanning.

Publications

Tzionas, D., and Gall, J.
3D Object Reconstruction from Hand-Object Interactions

International Conference on Computer Vision 2015 (ICCV'15)
[pdf] [BibTex] [video] [video spotlight] [extended abstract for OUI workshop] [poster]

Videos








YouTube     Vimeo


Datasets

Water bottle [Frames]  [Preview [Models/Motion/Camera]  [Videos_In/Out]  [Reconstruction]
Bowlling pin [Frames]  [Preview [Models/Motion/Camera]  [Videos_In/Out]  [Reconstruction]
Small bottle [Frames]  [Preview [Models/Motion/Camera]  [Videos_In/Out]  [Reconstruction]
Sphere
[Frames]  [Preview [Models/Motion/Camera]  [Videos_In/Out]  [Reconstruction]
   Camera Parameters
[Camera
   Models (**)
[Hand_Right
Ground-Truth Fingertip Points
[Contained above in *All_Files*]
Sequences Info
[Dataset_ReadMe]  [Dataset_SequencesINFO]
   Software to visualize results in 3D
[Files] or [Github]
   Software to visualize groundtruth
[Files] or [Github]

Model-files marked with (**) do not contain sequence-specific files (.SKEL and .MOTION)



Source Code

[Files] or [Github]


Related Projects

Capturing Hands in Action using Discriminative Salient Points and Physics Simulation, IJCV 2016 [Web]
Capturing Hand Motion with an RGB-D Sensor, Fusing a Generative Model with Salient Points, GCPR 2014 [Web]


Citation

If you find the material in this website useful for your academic work, please cite this work.


Contact

If you have general questions/comments concerning the paper or the website, please contact Dimitrios Tzionas



3D Object Reconstruction from Hand-Object Interactions




3D Object Reconstruction from Hand-Object Interactions