=Siggraph 2006 Emerging Technology=

Hand-Shadow Illusions and 3D DDR Based on Efficient Model Retrieval

Jeng-Sheng Yeh, Jen-Yuan Chiang, Li-Fong Lin, Ting-Hao Huang Ming-Yung Yu, Chung-Lin Wen and Professor Ming Ouhyoung
Graphics group, CMLab, NTU


Based on efficient 3D model retrieval[1] and 2D shape-based object recognition technology, we provide 3 major applications as an installation art system - Hand-Shadow Illusions, Interactive Gallery and 3D DDR. When technology meets entertainment, users will have a whole new experience.

=System Overview=

This system consists of 3 components. Image capturing, image processing and 2D/3D model retrieval.

=Hand-Shadow Illusions=

Users can play hand shadow and will have correspoinding response interactively.

=Interactive Gallery=

Users imitate the pose of a character in some famous art work, and will have interactive response of useful information about this work.

=Body Body Revolution (previous version:3D DDR)=

It is a full-body version DDR. Users make their pose match the hint images dynamically shown on the screen. Users pose their body within limited time with background music playing.


-Enhanced Life

This project integrates image processing, 2D/3D model retrieval, and traditional games to create new installation art.


To allow users to move their bodies or hands to create an image, and use the 2D/3D model retrieval system to evaluate their performances in an interesting game.


The system blends traditional children's games with an up-to-date 2D/3D model retrieval system.


Such a system installed in subway stations or public buildings will encourage people to relax, play, and think. The system may inspire another interaction method with the addition of multiple cameras and a retrival server as background knowledge. It could also be seen as another (not very accurate but very low cost) method for motion capture.


proposal (3.4MB)
BBR video
Hand Shadow video


[1] Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen and Ming Ouhyoung, "On Visual Similarity Based 3D Model Retrieval", Computer Graphics Forum (EUROGRAPHICS'03), Vol. 22, No. 3, pp. 223-232, Sept. 2003.