Technical Name Future Puppet Theater: An Immersive Multiplayer Real-Time Puppeteering System Based on Multi-Display Interaction
Project Operator National Yang Ming Chiao Tung University Institute of Applied Arts
Project Host 許峻誠
Summary
"Future Puppet Theater" is an immersive multiplayer real-time puppeteering system that integrates mobile phone and gesture technology. Using the "IoTtalk" platform, data can be converted into actions for virtual puppets. The system enables a virtual puppet theater experience for multiple users. By using VR gesture tracking and mobile phone sensors, virtual puppets are bound and controlled, allowing the public to experience puppetry through technology.
Scientific Breakthrough
In the field of HCI, digital puppetry uses motion capture and sensors to control virtual characters, and applied it in animation and cultural heritage preservation. Existing systems are complex and hard to learn. Our team developed a system that converts gestures and mobile phone data into virtual puppets' actions, suitable for public and professional user. It offers precision, ease of use, and multi-user collaboration, enabling a multi-user puppetry experience through immersive projection.
Industrial Applicability
The global decline of puppetry culture underscores the need for integrating technology in cultural heritage preservation to achieve SDGs. While many products use VR and immersive projection to enhance audience interaction, easy-to-set-up multi-screen interactive puppetry systems are lacking. Our system integrates mobile phone and gesture puppetry technology for performing arts, VR films, and public screen interactions, showcasing innovation and potential in both culture and industry.
Keyword Digital puppetry Gesture puppetry technology Virtual puppets Motion capture Immersive projection(CAVE system) Interactive entertainment Cultural heritage preservation Participatory theater Multi-screen interaction IoTtalk platform
  • Contact
  • Wei-Chen Yen