Technical Name 360° Home Routine Assistant: Fisheye Vision Recognition and Voice Interaction via Multimodal Representation Learning
Project Operator National Tsing Hua University
Project Host 孫民
Summary
To provide better elder home caring service, we recognize users’ daily activities through bird’s eye view 360 fisheye camera  and provide accurate assistance via voice system.

In order to provide the services listed above, scene semantic segmentation, object detection, and human pose estimation techniques are used in our product. All of the methods are taking place offline, preserving users’ privacy.
Scientific Breakthrough
We proposed the model can perform activity recognition for the fisheye imaging. And it achieves 2.75 times improvement than benchmark. Combining with the voice system interacting with people, we are able to further evolve the vision system into intelligent home assistant to provide a variety of functions.
Industrial Applicability
We proposed the model can perform activity recognition for the fisheye imaging. And it achieves 2.75 times improvement than benchmark. Combining with the voice system interacting with people, we are able to further evolve the vision system into intelligent home assistant to provide a variety of functions.
Keyword edge computing voice interaction semantic scene segmentation object detection human pose estimation detection in fisheye vision virtual data collection transfer learning activity recognition smart home assistant