Technical Name Panorama Video based Projective Texture 3D Reconstruction System
Project Operator National Taiwan University of ScienceTechnology
Project Host 姚智原
Summary
In this research, we propose a 3D scene reconstruction system. We use panorama image sequence as input,reconstruct a 3D scene. To solve the problem of texture image distortion caused by image discontinuity, our texture is projected onto the model according to the camera’s current position,the texture image is obtained by switching from panorama videos. We can add some interactive 3D objects in the 3D scene,our users can walk through in the 3D scene freely according to the route, rotate the camerainteract with 3D objects.
Scientific Breakthrough
The main contributions of our research as follows: 
 We propose an panorama stitching system, our system can take 4 fisheye images as inputstitch into a panorama image. 
 In the process of image alignment, we proposed a postprocessing method to solve the problem that the image stitching algorithm that cannot align the 360 panoramic images.
 We propose a complete scene reconstruction system where we use only panoramic images as inputreconstruct a 3D scene mesh. 
 We propose a point cloud alignment method which use SLAM system to estimate the path of the camera,we use the SLAM result to align each point cloud.
Industrial Applicability
This system can quickly create a 3D scene with a simple 360 panoramic camera, without relying on Lidar3D scanning devices. The 3D scene created by the system allows users walk freely in the 3D scene,has better interaction with virtual objects. This technic can be applied in the field for online interactive tour services that quickly virtualize physical exhibitions,build a virtual 3D tour for actual venues such as parkscampuses.
Keyword Digital Twin 3D reconstruction Panorama Stitching Online Exhibition Deep learning Spatial Simulation Cyber-Physical Integration Extended Reality Digital Retail Campus Tour
  • Contact
  • Chen Kuo Wei