Heterogeneous sensor fusion including multiple Radar/Camera sensor builds up an AI sensing and recognition system. The object speed, location, and distance are estimated and sent to edge micro computing devices for further analysis and application. Applied to Road Side Unit, this technology can detect large-ranged targets and guide the autonomous vehicles or smart vehicles equipping ADAS system. Furthermore, it is also helpful to remind the social distance of infectious disease as COVID-19, count the number of crowd and car at hot sightseeing spot, and update the statistical data on Google Map for epidemic prevention. This sensor fusion system achieves 60-80 meters of range and 10 frames per second at NVidia’s Xavier as well as 4G/5G capability. Compared to the lidar-based RSU, the hardware cost of the proposed RSU system can reduce up to 75% hardware cost and achieve reliable multiple object detection, and tracking capability.

Sensing systems on autonomous vehicles still exist blinding spots and don’t ensure the driving safety. Therefore, importing the networking road side unit, RSU, is necessary to improve the whole safety on the roads. The RSU including camera and radar sensor fusion function can real-time detect and track objects on field trial. The leader of telecommunicate company in Taiwan, Chunghwa Telecomm, is our partner of developing this embedded AI plus intelligent transportation system, ITS, for upgrading Taiwan’s ITS technology in near future.

The casualties of car accidents cause enormous losses every year in Taiwan. On In-coming super senior society, a safer ITS sensing system will play an important role. This technology can be applied on RSU for large-ranged target detection, guiding the autonomous vehicle, and enlarging detection range with sensing capability. Combined with the recognition, sensor fusion, and path-planning technologies, ADAS and specific unmanned vehicles are applicable.