We develop an end-to-end machine learning system that includes (1) a deep learning model for multiple learning and multiple driving tasks (MT) using various Deep Neural Networks (DNNs) to predict perception indicators from camera images, (2) control algorithms for achieving zero collisions with few indicators from DNNs and radar data from self-driving car, (3) comprehensive comparisons of different DNNs on the performance metrics of their dynamic inference accuracy and computational complexity and efficiency, and (4) an optimal (novel) DNN-MT model for autonomous driving. We have collected 1 million simulator images, 2TB of U.S., and 27.7GB of Taiwan traffic data. We are deploying our algorithms to vehicle perception and control devices to enable them to drive a real car on Taiwan roads. Fig. 1 illustrates the simulation cycle of the architecture. Fig. 2 displays the DNN-MT model. Fig. 3 demonstrates a preliminary road test of the software system on Taiwan＇s freeway.

We propose an end-to-end machine learning model that integrates multi-task (MT) learning, CNNs (convolutional neural networks), and control algorithms to achieve efficient inference and stable driving for self-driving cars. The CNN-MT model can simultaneously perform multiple learning tasks for estimating perception indicators and driving decisions based on the direct perception paradigm of autonomous driving. The model can be used to evaluate the inference efficiency and driving stability of different CNNs on the metrics of CNN＇s size, complexity, accuracy, processing speed, and collision number, respectively, in a dynamic traffic. We propose new algorithms for controllers to drive a car using the indicators and its short-range sensory data to avoid collisions in real-time testing.

This machine learning software system can be installed in any type and any model of vehicles with advanced driving assistance system. It uses many advanced software technologies in vehicle’s electronic devices, mobile phone, and Internet to achieve Level 2 autonomous driving. It can also be used in other industries such as electronic chips, IC design, and precision machinery. It is a core technology in self-driving cars with the most added values, contains rare intellectual properties, and is very difficult to replicate due to complex software engineering combing mathematics, artificial intelligence, automotive computer systems, programming, and data analysis, etc.