Deep learning plays a crucial role in the era of artificial intelligence. One of the biggest challenges in deep learning is to conduct learning representation to explore the meaningful semantics from the collection of heterogeneous documents. Such a representation is essential to develop different regression and classification solutions to a variety of natural language systems which range from speech assistant to search agent, question and answering, speech recognition and machine translation.  Neural Bayesian learning of sequence data is developed for natural language understanding.

Deep natural language processing and learning is basically constructed on the basis of variational autoencoder. Such a technology can faithfully represent the heterogeneous speech and text data via the specialized deep model in natural language system.  A speech dialogue system is built by integrating different speech and language processing and learning components based on deep reinforcement learning.

Deep natural language processing and learning aims to develop the technology for learning representation for sequence data where the temporal difference learning, the flexible semantic understanding and the neural model compressing are included. The perspectives of probabilistic reasoning and deep learning are systematically integrated to carry out modern natural language systems. Accordingly, the proposed word representation depends on past and future with multiple beliefs and information sources. Also, a new type of deep bidirectional transformer is presented to implement a powerful sentiment or semantic representation from speech or text data. Further, the quantized deep model provides the key to balance tradeoff between system performance and model compression in the implementation.

A software toolkit is built for natural language processing and learning where the probabilistic interpretation for deep model is provided to probe and analyze the learning procedure in deep neural networks. The solution to hardware implementation with adjustable memory cost is obtained. The semantic representation and the sentiment identification are developed. This basic technology is feasible to a number of applications in the areas of natural language processing which include recommendation system, question answering, information retrieval, document summarization, text categorization. The deep machine learning components are designed to implement the related systems including speech recognition, speaker recognition, machine translation, speech enhancement and spoken dialogue system.