A facial expression is a form of nonverbal communication that reflects a person＇s emotion by contracting the muscular patterns on the face. It includes the typical six emotions: disgust, sadness, happiness, fear, anger, and surprise. Another type of facial expression, named micro-expression, is a very brief and rapid facial emotion that is provoked involuntarily, revealing ones’ true feelings. A micro-expression is both micro (short duration with less than 1/5 of a second) and subtle (small intensity) in nature.  A complete automated micro-expression recognition system that is designed based on deep learning architecture is proposed to recognize the micro-expressions. Specifically, the feature extraction techniques involved are optical flow algorithms and deep learning architecture. Promising recognition rates were achieved compared to the state-of-the-art methods when evaluated on 3 prominent databases.

A novel shallow deep learning architecture (namely Shallow Triple Stream Three-dimensional CNN (STSTNet)) is proposed that is computationally light whilst capable of extracting discriminative high-level features and details of micro-expressions. The network learns from three optical flow features (i.e., optical strain, horizontal and vertical optical flow fields) computed based on the onset and apex frames of each video. The proposed method exhibited an unweighted average recall rate of 0.76 and an unweighted F1-score of 0.74 on the composite database consisting of 442 samples from the 3 publicly available databases. It was ranked in 2nd place in Facial-Micro-Expression Grand Challenge Workshop organized in IEEE International Conference on Automatic Face and Gesture Recognition (FG 2019).

By proposing a shallow deep learning architecture that can effectively and efficiently in identifying micro-expressions that are appropriate to be implemented in a real-time application, it is definitely beneficial to the society in several domains due to its viability in a broad range of applications. For example, it can be applied to monitor the depression in the applications of medical diagnosis, to detect suspects’ lies during police interrogation, securing public safety, and to facilitate effective negotiations, sales, and recruitment.