Response selector, as an essential component of 001 dialogue systems, aims to pick out an optimal 002 response in a candidate pool to continue the dia-003 logue. The current state-of-the-art methods are 004 mainly based on an encoding paradigm called 005 Cross-Encoder (Urbanek et al., 2019), which 006 separately encodes each context-response pair 007 and ranks the responses according to their fit-008 ness scores. However, such a paradigm is 009 both inefficient and ineffective. Specifically, 010 it has to repeatedly encode the same context for 011 each response, which results in heavy inference 012 cost. Also, without considering the relation-013 ship among the candidates, it is difficult to tell 014 which one is the best candidate purely based 015 on the fitness score of each candidate. To ad-016 dress this problem, we propose a new model 017 called Panoramic-Encoder , which accepts all 018 candidates and the context as inputs at once and 019 allows them to interact with each other through 020 a specially designed attention mechanism. Our 021 method also allows us to naturally integrate 022 some of the effective training techniques, such 023 as the in-batch negative training. Extensive ex-024 periments across four benchmark datasets show 025 that our new method significantly outperforms 026 the current state-of-the-art while achieving ap-027 proximately 3 × speed-up at inference time. 028

The general approaches for polarity analysis in dialogue, e.g. Multiple Instance Learning (MIL), have achieved significant progress. However, one significant drawback of current approaches is that the contribution of an utterance towards the polarity being a black-box. For existing methods, the polarity contained in each utterance, which we call meta-polarity, is not explicitly utilized. In this paper, we study the problem of adding interpretability to the overall polarity by predicting the meta-polarity at the same time. First, we reformulate a large advising dataset [1], where the meta-polarity of each utterance is given. Second, we propose an utterance classification model (UCM) and a two-stage progressive training method that strengthens the connection between the meta-polarity and the overall polarity. Experimental results show that our overall approach outperforms all competitive base-lines by substantial margins, achieving a new state-of-the-art performance on this dataset.

The NOESIS II challenge, as the Track 2 in the Eighth Dialogue System Technology Challenge (DSTC 8), is the extension of Track 1 in DSTC 7. Three new elements are incorporated into the extended track, i.e., dialogue with multiple participants, dialogue success, and dialogue disentanglement. These are vital for the creation of a deployed task-oriented dialogue system. This track is divided into four subtasks, the first two of which are evaluated in the form of response selection and the last two focus on dialogue analysis. This paper describes our methods developed for these four subtasks, which all employ deep contextualized utterance representations to make models aware of contextual information and to keep the intrinsic property of multi-turn dialogue systems. In the released evaluation results of Track 2 in DSTC 8, our proposed methods ranked fourth in subtask 1, third in subtask 2, and first in subtask 3 and subtask 4 respectively. In addition to the challenge tasks, we also compare our proposed methods with previous ones on public benchmark datasets. Experimental results show that our proposed methods outperform existing ones by large margins and achieve new state-of-the-art performances on multi-turn response selection and dialogue disentanglement.

Disentanglement is a problem in which multiple conversations occur in the same channel simultaneously, and the listener should decide which utterance is part of the conversation he will respond to. We propose a new model, named Dialogue BERT (DialBERT), which integrates local and global semantics in a single stream of messages to disentangle the conversations that mixed together. We employ BERT to capture the matching information in each utterance pair at the utterance-level, and use a BiLSTM to aggregate and incorporate the context-level information. With only a 3% increase in parameters, a 12% improvement has been attained in comparison to BERT, based on the F1-Score. The model achieves a state-of-the-art result on the a new dataset proposed by IBM and surpasses previous work by a substantial margin.

In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model aware of the speaker change information, which is an important and intrinsic property of multi-turn dialogues. Furthermore, a speaker-aware disentanglement strategy is proposed to tackle the entangled dialogues. This strategy selects a small number of most important utterances as the filtered context according to the speakers' information in them. Finally, domain adaptation is performed to incorporate the in-domain knowledge into pre-trained language models. Experiments on five public datasets show that our proposed model outperforms the present models on all metrics by large margins and achieves new state-of-the-art performances for multi-turn response selection.

“Interest”is a critical bridge between cognitive and effective issues in learning. Student’s interest has great impact on learning performance. Hence, it’s necessary to detect student’s interest and make them more engaged in the learning process for productive learning. Student’s interest can be detected based on the facial expression recognition, e.g., smile recognition. However, various head poses, different illumination, occlusion and low image resolution make smile recognition difficult. In this paper, a conditional random forest based approach is proposed to recognize spontaneous smile in natural environment. First, image patches are extracted within the eye and mouth regions instead of the whole face to improve the robustness and efficiency. Then, the conditional random forests based approach is presented to learn the relations between image patches and the smile/non-smile features conditional to head poses. Furthermore, a K-means based voting method is introduced to improve the discrimination capability of the approach. Experiments have been carried out with different spontaneous facial expression databases. The encouraging results suggest a strong potential for interest detection in natural environment.

Head pose estimation (HPE) is important in human–machine interfaces. However, various illumination, occlusion, low image resolution and wide scene make the estimation task difficult. Hence, a Dirichlet-tree distribution enhanced Random Forests approach (D-RF) is proposed in this paper to estimate head pose efficiently and robustly in unconstrained environment. First, positive/negative facial patch is classified to eliminate influence of noise and occlusion. Then, the D-RF is proposed to estimate the head pose in a coarse-to-fine way using more powerful combined texture and geometric features of the classified positive patches. Furthermore, multiple probabilistic models have been learned in the leaves of the D-RF and a composite weighted voting method is introduced to improve the discrimination capability of the approach. Experiments have been done on three standard databases including two public databases and our lab database with head pose spanning from −90° to 90° in vertical and horizontal directions under various conditions, the average accuracy rate reaches 76.2% with 25 classes. The proposed approach has also been evaluated with the low resolution database collected from an overhead camera in a classroom, the average accuracy rate reaches 80.5% with 15 classes. The encouraging results suggest a strong potential for head pose and attention estimation in unconstrained environment.

Head pose and facial feature detection are important for face analysis. However, many studies reported good results in constrained environment, the performance could be decreased due to the high variations in facial appearance, poses, illumination, occlusion, expression and make-up. In this paper, we propose a hierarchical regression approach, Dirichlet-tree enhanced random forests (D-RF) for face analysis in unconstrained environment. D-RF introduces Dirichlet-tree probabilistic model into regression RF framework in the hierarchical way to achieve the efficiency and robustness. To eliminate noise influence of unconstrained environment, facial patches extracted from face area are classified as positive or negative facial patches, only positive facial patches are used for face analysis. The proposed hierarchical D-RF works in two iterative procedures. First, coarse head pose is estimated to constrain the facial features detection, then the head pose is updated based on the estimated facial features. Second, the facial feature localization is refined based on the updated head pose. In order to further improve the efficiency and robustness, multiple probabilitic models are learned in leaves of the D-RF, i.e. the patch's classification, the head pose probabilities, the locations of facial points and face deformation models (FDM). Moreover, our algorithm takes a composite weight voting method, where each patch extracted from the image can directly cast a vote for the head pose or each of the facial features. Extensive experiments have been done with different publicly available databases. The experimental results demonstrate that the proposed approach is robust and efficient for head pose and facial feature detection.

A dynamic facial expression recognition method based on the auto-regressive (AR) models using combined features of both shape and texture features is proposed in this paper. The AR model is effective to model complicated facial motions. In this work, six AR models are first learned for six basic expressions based on the fusion of shape and texture features of the difference between the neutral image and expressive face image. The difference tends to focus the facial parts that are changed from the neutral to expressive face and eliminate the influence of identity of the facial image. The shape features are facial feature point displacements between the normalized neutral and expressive face images while the texture features are local texture. Then the AR models are used to generate the predicted sequence which is compared with the actual sequence. The corresponding expression is inferred from the most similar predicted sequence to the actual one. Finally a line segment based method is proposed to compute the similarity between the predicted and actual expression sequences. The experiments have been conducted based on the extended Cohn-Kanade database. Encouraging results suggest a strong potential for dynamic facial expression.