A Continuous Emotional Music Generation System Based on Facial Expressions

Facial expressions are one of the most intuitive ways of expressing the human heart, which most naturally expresses a person's present emotions. Music is good at expressing emotions, which is most likely to resonate among all art forms. Therefore, expression and music have a common emotional expression. The framework presented in this paper includes the facial continuous expression recognition model and the continuous emotion music generation model. To achieve continuous emotion recognition, we introduce the Russell 2-dimensional valence-arousal emotion space. The novelty of our approach is the joint prediction of discrete and continuous emotions through facial expression recognition networks, and then the music generation model is driven by the continuous emotion vector to generate music corresponding to the predicted facial emotion. Finally, we used a pre-trained emotion classification network and an emotion average opinion score (MOS) to evaluate the consistency of the generated music emotion and the facial emotion.