Fine-grained Sentiment Semantic Analysis and Matching of Music and Image

Both music and images are rich in emotional information. This paper focuses on the fine-grained emotion semantic within and between acoustic and visual contents and establish the relevant matching model. Through three subjective evaluation experiments, we obtained the fine-grained emotion evaluation score of music and images in different modalities. Combined with the factor analysis method, the sentiment semantic association of image, music and the joint action of music and images are analyzed. Based on the joint emotion space of images and music, we proposed several cross-modal matching methods from the perspective of emotional consistency. Finally, the validation experiments show that the synchronous matching algorithm with the common factor distance and emotion distance discrimination has the best rating results of viewing experience. The research provides experimental data for the intelligent retrieval and production of music and image content, which lays the foundation for the automatic matching of audio-visual works.