LLMs to the Moon? Reddit Market Sentiment Analysis with Large Language Models

Xiang Deng, Vasilisa Bashlovkina,Feng Han, Simon Baumgartner,Michael Bendersky

COMPANION OF THE WORLD WIDE WEB CONFERENCE, WWW 2023(2023)

引用 8|浏览123
暂无评分
摘要
Market sentiment analysis on social media content requires knowledge of both financial markets and social media jargon, which makes it a challenging task for human raters. The resulting lack of high-quality labeled data stands in the way of conventional supervised learning methods. In this work, we conduct a case study approaching this problem with semi-supervised learning using a large language model (LLM). We select Reddit as the target social media platform due to its broad coverage of topics and content types. Our pipeline first generates weak financial sentiment labels for Reddit posts with an LLM and then uses that data to train a small model that can be served in production. We find that prompting the LLM to produce Chain-of-Thought summaries and forcing it through several reasoning paths helps generate more stable and accurate labels, while training the student model using a regression loss further improves distillation quality. With only a handful of prompts, the final model performs on par with existing supervised models. Though production applications of our model are limited by ethical considerations, the model's competitive performance points to the great potential of using LLMs for tasks that otherwise require skill-intensive annotation.
更多
查看译文
关键词
Sentiment Analysis,Social Media,Finance,Large Language Model,Natural Language Processing
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要