MEGAnno+: A Human-LLM Collaborative Annotation System

Large language models (LLMs) can label data faster and cheaper than humansfor various NLP tasks. Despite their prowess, LLMs may fall short inunderstanding of complex, sociocultural, or domain-specific context,potentially leading to incorrect annotations. Therefore, we advocate acollaborative approach where humans and LLMs work together to produce reliableand high-quality labels. We present MEGAnno+, a human-LLM collaborativeannotation system that offers effective LLM agent and annotation management,convenient and robust LLM annotation, and exploratory verification of LLMlabels by humans.