Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark
arxiv(2024)
摘要
Large Language Models (LLMs) offer the potential for automatic time series
analysis and reporting, which is a critical task across many domains, spanning
healthcare, finance, climate, energy, and many more. In this paper, we propose
a framework for rigorously evaluating the capabilities of LLMs on time series
understanding, encompassing both univariate and multivariate forms. We
introduce a comprehensive taxonomy of time series features, a critical
framework that delineates various characteristics inherent in time series data.
Leveraging this taxonomy, we have systematically designed and synthesized a
diverse dataset of time series, embodying the different outlined features. This
dataset acts as a solid foundation for assessing the proficiency of LLMs in
comprehending time series. Our experiments shed light on the strengths and
limitations of state-of-the-art LLMs in time series understanding, revealing
which features these models readily comprehend effectively and where they
falter. In addition, we uncover the sensitivity of LLMs to factors including
the formatting of the data, the position of points queried within a series and
the overall time series length.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要