Attention as Robust Representation for Time Series Forecasting
CoRR(2024)
摘要
Time series forecasting is essential for many practical applications, with
the adoption of transformer-based models on the rise due to their impressive
performance in NLP and CV. Transformers' key feature, the attention mechanism,
dynamically fusing embeddings to enhance data representation, often relegating
attention weights to a byproduct role. Yet, time series data, characterized by
noise and non-stationarity, poses significant forecasting challenges. Our
approach elevates attention weights as the primary representation for time
series, capitalizing on the temporal relationships among data points to improve
forecasting accuracy. Our study shows that an attention map, structured using
global landmarks and local windows, acts as a robust kernel representation for
data points, withstanding noise and shifts in distribution. Our method
outperforms state-of-the-art models, reducing mean squared error (MSE) in
multivariate time series forecasting by a notable 3.6
core neural network architecture. It serves as a versatile component that can
readily replace recent patching based embedding schemes in transformer-based
models, boosting their performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要