MF2former: Multi-Feature Fusion Transformer for Traffic Flow Prediction

In urban planning, traffic flow prediction, a core component of Intelligent Transportation Systems, has made significant progress with the development of deep learning. The key problem of traffic flow prediction lies in capturing the complex spatio-temporal correlations in traffic flow. In recent years, more and more research has tended to apply Transformer-based models to solve this problem. However, Transformer-based models have two major limitations for traffic flow prediction: i) Most methods only focus on extracting data features within the attention head, while ignoring the correlation between these heads, making it difficult to integrate the multi-features of traffic data; ii) Most methods do not recognize the unique impact of nodes that serve as pivotal traffic hubs in the traffic networks, which cause the Transformer to excessively focus on the influence of non-pivotal nodes. In this study, we propose a novel Transformer-based model, the Multi-Feature Fusion Transformer (MF2former), aimed at addressing the above limitations of traffic flow prediction. MF2former incorporates the Augmented Synergistic Transformer Module to achieve a comprehensive multi-feature fusion of traffic data by enhancing the information capacity within self-attention heads and performing information fusion between these heads. Additionally, our model incorporates the Pivotal Node Module, which extracts pivotal nodes from all nodes and masks the global receptive field of the Transformer to enhance its focus on pivotal nodes in the traffic network. Our model is evaluated using two real-world traffic datasets, demonstrating superior performance compared to existing methods. This study provides a stable framework for accurate traffic flow prediction, offering valuable insights for urban planners and commuters.