Recent advances in generative artificial intelligence, particularly large language models (LLMs), have opened new opportunities for enhancing recommender systems (RecSys). Most existing LLM-based RecSys approaches operate in a discrete space, using vector-quantized tokenizers to align with the inherent discrete nature of language models. However, these quantization methods often result in lossy tokenization and suboptimal learning, primarily due to inaccurate gradient propagation caused by the non-differentiable argmin operation in standard vector quantization. Inspired by the emerging trend of embracing continuous tokens in language models, we propose ContRec, a novel framework that seamlessly integrates continuous tokens into LLM-based RecSys. Specifically, ContRec consists of two key modules: a sigma-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference. The tokenizer is trained with a continuous Variational Auto-Encoder (VAE) objective, where three effective techniques are adopted to avoid representation collapse. By conditioning on the previously generated tokens of the LLM backbone during user modeling, the Dispersive Diffusion module performs a conditional diffusion process with a novel Dispersive Loss, enabling high-quality user preference generation through next-token diffusion. Finally, ContRec leverages both the textual reasoning output from the LLM and the latent representations produced by the diffusion model for Top-K item retrieval, thereby delivering comprehensive recommendation results. Extensive experiments on four datasets demonstrate that consistently outperforms both traditional and SOTA LLM-based recommender systems. Our results highlight the potential of continuous tokenization and generative modeling for advancing the next generation of recommender systems.

Cross-domain recommendation is a core problem in content-to-e-commerce platforms. Its objective is to leverage user interactions with content to infer potential purchasing intent on the e-commerce side, thereby enhancing conversion rates and commercial value. However, in real industrial scenarios, cross-domain recommendation faces multiple challenges: significant semantic gaps exist between different domains, and user cross-domain behavior sequences are often massive in scale and rich in noise. Although large language models (LLMs) possess powerful semantic understanding and reasoning capabilities, their millisecond-level inference latency makes direct application in online recommendation systems difficult. To address these issues, this paper introduces AIR (Atomic Intent Reasoning), an LLM-driven cross-domain recommendation framework designed for industrial-grade deployment. By migrating LLM inference to the offline phase and dynamically constructing user intent representations through efficient retrieval and composition during online operations, it achieves approximately 400* inference acceleration while maintaining semantic consistency. Experimental results across multiple public datasets demonstrate that our method achieves state-of-the-art performance in cross-domain recommendation tasks. Furthermore, large-scale online A/B testing conducted in Kuaishou E-commerce's real-world business scenarios shows that our approach delivers stable and significant improvements across multiple core business metrics, including a +3.446

The rapid advancement of AI-powered smart glasses, one of the hottest wearable devices, has unlocked new frontiers for multimodal interaction, with Visual Question Answering (VQA) over external knowledge sources emerging as a core application. Existing Vision Language Models (VLMs) adapted to smart glasses are typically trained and evaluated on traditional multimodal datasets; however, these datasets lack the variety and realism needed to reflect smart glasses usage scenarios and diverge from their specific challenges, where accurately identifying the object of interest must precede any external knowledge retrieval. To bridge this gap, we introduce SUPERGLASSES, the first comprehensive VQA benchmark built on real-world data entirely collected by smart glasses devices. SUPERGLASSES comprises 2,422 egocentric image-question pairs spanning 14 image domains and 8 query categories, enriched with full search trajectories and reasoning annotations. We evaluate 26 representative VLMs on this benchmark, revealing significant performance gaps. To address the limitations of existing models, we further propose SUPERLENS, a multimodal smart glasses agent that enables retrieval-augmented answer generation by integrating automatic object detection, query decoupling, and multimodal web search. Our agent achieves state-of-the-art performance, surpassing GPT-4o by 2.19 percent, and highlights the need for task-specific solutions in smart glasses VQA scenarios.

Sequential recommendation methods are crucial in modern recommender systems for their remarkable capability to understand a user’s changing interests based on past interactions. However, a significant challenge faced by current methods (e.g., RNN- or Transformer-based models) is to effectively and efficiently capture users’ preferences by modeling long behavior sequences, which impedes their various applications like short video platforms where user interactions are numerous. Recently, an emerging architecture named Mamba , built on state space models (SSM) with efficient hardware-aware designs, has showcased the tremendous potential for sequence modeling, presenting a compelling avenue for addressing the challenge effectively. Inspired by this, we propose a novel generic and efficient framework ( SSD4Rec ) for sequential recommendations, which explores the seamless adaptation of Mamba for recommendations. Specifically, SSD4Rec marks the long-length item sequences with sequence registers and processes the item representations with a novel Masked Bidirectional Structured State Space Duality block. This not only allows for hardware-aware matrix multiplication but also empowers outstanding capabilities in variable-length and long-range sequence modeling. Extensive evaluations on four benchmark datasets demonstrate that the proposed model achieves state-of-the-art performance while maintaining near-linear scalability with user sequence length. Our implementation based on PyTorch is available at https://github.com/ZhangYifeng1995/SSD4Rec .

Accurate and efficient multivariate time series (MTS) analysis is increasingly critical for a wide range of intelligent applications. Within this realm, Transformers have emerged as the predominant architecture due to their strong ability to capture pairwise dependencies. However, Transformer-based models suffer from quadratic computational complexity and high memory overhead, limiting their scalability and practical deployment in long-term and large-scale MTS modeling. Recently, Mamba has emerged as a promising linear-time alternative with high expressiveness. Nevertheless, directly applying vanilla Mamba to MTS remains suboptimal due to three key limitations: (i) the lack of explicit cross-variate modeling, (ii) difficulty in disentangling the entangled intra-series temporal dynamics and inter-series interactions, and (iii) insufficient modeling of latent time-lag interaction effects. These issues constrain its effectiveness across diverse MTS tasks. To address these challenges, we propose DeMa, a dual-path delay-aware Mamba backbone. DeMa preserves Mamba's linear-complexity advantage while substantially improving its suitability for MTS settings. Specifically, DeMa introduces three key innovations: (i) it decomposes the MTS into intra-series temporal dynamics and inter-series interactions; (ii) it develops a temporal path with a Mamba-SSD module to capture long-range dynamics within each individual series, enabling series-independent, parallel computation; and (iii) it designs a variate path with a Mamba-DALA module that integrates delay-aware linear attention to model cross-variate dependencies. Extensive experiments on five representative tasks, long- and short-term forecasting, data imputation, anomaly detection, and series classification, demonstrate that DeMa achieves state-of-the-art performance while delivering remarkable computational efficiency.

Deep learning (DL), as a vital technique, has sparked a notable revolution in AI, resulting in a great change in human lifestyles. As one of the most representative DL techniques, the Transformer architecture has empowered numerous advanced models, especially the large language models (LLMs) that comprise billions of parameters, becoming a cornerstone in deep learning. Despite the impressive achievements, Transformers still face inherent limitations, particularly the time-consuming inference resulting from the quadratic computation complexity of attention calculation. Recently, a novel architecture named Mamba , drawing inspiration from classical state space models (SSMs), has emerged as a promising alternative for building foundation models, delivering comparable modeling abilities to Transformers while preserving near-linear scalability concerning sequence length. This has sparked an increasing number of studies actively exploring Mamba’s potential to achieve impressive performance across diverse domains. Given such rapid evolution, there is a critical need for a systematic review that consolidates existing Mamba-empowered models, offering a comprehensive understanding of this emerging model architecture. In this survey, we therefore conduct an in-depth investigation of recent Mamba-associated studies, covering three main aspects: the advancements of Mamba-based models , the techniques of adapting Mamba to diverse data , and the applications where Mamba can excel . Specifically, we first review the foundational knowledge of various representative deep learning models and the details of Mamba-1&2 as preliminaries. Then, to showcase the significance of Mamba for AI, we comprehensively review the related studies focusing on Mamba models’ architecture design, data adaptability, and applications. Finally, we present a discussion of current limitations and explore various promising research directions to provide deeper insights for future investigations.

With the advancement of web techniques, they have significantly revolutionized various aspects of people's lives. Despite the importance of the web, many tasks performed on it are repetitive and time-consuming, negatively impacting the overall quality of life. To efficiently handle these tedious daily tasks, one of the most promising approaches is to advance autonomous agents to incorporate human-like intelligence based on Artificial Intelligence (AI) techniques, referred to as AI Agents. AI Agents offer significant advantages in handling such tasks since they can operate continuously without fatigue or performance degradation. Therefore, leveraging AI Agents - termed WebAgents in the context of web - to automatically assist people in handling tedious daily tasks can dramatically enhance productivity and efficiency. Recently, Large Foundation Models (LFMs) containing billions of parameters have exhibited human-like language understanding and reasoning capabilities, showing proficiency in performing various complex tasks. This naturally raises the question: 'Can LFMs be utilized to develop powerful AI Agents that automatically handle web tasks, providing significant convenience to users?' To fully explore the potential of LFMs, extensive research has emerged on WebAgents designed to complete daily web tasks according to user instructions, significantly enhancing the convenience of daily human life. In this survey, we comprehensively review existing research studies on WebAgents across three key aspects: architectures, training, and trustworthiness. Additionally, several promising directions for future research are explored to provide deeper insights.

The recent surge in electric vehicles (EVs), driven by a collective push to enhance global environmental sustainability, has underscored the significance of exploring EV charging prediction. To catalyze further research in this domain, we introduce UrbanEV — an open dataset showcasing EV charging space availability and electricity consumption in a pioneering city for vehicle electrification, namely Shenzhen, China. UrbanEV offers a rich repository of charging data (i.e., charging occupancy, duration, volume, and price) captured at hourly intervals across an extensive six-month span for over 20,000 individual charging stations. Beyond these core attributes, the dataset also encompasses diverse influencing factors like weather conditions and spatial proximity. Comprehensive experiments have been conducted to showcase the predictive capabilities of various models, including statistical, deep learning, and transformer-based approaches, using the UrbanEV dataset. This dataset is poised to propel advancements in EV charging prediction and management, positioning itself as a benchmark resource within this burgeoning field.

Parking occupancy prediction enables many smart services in intelligent transportation systems, such as parking guidance and dynamic pricing. Previous studies have improved regional parking occupancy prediction accuracy through many methods, including optimizing model structure and pursuing effective training methods. Moreover, current research realizes that travel behavior is related to multiple sources of external factors, such as spatial relationship and weather. These external factors may not only be real-time information that can be expressed numerically, but also a state that can have a long-term impact and is difficult to represent with time-series information. It remains a challenge to consider both these temporal and heterogeneous information in the network. To fill the research gap, we propose a spatio-temporal parking occupancy prediction approach incorporating heterogeneous and real-time weather as external auxiliary data, named Meta-HWFST, which leverages the integration of innovative global-adjacent dual-attention, multivariate decoder and model-agnostic meta-learning pretraining. The proposed approach is evaluated on a real world dataset of 58 parking lots. The results empirically showed that Meta-HWFST achieved an average improvement of 32.45% in accuracy, compared to other representative models. We conduct a comprehensive ablation experiment to demonstrate the contribution of each module to the improvement. Moreover, we demonstrate that our model could reasonably assign weights through feature importance analysis.

There is a growing interest in utilizing large-scale language models (LLMs) to advance next-generation Recommender Systems (RecSys), driven by their outstanding language understanding and in-context learning capabilities. In this scenario, tokenizing (i.e., indexing) users and items becomes essential for ensuring a seamless alignment of LLMs with recommendations. While several studies have made progress in representing users and items through textual contents or latent representations, challenges remain in efficiently capturing high-order collaborative knowledge into discrete tokens that are compatible with LLMs. Additionally, the majority of existing tokenization approaches often face difficulties in generalizing effectively to new/unseen users or items that were not in the training corpus. To address these challenges, we propose a novel framework called TokenRec, which introduces not only an effective ID tokenization strategy but also an efficient retrieval paradigm for LLM-based recommendations. Specifically, our tokenization strategy, Masked Vector-Quantized (MQ) Tokenizer, involves quantizing the masked user/item representations learned from collaborative filtering into discrete tokens, thus achieving a smooth incorporation of high-order collaborative knowledge and a generalizable tokenization of users and items for LLM-based RecSys. Meanwhile, our generative retrieval paradigm is designed to efficiently recommend top-K items for users to eliminate the need for the time-consuming auto-regressive decoding and beam search processes used by LLMs, thus significantly reducing inference time. Comprehensive experiments validate the effectiveness of the proposed methods, demonstrating that TokenRec outperforms competitive benchmarks, including both traditional recommender systems and emerging LLM-based recommender systems.

Electric vehicles (EVs) are driving green and low-carbon transport in modern cities. It makes charging station occupancy prediction (CSOP) critual for intelligent transportation systems (ITS) to achieve a balance between the supply and demand in resolving the dynamics between EVs and changing stations. Even though several Big Data-based solutions have been discussed, they are still struggling to collaboratively utilize heterogeneous data and distributed computing resources located at both physically and logicially isolated charging stations to better support context-driven CSOP. To addres this challenge, we propose an Asynchronous Federated Meta-learning Mechanism (AFML) for CSOP, which can train a meta-model with strong adaptation ability in an asynchronous and collaborative manner. In general, it incorporates an adaptive reptile algorithm (AR) and an weighted aggregation strategy (WA) to jointly ensure the training efficiency and model adaptivity. Evaluations on real-world CSOP datasets demonstrate that compared to the second best method, AFML can significantly improve forecasting accuracy by 14%, accelerate model convergence by 9% and enhance model generalizability by 10%, illustrating its merits in support CSOP to embrace a smart and sustainable city.

Benchmarking the capabilities and limitations of large language models (LLMs) in graph-related tasks is becoming an increasingly popular and crucial area of research. Recent studies have shown that LLMs exhibit a preliminary ability to understand graph structures and node features. However, the potential of LLMs in graph pattern mining remains largely unexplored. This is a key component in fields such as computational chemistry, biology, and social network analysis. To bridge this gap, this work introduces a comprehensive benchmark to assess LLMs' capabilities in graph pattern tasks. We have developed a benchmark that evaluates whether LLMs can understand graph patterns based on either terminological or topological descriptions. Additionally, our benchmark tests the LLMs' capacity to autonomously discover graph patterns from data. The benchmark encompasses both synthetic and real datasets, and a variety of models, with a total of 11 tasks and 7 models. Our experimental framework is designed for easy expansion to accommodate new models and datasets. Our findings reveal that: (1) LLMs have preliminary abilities to understand graph patterns, with O1-mini outperforming in the majority of tasks; (2) Formatting input data to align with the knowledge acquired during pretraining can enhance performance; (3) The strategies employed by LLMs may differ from those used in conventional algorithms.

Proteins inherently possess a consistent sequence-structure duality. The abundance of protein sequence data, which can be readily represented as discrete tokens, has driven fruitful developments in protein language models (pLMs). A key remaining challenge, however, is how to effectively integrate continuous structural knowledge into pLMs. Current methods often discretize protein structures to accommodate the language modeling framework, which inevitably results in the loss of fine-grained information and limits the performance potential of multimodal pLMs. In this paper, we argue that such concerns can be circumvented: a sequence-based pLM can be extended to incorporate the structure modality through continuous tokens, i.e., high-fidelity protein structure latents that avoid vector quantization. Specifically, we propose a hybrid diffusion protein language model, HD-Prot, which embeds a continuous-valued diffusion head atop a discrete pLM, enabling seamless operation with both discrete and continuous tokens for joint sequence-structure modeling. It captures inter-token dependencies across modalities through a unified absorbing diffusion process, and estimates per-token distributions via categorical prediction for sequences and continuous diffusion for structures. Extensive empirical results show that HD-Prot achieves competitive performance in unconditional sequence-structure co-generation, motif-scaffolding, protein structure prediction, and inverse folding tasks, performing on par with state-of-the-art multimodal pLMs despite being developed under limited computational resources. It highlights the viability of simultaneously estimating categorical and continuous distributions within a unified language model architecture, offering a promising alternative direction for multimodal pLMs.

As one of the most representative DL techniques, Transformer architecture has empowered numerous advanced models, especially the large language models (LLMs) that comprise billions of parameters, becoming a cornerstone in deep learning. Despite the impressive achievements, Transformers still face inherent limitations, particularly the time-consuming inference resulting from the quadratic computation complexity of attention calculation. Recently, a novel architecture named Mamba, drawing inspiration from classical state space models (SSMs), has emerged as a promising alternative for building foundation models, delivering comparable modeling abilities to Transformers while preserving near-linear scalability concerning sequence length. This has sparked an increasing number of studies actively exploring Mamba's potential to achieve impressive performance across diverse domains. Given such rapid evolution, there is a critical need for a systematic review that consolidates existing Mamba-empowered models, offering a comprehensive understanding of this emerging model architecture. In this survey, we therefore conduct an in-depth investigation of recent Mamba-associated studies, covering three main aspects: the advancements of Mamba-based models, the techniques of adapting Mamba to diverse data, and the applications where Mamba can excel. Specifically, we first review the foundational knowledge of various representative deep learning models and the details of Mamba-1&2 as preliminaries. Then, to showcase the significance of Mamba for AI, we comprehensively review the related studies focusing on Mamba models' architecture design, data adaptability, and applications. Finally, we present a discussion of current limitations and explore various promising research directions to provide deeper insights for future investigations.

The increasing popularity of electric vehicles (EVs) in recent times has introduced considerable load conditions for urban power grids and transportation systems, which highlights the importance of accurately predicting charging demand to enhance charging efficiency. However, current forecasting methods still face challenges in effectively aligning diverse data and generating accurate predictions that can be applied to unseen scenarios. To overcome the challenges, this work introduces a novel perspective: employing large language models (LLMs) as EV charging demand predictors. First, we reformulate the prediction task into a text-to-text format, enabling seamless and effective alignment of various features within a unified language semantic space. Subsequently, we fine-tune a LLM using a meta-learning framework to adapt it specifically for EV charging prediction. Through comprehensive evaluations, it has been demonstrated that the proposed model, ChatEV, achieves outstanding performance in EV charging demand forecasting, particularly in scenarios with limited data.

Along with the proliferation of electric vehicles (EVs), optimizing the use of EV charging space can significantly alleviate the growing load on intelligent transportation systems. As the foundation to achieve such an optimization, a spatiotemporal method for EV charging demand prediction in urban areas is required. Although several solutions have been proposed by using data-driven deep learning methods, it can be that these performance-oriented approaches may struggle to correctly understand the underlying factors influencing charging demand, particularly charging prices. A representative case that highlights the challenge faced by existing methods is their potential misinterpretation of high prices during peak times, leading to an incorrect assumption that higher prices correspond to increased demand. To address the challenges associated with training an accurate and reliable prediction model for EV charging demand, this paper proposes a novel approach called PAG, which leverages the integration of graph and temporal attention mechanisms for effective feature extraction and introduces physics-informed meta-learning in the pre-training step to facilitate prior knowledge learning. Evaluation results on a dataset of 18,061 EV charging piles in Shenzhen, China, show that the proposed approach can achieve state-of-the-art forecasting performance and the ability to understand the adaptive changes in charging demands caused by price fluctuations.

A growing number of electric vehicles (EVs) is putting pressure on smart charging services. As a foundation of informing drivers of vacant charging facilities and rationalizing pricing, an effective approach for predicting citywide spatio-temporal EV charging demand that incorporates pricing information is required. Although many deep learning models have been carried out with the orientation of improving prediction accuracy and achieved good results, there is still a lack of prediction models that are economically interpretable and can consider the spillover effect of price adjustments on EV charging demand. To fill the research gaps, we propose a learning approach for accurate EV charging demand prediction and reasonable pricing, named PIAST, which enables the integration of convolutional feature engineering, spatio-temporal dual attention mechanism and physics-informed neural network training. On a dataset containing 18,061 EV charging piles, we demonstrate the state-of-the-art performance of the proposed model. The results empirically showed that the proposed approach achieved an average improvement of 14.78% in accuracy, compared to other representative models. Moreover, the model can uncover the price elasticity of demand during training, making the model interpretable. Finally, price adjustment experiments are conducted to demonstrate the self-influence and spillover effects of price.

Estimating price elasticity of demand for electric vehicle charging contributes to the accurate determination of charging price, thereby improving electric vehicle adoption and energy sustainability. However, few studies have studied the impact of electricity price on electric vehicle charging behavior, especially the demand spillover effect caused by price fluctuations. To fill the gaps, on a citywide dataset of public charging piles in Shenzhen, China, first, correlation coefficients and hypothesis tests are used to determine the relationship between charging demand and price. A learning model incorporating two-layer graph attention, temporal pattern attention, and knowledge-embedded meta-learning is developed for accurate spatio-temporal regression. Impulse response analysis is conducted to unravel several noteworthy phenomena: (1) public charging demand is inelastic to electricity price, with an average elasticity of -0.76, . 76 , and distinction between different functional areas and times is revealed; (2) negative price impulses marginally change the elasticity, while positive ones make electric vehicle charging users more price sensitive, and (3) the spillover effects caused by price increases and decreases bring 89.48% and 53.88% of its local demand changes to neighbors, respectively, with a scope of 3.45 kilometer. These findings provide policy implications for promoting electric vehicle charging to facilitate renewable energy transition.

Recent booming successes of electric vehicles (EVs) motivate emerging exploration of spatio-temporal (ST) EV charging demand forecasting to inform policy making. Recent studies have contributed to remarkable accuracy improvement by developing deep learning methods. However, when they access massive amounts of data and frequently exchange data through the Internet of Things (IoT), data silos and inefficient training emerge as main challenges. To tackle these challenges, this study proposes an integrated approach for regional EV charging demand forecasting, named federated meta learning-based graph convolutional network, which consists of two modules, namely, 1) ST learning module, which introduces spatial and temporal attentions to capture the underlying charging patterns between different regions and cities effectively and 2) distributed pretraining module, which incorporates federated learning and meta-learning to enhance the adaptivity and generalisability of the forecasting model. A comprehensive evaluation based on a real-world data set of 25246 public EV charging piles shows that the proposed model outperforms other representative models with 1) an average improvement of 29.9% in forecasting errors; 2) an acceleration of 65% in convergence speed; and 3) a sound adaptability to support varying charging demand.

Abstract Parking occupancy prediction is an important reference for travel decisions and parking management. However, due to various related factors, such as commuting or traffic accidents, parking occupancy has complex change features that are difficult to model accurately, thus making it difficult for parking occupancy to be accurately predicted. Moreover, how to give appropriate weights to these changing features in prediction becomes a new challenge in the era of machine learning. To tackle these challenges, a parking occupancy prediction method called time series decomposition–long and short‐term memory neural network (LSTM)–temporal pattern attention mechanism, which consists of three modules, namely 1) time series decomposition: modelling parking occupancy changes by extracting features such as trend, period, and effect; 2) encoder: extracting temporal correlations of feature sequences with LSTM; 3) temporal pattern attention mechanism: assigning attention to different features, are proposed. The evaluation results of 30 parking lots in Guangzhou city show that the proposed model 1) improves accuracy over the baseline model LSTM by 9.14% on average; 2) performs outstanding in four prediction time intervals and six types of parking lots, proving its validity and generality; 3) demonstrates its rationality and interpretability through ablation experiments and Shapley additive explanation.