Accurate customer LifeTime Value (LTV) predictions are of critical importance for evaluating the efficiency of customer management strategies, which could enhance advertising placement for better decision-making in ad systems. However, existing solutions for LTV prediction usually rely on determined historical sequence data, which are challenging to apply in Baidu ads due to two unique features including (i) uncertain behavior sequence of customers caused by dynamic advertising strategies, (ii) complex and long-tail distribution of LTV caused by continuous customer behaviors and unique business pattern of search and news feed ads in Baidu. To incorporate these new factors, we propose an Uncertain behavior Sequence modeling framework to predict customer LifeTime Value (USE-LTV), where we (i) utilize a transformer module to extract uncertain sequence features, and develop a dynamic weight mechanism to capture differentiated information under uncertain behavior sequence, (ii) design an continuous loss function tailored to the real-world long-tail exponential LTV distribution in Baidu ads. We extensively evaluate our method based on the industrial-scale real-world data from Baidu, one of the world's largest ads platform, demonstrate that USE-LTV achieves 11.64% of NMAE improvement for a-year LTV compared to the state-of-the-art method.
The paper introduces White Rabbit, a novel method for discovering high-quality, meaningful paths between entities in online Knowledge Graphs (KGs). Traditional exploration methods, such as SPARQL endpoints, struggle due to the large size and complexity of KGs. The proposed approach addresses this by introducing the problem of context-aware path finding, ensuring that retrieved paths are coherent and involve highly relevant entities. White Rabbit uses embeddings to score entity neighbors, a queue-based prioritization mechanism, and an iterative refinement process to improve efficiency and relevance. The system is demonstrated live, allowing participants to test it and compare it against baseline methods (structural approaches, pretrained embeddings, and large language models). Results show that White Rabbit enhances both the efficiency of exploration and the quality of discovered paths.
Open-vocabulary object detection (OVD) in remote sensing (RS) has shown remarkable generalization capabilities across diverse RS imagery through alignment between image and text embeddings. Such methods have further improved detection performance by incorporating additional scene-level context from both visual and textual domains. However, existing methods approximate scene context by simply averaging the text embeddings of the image's object labels, which is insufficient to capture the rich linguistic context present in RS scenes. To address this limitation, we propose a novel Feature-Aligned Scene Encoding (FASE), which constructs comprehensive scene representations through high-quality captions generated by a specialized vision-language model. Our Feature Alignment Module (FAM) creates a robust scene representation by fusing domain-specific caption embeddings with general text features through dual-branch fusion with gating and cross-attention. This resulting representation then facilitates the alignment with visual features. By utilizing enhanced scene encoding only during training, our method internalizes rich contextual knowledge without increasing inference complexity. Experiments on multiple benchmarks demonstrate significant improvements over state-of-the-art methods, validating the effectiveness of our approach for OVD in RS.
Network resilience refers to the ability of a network to maintain its functionality despite perturbations, where resilience/robustness is shown when a substantial proportion of its nodes remain engaged even under changes. Such phenomenon is common in real-world networks, such as computing power networks. Previous works demonstrate that the coreness of a user/node effectively captures the dynamics of user engagement. However, most existing works only consider changes in a single coreness value and thus fail to measure the overall network resilience. Subsequent works are either inefficient or do not consider coreness-decreased scenario. In this paper, we propose and study the collapsed follower maximization problem, aiming to maximize the number of coreness-decreased vertices by finding a group collapser (collapsing a set of vertices) with a given budget. We prove that the problem is NP-hard and W[2]-hard parameterized by the budget b. To address the problem, we first present a Greedy algorithm that iteratively finds the best collapser in each of the budget b iterations. To further optimize the Greedy algorithm, we propose GreedyOpt, which leverages the shell component structure to accelerate the computation of follower for one collapser and prune the search space. Extensive experimental results on 8 real-world datasets show that the effectiveness and efficiency of our algorithms.
Current large vision-language models (VLMs) exhibit remarkable performance in basic video understanding tasks. However, existing VLMs are still limited to surface-level perception and lack fine-grained spatio-temporal understanding and combinatorial reasoning capabilities. Existing methods typically rely on expensive human annotations or subtitle extraction, yet they struggle to effectively model temporal relations between frames. This paper proposes a structured representation based on temporal triplets to address two major challenges in traditional approaches: temporal fragmentation and entity reference ambiguity. By modeling objects, attributes, and relationships within the video and incorporating temporal information, we convert semantic content from keyframes into a sequence of temporal triplets. This structured representation is then used as input for zero-shot video question answering (VideoQA). Experiments were conducted on four benchmark VideoQA datasets: NExT-QA, STAR, MSVD-QA, and MSRVTT-QA, showing that our method achieves competitive performance without requiring fine-tuning, validating its generality and effectiveness.
Instruction tuning has become a cornerstone for unlocking the full potential of large language models. Among the key factors, complex and diverse instructions play a crucial role in aligning these models with a wide range of downstream tasks. However, current methodologies for constructing large-scale instruction datasets tend to favor powerful models, such as GPT-4, based on the empirical assumption that larger models inherently possess superior capabilities. In this study, we challenge this prevailing assumption and delve into the untapped potential of smaller language models (SLMs) in the context of instruction evolution. Through extensive experiments across three distinct scenarios of instruction evolution, we find that SLMs can generate more effective instructions compared to their larger counterparts. Further analysis reveals that SLMs exhibit a broader output space during instruction evolution, leading to the creation of more complex and diverse instructional variants. Additionally, we observe that existing evaluation metrics fall short in capturing the nuanced impact of instructions. To address this limitation, we propose Instruction Complex-Aware IFD (IC-IFD), an enhanced framework that incorporates instruction complexity into the original IFD score. This approach enables a more accurate assessment of the effectiveness of instruction data, paving the way for more refined instruction tuning strategies.
Recommending optimal medication combinations for patients is a crucial application of artificial intelligence in healthcare. Recent works typically use patients' electronic health record combined with their current health conditions. However, these efforts have the following issues: 1) they often reference historical visits unrelated to the current situation, and 2) there is a latent risk of side effects from historical prescriptions. Such issues raise concerns about the safety of medication recommendation. To address this, we propose GPSRec, a novel Graph augmented Patient Similarity network for medication Recommendation. By leveraging dual similarity measures to selectively integrate historical visits, GPSRec effectively filters out irrelevant information, improving the accuracy of recommendation. We further present a training strategy, which combines a pre-training method and a dual threshold loss adjustment, reduces the risk of adverse drug-drug interactions, enhancing the safety of recommendation. Extensive experiment results on two real datasets demonstrate that GPSRec significantly outperforms state-of-the-art methods. Notably, it achieves 30.11% and 24.92% improvements in safety, respectively, with higher accuracy.
While Large language models (LLMs) have shown to exhibit remarkable performance in a wide range of NLP tasks, they often struggle to interpret and reason over multi-hop questions in open-domain question answering (ODQA) settings. While popular prompt approaches such as Chain-of-Thought and Plan-and-Solve facilitate more manageable questions for OQDA via task decomposition, these approaches are prone to generating erroneous and redundant intermediate steps in multi-hop queries due to limited capacity for modeling complex entity relationships. In this paper, we introduce a novel prompt approach for multi-hop QA viz., MoDeGraph (Multi-Hop Dependency Graphs), that is designed to steer LLMs to extract and model entity relationships in complex questions. MoDeGraph constructs a dependency graph from LLM-generated entity-relation triples to enable more coherent and human-like multi-step reasoning. Experimental results in knowledge-intensive tasks for multi-hop QA demonstrate our approach produces more coherent and faithful reasoning chains as well as consistent increase in QA performance across several benchmark datasets.
Temporal link prediction, which aims to predict the future status of edges between target nodes, is vital for current prevalent online services. Most existing methods ignore node-level behavior patterns, which play a decisive role in temporal link prediction, as nodes that behave similarly are more likely to interact in the future. In this paper, we propose a novel continuous-time model, the Interaction Pattern-aware neural Network (IPNet), to capture node-level behavior patterns and network evolution by encoding interaction sequences and contextual windows. We further devise a random walk sampling strategy to enhance the extraction of these windows, preserving node-centric structural evolution. Experimental results on seven real-world networks demonstrate that IPNet outperforms state-of-the-art methods in both transductive and inductive link prediction tasks. The code can be accessed via https://github.com/CoderZQY/IPNet.
Detecting abnormal climate events across temporal and spatial scales is crucial to the understanding of local and regional climate trends. This demonstration introduces ClimBurst, a dynamic tool to detect climate bursts, which are unusually high or low values of one or more climate variables over some time interval. ClimBurst detects bursts without prior assumptions about their temporal duration. The demonstration will allow users to interact directly with our system to see both a summary showing the presence/absence of bursts over a user-specified year and spatial range. The demonstration will also allow users to perform time-travel queries to see how bursts propagate over space and time.
How can we recommend bundles that reflect users' changing preferences over time? Sequential bundle recommendation aims to recommend bundles of items while capturing users' evolving preferences over time. Unlike traditional bundle or sequential recommendation, this task requires modeling both the structural composition of bundles and the temporal dynamics of user behavior. We identify three major challenges: (1) dynamic user preferences across bundle interactions, (2) user-dependent attention to different items in the same bundle, and (3) users' diverse preferences on bundling strategies. To address these, we propose CoReSBR (Contextualized Representation for Sequential Bundle Recommendation), an adaptive framework that constructs bundle representations contextualized by time-aware user preferences. CoReSBR encodes recent user interactions to reflect preference shifts, assigns attention-based weights on items in bundles using user embedding as query, and integrates multiple bundling strategies through user-specific combination. Extensive experiments on real-world datasets demonstrate that CoReSBR outperforms the state-of-the-art methods, achieving up to 8.91% higher nDCG and 8.05% higher recall.
Audio-visual deepfake detection methods demonstrate strong performance on academic datasets but fail significantly when applied to real-world. To address the shortcomings of previous approaches, we utilize landmarks dynamic information. First, we propose Landmark-based Distillation (LBD), motivated by I-JEPA's representation learning approach. LBD utilizes KL-divergence to align facial landmark predictions from visual and audio encoders, enforcing focus on geometric facial features rather than spurious background information. Second, we introduce Multimodal Temporal Information Alignment (MTIA), which employs contrastive learning to enhance temporal consistency between audio and visual representations. We conduct experiments on academic datasets and web-based deepfakes collected from diverse social media platforms, serving as real-world examples. Our proposed landmark-guided distillation framework achieves computational efficiency while improving multimodal video deepfake detection performance across a diverse range of deepfakes compared to existing methods. The code is available at https://github.com/Ckck12/Beyond-Masking.
Retrieval-Augmented Image Captioning enhances the model's understanding of real-world images by retrieving external knowledge. Existing methods mainly use original captions or isolated entities related to the query image to help generate captions. However, these methods make the model either imitate the caption style or fail to capture the relationship between entities, resulting in a lack of diversity or inaccuracy in the generated captions. To address these issues, we propose SEAR, a novel framework that utilizes external Synthesized Entity-Aware knowledge Representations to improve captioning performance. Specifically, SEAR clusters images based on scene-level and entity-level features, and synthesizes each clustered images into representative images as retrieval indexes, and simultaneously utilizes a large model to extract and supplement structured knowledge graphs from the corresponding cluster captions. Furthermore, we design a knowledge-graph pruner to prune the knowledge graph by retaining the most relevant subgraphs to the query image. By undertaking these steps in an integrated manner, SEAR enables the model to acquire non-redundant and structured information for generating captions and avoid data-related privacy issues. Extensive experiments on MSCOCO, Flickr30k, and NoCaps demonstrate the effectiveness of our method both in-domain and out-of-domain, outperforming existing lightweight RAIC methods and remaining competitive with heavyweight models.
Developing scalable wearable human activity recognition (wHAR) models is challenging due to domain shifts that substantially degrade performance across downstream tasks. Unsupervised domain adaptation (UDA) seeks to improve generalization by transferring knowledge from labeled source domains to unlabeled target domains. However, conventional UDA methods primarily align marginal feature distributions while neglecting feature-label dependencies, often leading to negative transfer and sub-optimal performance. Motivated by these limitations, we propose a novel optimization framework that tackles two key challenges: (i) generating reliable pseudo-labels for the unlabeled target domain and (ii) minimizing conditional discrepancies across domains. To address (i), we employ temperature-based entropy minimization (TEM), which calibrates prediction confidence by scaling logits with a temperature parameter to produce robust pseudo-labels. For (ii), we introduce a polynomial kernel-based cross-covariance (PkCC) loss, a high-order statistics-driven approach that maps features into a reproducing kernel hilbert space (RKHS) to capture richer feature-label dependencies and reduce conditional distribution gaps between domains. In addition, we demonstrate that CoDAN readily extends to partial UDA (pUDA), where the target label space is a subset of the source, and extensive evaluations on public wHAR datasets with diverse label spaces validate its superior performance over state-of-the-art methods in both UDA and pUDA scenarios.
As generative artificial intelligence (AI) becomes increasingly prevalent in creative industries, intellectual property issues have come to the forefront, especially regarding AI-generated content that closely resembles human-created works. Recent high-profile incidents involving AI-generated outputs reproducing copyrighted materials underscore the urgent need to reassess current copyright frameworks and establish effective safeguards against infringement. To this end, this tutorial provides a structured overview of copyright challenges in generative AI across the entire development lifecycle. It begins by outlining key copyright principles relevant to generative models, then explores methods for detecting and evaluating potential infringement in generated outputs. The session also introduces strategies to safeguard creative content and training data from unauthorized replication, including mitigation techniques during model training. Finally, it reviews existing regulatory frameworks, highlights unresolved research questions, and offers recommendations to guide future work in this evolving area.
Scaling-law has guided the language model design for past years, e.g., GPTs, enabling the estimation of expected model performance with respect to the size of learnable parameters and the scale of training samples. It is worth noting that the scaling laws of NLP cannot be directly applied to recommendation systems due to the following reasons: (1) The amount of training samples and model parameters is typically not the bottleneck for the model. Our recommendation system can generate over 50 billion user samples daily, and such a massive amount of training data can easily allow our model parameters to exceed 200 billion, surpassing many LLMs (about 100B). (2) It is essential to control FLOPs carefully in recommendation system. In training, we need to process a vast number of recommendation samples every day. During online inference, we must respond within milliseconds (LLMs usually take a few seconds). Considering the above differences with LLM, we can conclude that: for a RecSys model, compared to model parameters, the FLOPs is a more expensive factor that requires careful control. In this paper, we propose our milestone work, MARM (Memory Augmented Recommendation Model), which explores a new cache scaling-law successfully. By caching part of complex module calculation results, our MARM extends the single-layer attention-based sequences interests modeling module to a multi-layer setting with minor inference FLOPs cost (i.e, module time complexity O(n(2) * d) -> O(n * d)). Equipped with the cache idea, our MARM solution significantly overcomes computational bottlenecks and can seamlessly empower all interest extraction modules for user sequences, and even other models. To support our MARM, we construct a 60TB cache storage center for offline training and online serving. Comprehensive experiment results show that our MARM brings offline 0.43% GAUC improvements and online 2.079% playtime per user gains. Our MARM has been deployed on a real-world short-video platform, serving tens of millions of users daily.
Similar Text Retrieval (STR) is an essential scenario in the field of information retrieval (IR). Unfortunately, existing mainstream vector-based retrieval methods cannot meet the recall rate requirements in STR scenarios (with a recall rate of less than 72%). This is because existing works have solely focused on the local information of text segments, that is, the text segments themselves (i.e., semantic information) and the relationships between them (i.e., structured information). Our key insight is that utilizing the global information of text segments (i.e., summarized information 1) is crucial for improving the recall rate in STR, because the distinction of summarized information helps to filter out confusing vectors during retrieval. However, existing methods using summarized info still have a critical challenge. Their vectorization-based approaches fail to effectively model the global relationship in the summarized information, resulting in a further 79% deterioration in recall rate. To address this challenges, we present the SG-Filter, a novel retrieval framework that integrates summarized information by designing the hierarchical summarized-semantic index and the adaptive filtering strategy applied on it. (1) We propose a hierarchical summarized-semantic index by designing a summarized graph to model the summarized information. Specifically, we exploit the global information at both document and text segment levels through co-occurrence relationships and semantic associations. (2) We propose an adaptive filtering strategy that automatically determines which summarized words to filter per retrieval for effectively utilization of summarized information. (3) To ensure robustness and low retrieval latency, we propose a multi-path merge recall strategy to obtain summarized and semantic information at varying proportions, and develop an efficient vector retrieval method with filtering conditions. Experiments show that SG-Filter significantly increases recall rate by 10.53% similar to 22.92% on average compared with existing vector-based retrieval methods in STR. SG-Filter also ensures retrieval latency remains within tens of milliseconds. The code is open-sourced in https://github.com/strong-leaf/SG-Filter
Language Models (LMs) achieve outstanding performance across diverse applications but often produce biased outcomes, raising concerns about their trustworthy deployment. These concerns call for fairness research specific to LMs; however, most existing work in machine learning assumes access to model internals or training data, conditions that rarely hold in practice. As LMs continue to exert growing societal influence, it becomes increasingly important to understand and address fairness challenges unique to these models. To this end, our tutorial begins by showcasing real-world examples of bias to highlight their practical implications and uncover underlying sources. We then define fairness concepts tailored to LMs, review methods for bias evaluation and mitigation, and present a multi-dimensional taxonomy of benchmark datasets for fairness assessment. We conclude by outlining open research challenges, aiming to provide the community with both conceptual clarity and practical tools for fostering fairness in LMs. All tutorial resources are publicly accessible at https://github.com/vanbanTruong/fairness-in-large-language-models.
The emergence of large generative models is transforming the landscape of recommender systems. One of the most fundamental components in building these models is action tokenization, the process of converting human-readable data (e.g., user-item interactions) into machine-readable formats (e.g., discrete token sequences). In this tutorial, we present a comprehensive overview of existing action tokenization techniques, converting actions to (1) item IDs, (2) textual descriptions, and (3) semantic IDs. We then make an in-depth discussion on the challenges and open questions of building large generative recommendation models from the perspective of action tokenization. Materials of this tutorial are available at: https://large-genrec.github.io/.
