Large language models (LLMs) are increasingly deployed as conversational agents, owing to their ability to sustain coherent interactions. However, the stochastic nature of LLMs and well-documented risks such as hallucinations and unsafe outputs pose significant challenges for ensuring safe deployment. We introduce a novel monitoring system that we designed and implemented to tackle these challenges in deployments of LLM conversational agents. Our system combines expertise from psychiatry and computer science to support risk detection and multi-stakeholder decision-making. The system consists of two main modules: continuous regression testing and real-time monitoring. Regression testing leverages multi-turn synthetic conversations to evaluate the safety of LLMs in deployment and runs automatically in a continuous integration pipeline. Real-time monitoring incorporates both manual and automated evaluation of user-model interactions, including efficient evaluation from mental health professionals via a web-based interface. To demonstrate how our system works in practice, we conducted a preregistered online behavioral experiment ‘AI Safety Challenge’ as a case study, which demonstrates the feasibility of our system in live operation and suggests a modest efficiency gain under a fixed expert review budget. Our monitoring system functions as a platform where different types of expertise can be combined. Clinical expertise informs dataset design, evaluation procedures, and real-time monitoring, complementing computational safety metrics to deliver a robust monitoring infrastructure. Best practices and state-of-the-art methods from computer science are employed to build a robust and scalable architecture. Executable code 1 and datasets 2 are provided to support reuse, extension, and reproducibility in future applications. Warning: This paper includes model-generated outputs that may be offensive or distressing, including references to self-harm.
Generating objectives and reasoning about them is key for intelligent planning agents to become more autonomous. Once a set of achievable goals has been chosen, a symbolic planner can find an executable plan to fulfil them. However, finding these goals is a complex problem when the aim is to optimise conflicting metrics such as goal benefit, plan cost, and its makespan. In this paper, we first propose a method to identify a set of potential goals in a temporal PDDL domain. Next, we find the achievable subset of goals that yields the most benefit by posing the problem as an integer linear program. Finally, a planner is used to solve the resulting subset of goals. The proposed method is generic and planner-independent, generating sets of goals that are neither provably unreachable nor mutually exclusive, thereby increasing the likelihood of finding a solvable plan. We demonstrate the effectiveness of the proposed approach in comparison with a baseline oversubscription planner.
Multi-behavior recommendation aims to predict the items a user is likely to purchase by analyzing user-item interactions across diverse behaviors. Existing approaches are generally categorized into representation learning (RL) and graph ranking (GR). RL methods generate user and item embeddings in a global manner using graph neural networks and multi-behavioral signals. However, their performance heavily relies on the expressiveness of the learned embeddings, which is often limited by over-smoothing and bias toward frequent interactions. In contrast, GR methods compute user-specific ranking scores over the user-item graph in a query-dependent manner, enabling more direct and fine-grained personalization. Despite this potential, GR has been predominantly studied in single-behavior settings and remains underexplored for multi-behavior recommendation. In this paper, we propose CascadingRank , a novel personalized graph ranking method for multi-behavior recommendation. To achieve better personalization, we first construct a cascading behavior graph that captures the cascading sequence of user behaviors (e.g., viewing, adding to cart, and purchasing). We then compute personalized ranking scores in a staged manner along this graph, where each stage is conditioned on scores of preceding behaviors and query preferences. The model captures behavior-specific influence through cascading stages in graph ranking, yielding more accurate recommendations. To the best of our knowledge, this is the first work to systematically investigate graph ranking as a principled solution for multi-behavior recommendation. Extensive experiments on three real-world datasets show that CascadingRank outperforms state-of-the-art methods, achieving up to 9.56% and 7.16% improvements in HR@10 and NDCG@10, respectively.
Early rumor detection predicts verification labels from sparse and incomplete propagation cascades. We introduce RumorDiffusion, a prefix-conditioned framework that combines class-conditional topology diffusion with a Transformer arbiter. The generator learns topology-and-feature distributions from mature training cascades. At inference, it uses the observed prefix to generate a potential vector for each class hypothesis, and the arbiter converts the resulting vectors into verification probabilities. A two-stage training scheme pretrains the generator, then freezes it while the arbiter learns discriminative aggregation. Repeated stochastic prediction supplies averaged probabilities and a disagreement statistic. Validation-based temperature scaling and confidence thresholds support selective early decisions from these predictions. Experiments on Twitter15, Twitter16, Weibo, and PHEME evaluate classification, component contributions, calibration, sampling cost, and source-only transfer across early observation windows. RumorDiffusion achieves the highest accuracy and Macro-F1 in three of four main comparisons and nine of twelve early dataset-window comparisons. Its main-comparison Macro-F1 reaches 0.838 on Twitter15, 0.854 on Twitter16, and 0.908 on Weibo. Component ablations quantify the contribution of class-conditioned representations. Temperature scaling reduces expected calibration error (ECE) while preserving Macro-F1. RumorDiffusion also achieves the highest transfer Macro-F1 and lowest ECE in both evaluated domain pairs. These results establish prefix-conditioned generation as an effective basis for calibrated early verification. Our code is available at https://github.com/cgao-comp/RumorDiffusion .
Continuous-Time Event Sequences (CTESs) are discrete events occurring at irregular times along a continuous timeline and are common in domains such as healthcare and finance. Marked Temporal Point Processes (MTPPs) model CTESs by capturing both event timing and type. While widely used, adversarial evaluation and attacks on MTPPs remain largely unexplored. In this work, we propose adversarial attacks specifically designed for MTPP models. A key requirement for effective adversarial attacks is imperceptibility, i.e., the perturbations should remain subtle or undetectable. While in domains such as images or text this is typically enforced by constraining perturbations within a fixed Lp norm ball, applying similar constraints to CTESs is non-trivial due to their sequential nature and variability in time scales and sequence lengths. To address this challenge, we first permute the events and then introduce additive noise to the arrival timestamps. However, finding the worst-case adversarial example under this setting becomes a hard combinatorial problem, involving a search over a factorially large permutation space. To overcome this, we introduce a novel differentiable framework, PERMTPP, that enables adversarial attacks by learning to minimize the model's likelihood while simultaneously constraining the distance between the original and perturbed CTESs. Experiments on four real-world datasets demonstrate that PERMTPP achieves strong offensive and defensive performance, along with significantly lower inference time.
