Large Language Models (LLMs) are increasingly deployed in settings where Chain-of-Thought (CoT) is interpreted by users. This creates a new safety risk: attackers may manipulate the model's observable CoT to make malicious behaviors. In open-weight ecosystems, such manipulation can be embedded in lightweight adapters that are easy to distribute and attach to base models. In practice, persistent CoT hijacking faces three main challenges: the difficulty of directly hijacking CoT tokens within one continuous long CoT-output sequence while maintaining stable downstream outputs, the scarcity of malicious CoT data, and the instability of naive backdoor injection methods. To address the data scarcity issue, we propose Multiple Reverse Tree Search (MRTS), a reverse synthesis procedure that constructs output-aligned CoTs from prompt-output pairs without directly eliciting malicious CoTs from aligned models. Building on MRTS, we introduce Two-stage Backdoor Hijacking (TSBH), which first induces a trigger-conditioned mismatch between intermediate CoT and malicious outputs, and then fine-tunes the model on MRTS-generated CoTs that have lower embedding distance to the malicious outputs, thereby ensuring stronger semantic similarity. Experiments across multiple open-weight models demonstrate that our method successfully induces trigger-activated CoT hijacking while maintaining a quantifiable distinction between hijacked and baseline states under our evaluation framework. We further explore a reasoning-based mitigation approach and release a safety-reasoning dataset to support future research on safety-aware and reliable reasoning. Our code is available at https://github.com/ChangWenhan/TSBH_official.

With the growing computational cost of large models, leveraging pre-trained models and applying fine-tuning have become standard practices in machine learning. While these approaches improve accessibility and efficiency, they also introduce new security and ethical risks. Models can be misused for prohibited or high-risk applications, even when publishers explicitly forbid such use. This paper proposes Backdoor-assisted Inversion for Abuse Verification (BIAV), a framework that enables model publishers to verify unauthorized or unethical model use. In BIAV, the publisher embeds one backdoor corresponding to predefined high-risk usage patterns and declares that the model must not be employed in these contexts. If a user violates this policy and fine-tunes the model on sensitive images aligned with the sensitive pattern, the publisher can reveal the sensitive images for verification. Unlike traditional backdoor attacks designed for malicious behavior, BIAV leverages the backdoor to trace misuse. In fine-tuning scenarios, such hidden triggers are nearly impossible for users to detect, as models behave normally on clean images. BIAV associates each backdoor with a sensitive concept (e.g., “fighter aircraft” in a model meant for civilian aviation). When fine-tuned on images containing such sensitive concept, these inputs disproportionately affect the parameter updates. By analyzing the weight changes before and after fine-tuning, the publisher can estimate the corresponding gradients and reconstruct the sensitive images via gradient inversion. Notably, BIAV remains effective under multi-epoch training and multiple-image settings where conventional inversion methods fail. Experiments show that BIAV successfully recovers sensitive, backdoor-tagged images from local fine-tuning data. These results demonstrate that BIAV provides a practical mechanism for verifying model misuse after deployment.

Machine unlearning enables pre-trained models to eliminate the effects of partial training samples. Previous research has mainly focused on proposing efficient unlearning strategies. However, the verification of machine unlearning, or in other words, how to guarantee that a sample has been successfully unlearned, has been overlooked for a long time. Existing verification schemes typically rely on machine learning attack techniques, such as backdoor attacks or membership inference attacks. As these techniques are not formally designed for verification, they are easily bypassed when an untrustworthy model provider in MLaaS undergoes rapid fine-tuning to meet the verification conditions only, rather than executing real unlearning. In this paper, we propose a formal verification scheme, IndirectVerify, to determine whether unlearning requests have been successfully executed. We design influential sample pairs: one referred to as trigger samples and the other as reaction samples. Users send unlearning requests regarding trigger samples and use reaction samples to verify if the unlearning operation has been successfully carried out. We propose a perturbation-based scheme to generate those influential sample pairs. The objective is to perturb only a small fraction of training samples to trigger samples, leading to the misclassification of reaction samples. This indirect influence will be used for our verification purposes. In contrast to existing schemes that employ the same samples for all processes, our scheme, IndirectVerify, provides enhanced robustness, making it less susceptible to bypassing processes.

Machine unlearning, a process enabling pre-trained models to remove the influence of specific training samples, has attracted significant attention in recent years. Although extensive research has focused on developing efficient machine unlearning strategies, we argue that these methods mainly aim at removing samples rather than removing samples' influence on the model, thus overlooking the fundamental definition of machine unlearning. In this paper, we first conduct a comprehensive study to evaluate the effectiveness of existing unlearning schemes when the training dataset includes many samples similar to those targeted for unlearning. Specifically, we evaluate: Do existing unlearning methods truly adhere to the original definition of machine unlearning and effectively eliminate all influence of target samples when similar samples are present in the training dataset? Our extensive experiments, conducted on four carefully constructed datasets with thorough analysis, reveal a notable gap between the expected and actual performance of most existing unlearning methods for image and language models, even for the retraining-from-scratch baseline. Additionally, we also explore potential solutions to enhance current unlearning approaches.

Large language model (LLM)-based agents have recently gained considerable attention due to the powerful reasoning capabilities of LLMs. Existing research predominantly focuses on enhancing the task performance of these agents in diverse scenarios. However, as LLM-based agents become increasingly integrated into real-world applications, significant concerns emerge regarding their accumulation of sensitive or outdated knowledge. Addressing these concerns requires the development of mechanisms that allow agents to selectively forget previously learned knowledge, giving rise to a new term LLM-based agent unlearning. This paper initiates research on unlearning in LLM-based agents. Specifically, we propose a novel and comprehensive framework that categorizes unlearning scenarios into three contexts: state unlearning (forgetting specific states or items), trajectory unlearning (forgetting sequences of actions) and environment unlearning (forgetting entire environments or categories of tasks). Within this framework, we introduce a natural language-based unlearning method that trains a conversion model to transform high-level unlearning requests into actionable unlearning prompts, guiding agents through a controlled forgetting process. Moreover, to evaluate the robustness of the proposed framework, we introduce an unlearning inference adversary capable of crafting prompts, querying agents, and observing their behaviors in an attempt to infer the forgotten knowledge. Experimental results show that our approach effectively enables agents to forget targeted knowledge while preserving performance on untargeted tasks, and prevents the adversary from inferring the forgotten knowledge.

With the rapid development of artificial intelligence, large language models (LLMs) have made remarkable advancements in natural language processing. These models are trained on vast datasets to exhibit powerful language understanding and generation capabilities across various applications, including chatbots, and agents. However, LLMs have revealed a variety of privacy and security issues throughout their life cycle, drawing significant academic and industrial attention. Moreover, the risks faced by LLMs differ significantly from those encountered by traditional language models. Given that current surveys lack a clear taxonomy of unique threat models across diverse scenarios, we emphasize the unique privacy and security threats associated with four specific scenarios: pre-training, fine-tuning, deployment, and LLM-based agents. Addressing the characteristics of each risk, this survey outlines and analyzes potential countermeasures. Research on attack and defense situations can offer feasible research directions, enabling more areas to benefit from LLMs.

Due to the growing emphasis on privacy and data governance in machine learning, federated unlearning, an emerging concept in the domain of federated learning, stems from the growing need to address the dynamic nature of data and the evolving requirements related to privacy, compliance, and data management. However, there are some security risks during the unlearning process, including the potential for adversarial manipulation of model integrity, privacy breaches, and performance degradation in a federated learning framework. Although existing research has proposed various defenses to mitigate these risks, significant vulnerabilities remain that can be exploited to undermine the integrity and effectiveness of the unlearning process. Current attack methods are limited by their detectability during training, lack of persistence, and reliance on test-time triggers, which reduces their overall effectiveness. In this paper, we introduce camouflaged poisoning attacks, a novel attack paradigm relevant to federated unlearning. In this approach, some adversary clients initially infuse a small number of meticulously designed points into the dataset, ensuring that the model's predictions are barely influenced. The adversary then makes a request to the exclusion of some of these malicious clients. At this juncture, the attack is activated, leading to a detrimental impact on the model's predictions. The outcomes reveal a substantial potential for these strategies to compromise the effectiveness of models in unlearning scenarios. The essence of this attack involves the creation of deceptive clients that conceal the influence of a contaminated dataset during the federated unlearning process.

Online service systems operate under high concurrency, dynamic workloads, and distributed execution environments, which increase their susceptibility to faults and performance degradation. To ensure stable operation, these systems commonly monitor performance metrics, where timely anomaly detection enables early intervention and fault prevention. Existing metrics-based anomaly detection methods are typically unsupervised, reconstructing unlabeled data to model normal patterns and detecting anomalies as deviations from these patterns. However, they often overlook the presence of unlabeled anomalous data (noise) within the training set, which hinders the model's ability to accurately learn normal behavior and degrades detection accuracy. To address this limitation, we propose a noise-robust, representation-enhanced reconstruction-based anomaly detection method for metrics. The method trains a reconstruction model to capture normal time-series patterns and identifies noisy samples based on their deviation from the latent representation center, assigning them lower training weights. Both the reconstruction outputs and the representation centers jointly define the normal pattern of each sample, reducing the influence of anomalous noise during modeling. Extensive experiments show that our method achieves an F1 score of 97.00% on four widely used datasets, surpassing existing state-of-the-art approaches. Ablation studies further confirm the effectiveness of each module.

The diffusion model has gained considerable attention as an advanced deep generation model, showcasing remarkable performance. However, concerns have emerged regarding privacy and equity due to potential model misuse, especially for some sensitive features such as gender. Our focus is on addressing the challenge of machine unlearning within an unconditional diffusion model that selectively omits specific features. Our objective is to modify images generated by a pre-trained model by selectively removing particular image components. To achieve this, we propose a novel learning framework for unconditional image-to-image diffusion models. This framework integrates a scoring model into a pre-trained diffusion model, allowing for refinement without the need for a complete retraining process. Experimental validation using datasets such as MNIST and CelebA demonstrates the effectiveness of our approach. It highlights the model’s capability to generate and remove target features while maintaining the fidelity of the original image.

We report a structural mismatch between a data point’s {learnability}—how quickly it improves the loss—and its {forgettability}—how much it anchors the final parameters—an aspect ignored by prior machine unlearning frameworks such as SISA, Fisher-Forget, and influence-based fine-tuning. To make this gap measurable we introduce Unlearning Gradient Sensitivity (UGS), an influence score computable with a single Hutch++ sketch, and derive the Learnability–Forgettability Divergence (LFD), the Jensen–Shannon distance between the model’s learning and forgetting distributions. We prove that UGS dispersion decays exponentially only under explicit regularisation and that LFD converges to zero when its weight grows sub-linearly relative to the UGS term. Building on these findings, we introduce Dual-Aware Training (DAT)—a lightweight regularization method that reduces variability in how easily data points can be forgotten and aligns learning and forgetting behaviors during training. On CIFAR-10, MNIST, and IMDB, DAT maintains the original model accuracy while cutting forgettability divergence in half and significantly lowering the cost of certified unlearning, showing that it’s effective to make models forgettable from the start.

Reliable refusal behavior requires Large Language Models (LLMs) to reject harmful prompts with only answering benign ones. Incorrect refusal behavior can either expose users to harmful responses or prevent users from obtaining useful answers. Training-time alignment improves refusal behavior by updating model parameters with safety data, but requires additional computation and training. In contrast, inference-time alignment aims to modify LLM behavior during inference without updating the underlying model parameters. Existing inference-time methods mainly rely on in-context safety prompting, activation steering, or decoding control. However, most of them intervene without first determining whether the initial response is already appropriate, potentially altering a correct refusal or a useful answer. Effective selective intervention therefore requires identifying prompt intent beyond sensitive keywords, covering semantic variations that fixed rules may miss, and adapting the verifier to different base models. To address these challenges, we propose Response Inspection and Selective Actions (RISA), an inference-time framework that inspects the initial response and selectively corrects refusal errors without updating the base model. RISA first uses fixed contextual rules to assign refusal scores to clear cases. For unmatched cases, it derives a refusal score from the final-layer prompt hidden state using a calibrated linear probe. To adapt to different base models, RISA separately calibrates the probe score, representation-support boundary, and action thresholds. At runtime, RISA combines the prompt score with the initial refusal status and applies an action policy to intervene only when necessary. Experimental results demonstrate that RISA improves refusal reliability while largely preserving model utility, offering a practical solution for response-aware refusal calibration in LLMs.

Multi-LLM collaborative systems have attracted significant attention as a promising solution for complex tasks, enabling multiple large language models (LLMs) with different domains to work together toward a common goal. Different collaborative structures (e.g., Centralized, Horizontal, and Joint Interaction) and communication methods (e.g., direct, summary, and vote) give the system with enhanced flexibility and reasoning capability. However, these same mechanisms also introduce potential security and privacy risks, such as the generation of incorrect responses and the leakage of sensitive information. Based on the above unique characteristics of collaborative systems, we propose three attack methods (named Decision Poisoning Attack, Indirect Echoleak Attack and Information Collision Attack) that exploit the interactions between LLMs to achieve different objectives like system manipulation and privacy leakage. Extensive experiments demonstrate the effectiveness of the proposed attack on three structures and three communication methods, highlighting the security vulnerabilities and potential risks in Multi-LLM collaborative systems. We further discuss possible defense methods that can mitigate the attack performance. Our work show that (1) the key factor for a successful attack on collaborative systems is ensuring the malicious instruction persists and propagates throughout the inter-LLMs communication. (2) both the system architecture and the communication method can affect attack effectiveness, offering valuable insights for the design of more secure Multi LLM collaborative systems in the future.

In the rapid advancement of artificial intelligence, privacy protection has become crucial, giving rise to machine unlearning. Machine unlearning is a technique that removes specific data influences from trained models without the need for extensive retraining. However, it faces several key challenges, including accurately implementing unlearning, ensuring privacy protection during the unlearning process, and achieving effective unlearning without significantly compromising model performance. This paper presents a novel approach to machine unlearning by employing Layer-wise Relevance Analysis and Neuronal Path Perturbation. We address three primary challenges: the lack of detailed unlearning principles, privacy guarantees in zero-shot unlearning scenario, and the balance between unlearning effectiveness and model utility. Our method balances machine unlearning performance and model utility by identifying and perturbing highly relevant neurons, thereby achieving effective unlearning. By using data not present in the original training set during the unlearning process, we satisfy the zero-shot unlearning scenario and ensure robust privacy protection. Experimental results demonstrate that our approach effectively removes targeted data from the target unlearning model while maintaining the model's utility, offering a practical solution for privacy-preserving machine learning.

In recent years, advancements in large language models have led to significant innovation and critical progress in AI. However, some of these innovations are raising privacy and security concerns. Machine unlearning has therefore emerged as a potential solution to mitigate such risks. Yet, while erasing data records from traditional models is relatively straightforward, making a large language model “forget” what it has learned is often very challenging. This is not just because they include so many parameters, it is also because the knowledge they possess is intricately entangled. Further, the privacy risk of unlearned data remains neglected in most unlearning solutions. To overcome these limitations, we took advantage of information retrieval and developed an efficient privacy-preserving unlearning mechanism. Our solution eliminates the impact of targeted information by removing high-risk semantic meanings from the model’s output. It also incorporates differentially-private randomization to make the unlearned information statistically indiscernible. Most importantly, the algorithm requires neither parametric fine-tuning nor in-context prompt calibration. A theoretical analysis demonstrates that this method satisfies rigorous privacy and unlearning guarantees. Additionally, experiments on real-world datasets prove that the method is both effective and has the capacity to handle practical unlearning tasks for large language model applications.

As deep image forgery powered by AI generative models, such as GANs, continues to challenge today's digital world, detecting AI-generated forgeries has become a vital security topic. Generalizability and robustness are two critical concerns of a forgery detector, determining its reliability when facing unknown GANs and noisy samples in an open world. Although many studies focus on improving these two properties, the root causes of these problems have not been fully explored, and it is unclear if there is a connection between them. Moreover, despite recent achievements in addressing these issues from image forensic or anti-forensic aspects, a universal method that can contribute to both sides simultaneously remains practically significant yet unavailable. In this paper, we provide a fundamental explanation of these problems from a frequency perspective. Our analysis reveals that the frequency bias of a DNN forgery detector is a possible cause of generalization and robustness issues. Based on this finding, we propose a two-step frequency alignment method to remove the frequency discrepancy between real and fake images, offering double-sided benefits: it can serve as a strong black-box attack against forgery detectors in the anti-forensic context or, conversely, as a universal defense to improve detector reliability in the forensic context. We also develop corresponding attack and defense implementations and demonstrate their effectiveness, as well as the effect of the frequency alignment method, in various experimental settings involving twelve detectors, eight forgery models, and five metrics.

Large Language Models (LLMs) have enabled autonomous agents capable of complex reasoning and collaboration, yet coordinating such agents in decentralized environments remains challenging due to the lack of centralized control for aligning behavior and sustaining long-term cooperation. This paper addresses this challenge from a mechanism-design perspective and proposes a behavior-shaping incentive mechanism for decentralized LLM-based multi-agent systems. The mechanism explicitly models agent utility as a function of task rewards, capability mismatch, and workload, and couples short-term utility optimization with long-term trust formation through dynamic reputation updates and capability-weight adaptation. By jointly shaping task assignment probabilities, reputation evolution, and skill profiles over repeated interactions, the mechanism incentivizes cooperative behavior and discourages strategic misrepresentation or overload. To operationalize this incentive design in a trust-minimized setting, we implement a lightweight blockchain-based enforcement layer that provides verifiable identity binding, task assignment commitment, and immutable logging of incentive-relevant state transitions. We evaluate the proposed mechanism through 50-round simulations involving 20 agents and 100 tasks, using GPT-4-based agents integrated with Solidity smart contracts. The results demonstrate improved task success rates, stable utility distributions, and emergent agent specialization, highlighting the effectiveness of incentive-centric design for trustworthy coordination among decentralized LLM agents.

The exponential growth of connected devices and embodied intelligent systems in B5G and 6G networks demands secure, adaptive, and autonomous communication among distributed agents. However, ensuring privacy-preserving coordination among these Agentic AI systems remains a major challenge, particularly in decentralized environments where transparency conflicts with confidentiality. To address this issue, we propose a Smart-contract-based Embodied Covert Agent Communication Architecture (SECA), which integrates Ethereum election contracts with image steganography to enable covert, highbandwidth communication among intelligent agents. In this framework, Blockchain-based agents utilize candidate images as visual carriers to embed encrypted messages, achieving imperceptible data exchange during on-chain interactions. We further design a Stackelberg Minimal Control Algorithm (SMCA) that enables adaptive manipulation of voting agents to ensure communication reliability with minimal control cost. Experimental results demonstrate that our approach achieves a transmission bandwidth up to 103 & times; higher than ORIM-based covert channels and passes multiple detection benchmarks (K-S and chi 2 tests), all without incurring additional gas costs. This work provides a foundational perspective for secure Agentic AI communication frameworks, bridging embodied intelligence, decentralized networking, and covert information transmission in emerging 6G environments.

Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements. However, auditing whether unlearning algorithms have truly erased the influence of specific data remains an open challenge. The lack of reliable and practical auditing mechanisms can lead to critical privacy risks, such as residual information leakage. This paper initiates a systematic investigation into whether existing unlearning algorithms can truly forget the designated data. We propose the first practical and general-purpose auditing framework for machine unlearning, inspired by the concept of proof of ignorance. Our framework addresses the key practicality limitations of existing methods by eliminating the need for retraining-from-scratch baselines, avoiding the training of large numbers of shadow models, and requiring no intrusive intervention in the original training process. To evaluate the effectiveness of our framework, we first conduct validation experiments to verify its soundness and completeness. We then perform comprehensive experiments across six datasets and ten representative unlearning methods. The results demonstrate that our framework reliably distinguishes between successful and failed unlearning. In particular, we observe that retraining-based and fine-tuning-based methods can achieve effective unlearning, even when the target data remain in the original dataset. In contrast, de-optimization-based methods fail to achieve true unlearning and instead degrade the model's performance. Fisher/Hessian-based methods also fail to unlearn requested data, even formal certification is provided. Moreover, we show that our framework is robust against fake unlearning attempts and generalizes well to large language models.

Watermarking has emerged as a key defense against the misuse of machine-generated images (MGIs). Yet the robustness of these protections remains underexplored. To reveal the limits of SOTA proactive image watermarking defenses, we propose HIDE SEEK (HS), a suite of versatile and cost-effective attacks that reliably remove embedded watermarks while preserving high visual fidelity.

Generative models, such as text-to-image diffusion models, have shown exceptional ability in generating high-quality images but pose a risk of generating inappropriate content (e.g., harmful or copyrighted material). Concept unlearning is one of the most prominent methods for ensuring safety and compliance of the diffusion models. However, existing methods overly focus on unlearning itself while neglecting its impact on the retained distribution. Previous studies have shown that the key challenge in unlearning research lies not only in achieving unlearning but also in minimizing the impact on the retained distribution, i.e., maintaining consistency before and after unlearning. The most severe manifestation of this challenge is the unintended degradation of unrelated concepts, which is a common flaw in many concept unlearning methods. These methods cannot sustain unlearning training because excessive unlearning updates lead to severe degradation in model performance on the retained set. Our analysis shows that this inconsistency is derived from a severe imbalance between the unlearning loss and the retention loss in the unlearning process, which leads to destructive updates of the feature space. To address the above challenges, we propose an unlearning framework named Consistent Unlearning, which prioritizes the maintenance of retention consistency. This framework leverages knowledge distillation to rebalance the unlearning and retention losses from multiple perspectives and introduces a feature space optimization objective to ensure the unlearning process is stable and controllable. Our method enables accurate and effective unlearning, making the behavior of the unlearned model on the retained dataset nearly indistinguishable from the original model. Extensive experiments on object, style, celebrity, and NSFW unlearning tasks demonstrate that our method effectively removes target concepts while achieving state-of-the-art consistency on the retained set.