Current evaluation methods for large language models (LLMs) primarily rely on static benchmarks, presenting two major challenges: limited knowledge coverage and fixed difficulties that mismatch with the evaluated LLMs. These limitations lead to superficial assessments of LLM knowledge, thereby impeding the targeted model optimizations. To bridge this gap, we propose JudgeAgent, a knowledge-driven and dynamic evaluation framework for LLMs. To address the challenge of limited knowledge coverage, JudgeAgent leverages LLM agents equipped with context graphs to traverse knowledge structures systematically for question generation. Furthermore, to mitigate data contamination and difficulty mismatch, it adopts a difficulty-adaptive and multi-turn interview mechanism. Thereby, JudgeAgent can achieve comprehensive evaluations and facilitate more effective improvement of LLMs. Empirical results demonstrate that JudgeAgent enables more comprehensive evaluations and facilitates effective model iterations, highlighting the potential of this knowledge-driven and dynamic evaluation paradigm. The source code is available on https://github.com/DataArcTech/JudgeAgent.

Inter-document event relation extraction aims to identify relations between document-thematic events. Specifically, a document-thematic event refers to the central event conveyed by a document, with the common assumption that each document contains only one document-thematic event. Existing methods face the following challenges: (1) difficulty in capturing event association cues. There is a lack of shared context between events, making it challenging to directly obtain relevant cues that indicate their relations; (2) a lack of sufficient evidence for event relation judgment. When two documents show no obvious content overlap, clear evidence to determine their event relation is often unavailable. To address these challenges, this paper proposes a knowledge-augmented cue mining model for inter-document eventrelation extraction, termed KACM. The KACM not only mines association cues from within the document content but also leverages external knowledge to assist in relation inference. Experimental results on the DTER dataset demonstrate that KACM can effectively perform inter-document event relation extraction in terms of causal, temporal, and coreference relations.

Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive reports. In contrast to existing studies that mainly focus on unstructured web content, a more challenging DR task should additionally utilize structured knowledge to provide a solid data foundation, facilitate quantitative computation, and lead to in-depth analyses. In this paper, we refer to this novel task as Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge. Furthermore, we propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that reasons over both kinds of knowledge and integrates the texts, figures, and tables into coherent multimodal reports. The key design is the Structured Knowledge Analyzer, which utilizes both coding and vision-language models to produce figures, tables, and corresponding insights. To support systematic evaluation, we construct KDR-Bench, which covers 9 domains, includes 41 expert-level questions, and incorporates a large number of structured knowledge resources (e.g., 1,252 tables). We further annotate the main conclusions and key points for each question and propose three categories of evaluation metrics including general-purpose, knowledge-centric, and vision-enhanced ones. Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics, and even surpasses the Gemini DR agent on vision-enhanced metrics, highlighting its effectiveness in deep, structure-aware knowledge analysis. Finally, we hope this work can serve as a new foundation for structured knowledge analysis in DR agents and facilitate future multimodal DR studies.

LLMs hold great promise for healthcare applications, but fast-changing medical knowledge can quickly make their outputs outdated or inaccurate, limiting use in high-stakes settings. Model editing can update LLMs without full retraining, but parameter-based methods often break locality and are risky in medicine, making retrieval-based editing a better fit. However, applying model editing methods to the medical domain has two key challenges: (1) retrieval-based methods suffer from representation overlap within the medical knowledge space that causes inaccurate retrieval and reduces editing accuracy; (2) existing medical editing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world applications. To address these challenges, we construct MedVersa, an expanded benchmark that evaluates single and batch edits across broader medical coverage under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query–key module for precise matching with an attention-based prompt encoder for informative guidance. Experiments across various medical benchmarks show that our MedREK consistently improves key metrics and provides the first validated solution for batch editing in medical LLMs.

Retrieval-Augmented Generation (RAG) has become a prevailing paradigm for enhancing Large Language Models (LLMs) with non-parametric knowledge. Vanilla RAG efficiently handles simple queries but struggles with relational or multi-hop reasoning. Graph-based RAG alleviates this issue but incurs higher inference complexity and latency. In practice, user queries can differ significantly in their complexity, rendering a fixed RAG strategy suboptimal. However, existing hybrid text-graph RAG methods typically rely on heuristic and LLM-based routing, resulting in unnecessary overhead and strong dependence on the underlying LLM. To address these challenges, we propose R^2Adapter, a lightweight plug-in Routing and Rewriting Adapter designed to allocate queries between vanilla and graph-based RAG dynamically. By routing only the queries that genuinely benefit from graph-based reasoning, R^2Adapter reduces unnecessary graph retrieval overhead. Additionally, uncertain graph-routed queries are rewritten to better expose their multi-hop reasoning requirements, improving retrieval quality without additional supervision. Extensive experiments on three multi-hop QA benchmarks demonstrate that R^2Adapter reduces graph-based RAG usage by up to 59

With the rapid development of multi-modal large models, the Visual–Language–Action (VLA) model has gradually become a new paradigm for autonomous robot operations. The VLA model encodes experimental images and text instructions separately using an image encoder and a text encoder. The encoded multi-modal vector information is then fed into a large language model (LLM) to generate the next action. While they inherit the generalization capabilities of large language models, VLA models often struggle to ensure accuracy and reliability in complex scenes. Some studies have attempted to improve VLA performance by enhancing the fine-tuning process or introducing staged operations; however, these improvements often overlook the stable extraction of important visual features, which are crucial for VLA models. In typical VLA tasks, the instruction text inherently contains semantic information related to image elements. Research has shown that leveraging text supervision for visual feature extraction can enhance feature quality. In this paper, we propose a semantically supervised visual encoder called SeDINO (Semantically Supervised DINO), which efficiently fuses DINO’s element localization capabilities with CLIP’s semantic information. We further employ an MLP (Multi-Layer Perceptron) network to align the semantic vectors output by the CLIP text encoder with the image feature vectors derived from DINO, fully leveraging DINO’s element localization and CLIP’s semantic interaction capabilities. We validate SeDINO on six mainstream image datasets, and it demonstrates superior segmentation performance compared to current leading models. Additionally, we incorporate the proposed SeDINO into the VLA framework, using OpenVLA-7B and DINOv2-base as backbone models, and evaluate it on the LIBERO dataset and real-world scenarios.

Knowledge Graphs (KGs) are widely used to mitigate the limitations of Large Language Models (LLMs), such as outdated knowledge and hallucinations. Existing LLM-KG integration frameworks typically rely on predefined operators to retrieve factual knowledge from KGs and inject it into prompts for answer generation. This paradigm faces two critical bottlenecks: 1) Inflexibility: The predefined operators are limited in scope and thus lack sufficient compositional expressiveness to fully capture the complex semantics required by KG questions. 2) Unscalability: Direct injection of factual knowledge into prompts limits scalability in handling large-scale factual knowledge. To address these two bottlenecks, we propose Code-on-Graph (CoG), a programmatic reasoning framework for LLM-KG integration. Specifically, given the factual knowledge retrieved at each reasoning step, CoG first identifies the corresponding KG schemas and represents these schemas as Python classes, which serve as abstract interfaces to the retrieved facts. It then generates executable code grounded in these classes, with the retrieved facts instantiated as objects of the corresponding classes during execution. This design enables flexible code-based reasoning while avoiding the direct injection of large-scale factual knowledge into prompts. Experiments on WebQSP, CWQ, and GrailQA demonstrate that CoG outperforms prior state-of-the-art models by up to 10.5

Memory enables Large Language Model (LLM) agents to perceive, store, and use information from past dialogues, which is essential for personalization. However, existing methods fail to properly model the temporal dimension of memory in two aspects: 1) Temporal inaccuracy: memories are organized by dialogue time rather than their actual occurrence time; 2) Temporal fragmentation: existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. To address these limitations, we propose Temporal Semantic Memory (TSM), a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. During memory construction, it first builds a semantic timeline rather than a dialogue one. Then, it consolidates temporally continuous and semantically related information into a durative memory. During memory utilization, it incorporates the query's temporal intent on the semantic timeline, enabling the retrieval of temporally appropriate durative memories and providing time-valid, duration-consistent context to support response generation. Experiments on LongMemEval and LoCoMo show that TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy, demonstrating the effectiveness of the proposed method.

Event ontology expansion aims to discover emerging event types from data and extend them to appropriate positions in the existing event ontology.. Existing methods typically cluster contextualized trigger representations and attach induced clusters to the ontology based on instance-level similarity. However, ontology expansion requires concept-level semantics that characterize event types, whereas contextualized trigger representations often conflate these semantics with surface contextual variation, leading to unstable clustering and unreliable hierarchy expansion. To address this issue, we propose ConceptE, a conceptualization-enhanced framework for event ontology expansion. ConceptE first derives concept-level semantics by prompting an LLM with the sentence and event trigger, producing a concise concept name and a natural-language description. It then jointly encodes these semantics with trigger information to build concept-enhanced representations aligned with ontology-level reasoning. This representation design supports more coherent event clustering, more reliable hierarchy expansion, and ontology-consistent type naming. Experiments on ACE, ERE, and MAVEN demonstrate that ConceptE consistently outperforms state-of-the-art approaches across all subtasks of event ontology expansion. In particular, it achieves improvements of up to 12.37% in BCubed-F1 for event clustering and 6.48% in Taxo_F1 for hierarchy expansion, demonstrating the effectiveness of the proposed ConceptE method.

Temporal Knowledge Graphs (TKGs), which utilize quadruples in the form of (subject, predicate, object, timestamp) to describe temporal facts, have attracted extensive attention. N-tuple TKGs (N-TKGs) further extend traditional TKGs by utilizing n-tuples to incorporate auxiliary elements alongside core elements (i.e., subject, predicate, and object) of facts, so as to represent them in a more fine-grained manner. Reasoning over N-TKGs aims to predict potential future facts based on historical ones. However, existing N-TKG reasoning methods often lack explainability due to their black-box nature. Therefore, we introduce a new Reinforcement Learning-based method, named MT-Path, which leverages the temporal information to traverse historical n-tuples and construct a temporal reasoning path. Specifically, in order to integrate the information encapsulated within n-tuples, i.e., the entity-irrelevant information within the predicate, the information about core elements, and the complete information about the entire n-tuples, MT-Path utilizes a mixture policy-driven action selector, which bases on three low-level policies, namely, the predicate-focused policy, the core-element-focused policy and the whole-fact-focused policy. Further, MT-Path utilizes an auxiliary element-aware GCN to capture the rich semantic dependencies among facts, thereby enabling the agent to gain a deep understanding of each n-tuple. Experimental results demonstrate the effectiveness and the explainability of MT-Path.

As a major fuel source for the tricarboxylic acid cycle, lactate controls energy metabolism through cell-to-cell or tissue-to-tissue lactate shuttles via monocarboxylate transporters (MCTs). Although lactate is shown to influence macrophage functions via histone lactylation, the specific functions of MCTs in macrophages remain incompletely understood. This study discovers that MCT1 and MCT4 have contrasting effects on regulating macrophage polarization. M1 polarization is associated with increase of MCT4 while M2 polarization is accompanied with increase of MCT1. MCT1 is mainly localized in mitochondria while MCT4 is localized on the plasma membrane. M1 polarization elevates lactate efflux from the cytoplasm to extracellular space, while M2 polarization increases intracellular lactate flux to mitochondria. At cellular level, blocking MCT1 exacerbates LPS-induced M1-like polarization and impairs mitochondria function. At animal level, deletion of MCT1 in macrophages exacerbates glucose intolerance, suppresses insulin secretion and increases islet cell death in high-fat diet fed mice. Mechanistically, lactate reduces insulin secretion through GPR81-cAMP-PKA signaling pathway. These findings not only disclose that the MCT1-mediated intracellular lactate shuttle to the mitochondria plays a pivotal role in governing macrophage polarization but also uncovers a functional interplay between macrophages and β cells in maintaining glucose homeostasis.

Traditional knowledge graph technologies still face significant challenges in converting human knowledge, expressed in natural language, into a formal language-based knowledge graph and utilizing it effectively. In recent years, code large language models (LLMs) have demonstrated remarkable capabilities in understanding both natural and formal languages, as well as in translating between them. These advancements are expected to drive significant breakthroughs in developing next-generation knowledge graph technologies. This paper reviews the application of code LLMs in KGs. Firstly, this paper systematically analyzes the role of code LLMs in enhancing key knowledge graph technologies across three critical areas: construction, reasoning, and question-answering. Secondly, a relatively detailed introduction to the existing methodologies in these areas is provided. Finally, this paper summarizes the current state in the field and offers insights into the future of knowledge graph technologies empowered by code LLMs. In the future, knowledge representation based on programming languages is expected to enable more efficient, automated, and complex operations on knowledge graphs, realizing knowledge programming.

Answering natural language questions with temporal intent on knowledge graphs (TKGQA) has attracted rising attention in recent years. TKGQA contains several subtasks, such as implicit expression parsing, relevant facts searching, and subgraph logic reasoning, and they are composed in a pipelined manner to finish the final task. Previous work learns isolated models for each subtask, which severely restricts the knowledge sharing between related subtasks and settings, leading to underperformance. Recently, Large Language Models (LLMs) have unlocked strong multi-task capabilities from reading instructive prompts. Motivated by these, we propose a unified generation framework based on instruction tuning, called InstructTQA, for TKGQA, which unifies the key subtasks in TKGQA via text generation. Besides, to improve the time-sensitivity of LLMs, we also design several auxiliary subtasks. By capturing inter-task dependency and collaboratively learning general knowledge from different subtasks, InstructTQA demonstrates superior performance compared to the state-of-the-art one on the MultiTQ dataset, thereby substantiating its efficacy in addressing the TKGQA task.

In this paper, we aim to enhance the robustness of Universal Information Extraction (UIE) by introducing a new benchmark dataset, a comprehensive evaluation, and a feasible solution. Existing robust benchmark datasets have two key limitations: 1) They generate only a limited range of perturbations for a single Information Extraction (IE) task, which fails to evaluate the robustness of UIE models effectively; 2) They rely on small models or handcrafted rules to generate perturbations, often resulting in unnatural adversarial examples. Considering the powerful generation capabilities of Large Language Models (LLMs), we introduce a new benchmark dataset for Robust UIE, called RUIE-Bench, which utilizes LLMs to generate more diverse and realistic perturbations across different IE tasks. Based on this dataset, we comprehensively evaluate existing UIE models and reveal that both LLM-based models and other models suffer from significant performance drops. To improve robustness and reduce training costs, we propose a data-augmentation solution that dynamically selects hard samples for iterative training based on the model’s inference loss. Experimental results show that training with only 15 % of the data leads to an average 8.1 % relative performance improvement across three IE tasks. Our code and dataset are available at: https://github.com/ICT-GoKnow/RobustUIE.

LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK.

Knowledge Base Question Answering (KBQA) aims to answer natural-language questions over a structured Knowledge Base (KB). Recent work improves KBQA by adopting an agentic reasoning paradigm, in which Large Language Models (LLMs) iteratively decompose a question, generate its corresponding logical queries, and interact with the KB to derive the answer. However, these methods typically fine-tune LLMs on reasoning trajectories synthesized via process supervision, which offers weak incentives for exploration and thus fails to strengthen the agentic reasoning ability. In this paper, we propose KnowCoder-A1, an LLM that can autonomously perform agentic reasoning on KBs to obtain answers. To incentivize autonomous exploration, KnowCoder-A1 trains the LLM under outcome-only supervision via a multi-stage curriculum reinforcement learning with an easy-to-hard curriculum. To establish foundational agentic capabilities, KnowCoder-A1 first fine-tunes the LLM on a small set of high-quality trajectories obtained through outcome-based rejection sampling. Then, to alleviate the reward sparsity inherent in outcome-only supervision, it applies multi-stage curriculum RL with reward schedules that progress from easy to hard. Trained with outcome-only supervision, KnowCoder-A1 exhibits powerful reasoning behaviors and consistently outperforms prior approaches across three mainstream datasets. Notably, on the zero-shot subset of GrailQA, KnowCoder-A1 achieves up to an 11.1

Aiming at the ambiguous and unclarity logical relationship among disposal text information in gas emergencies, a two-stage method focusing on gas accident case text is proposed for constructing the event knowledge graph of gas emergency disposal. First, a gas text joint extraction method (GTJM) for gas accident cases is proposed at the first stage. The multigranularity training tasks in the enhanced representation through knowledge integration (ERNIE) pretrained model are used to address the issue of insufficient context understanding when processing long texts in gas cases while combining multihead attention mechanism to obtain richer semantic information. The cascade binary tagging framework (CasRel) is then utilized to extract entities and relationships contained in the background and disposal information of accident cases, to tackle the problem of overlapping entity extraction in gas accident case texts. Second, in the second stage, the bidirectional encoder representation from transformers (BERT) is used to capture the contextual features of compressed events, while a novel Incept-text convolutional neural network (TextCNN) focuses on the global semantic information within compressed events. The combination of both models could resolve the problem of sequential relationship identification between long events much better. Finally, a clear event logic knowledge graph is formed for gas accident disposal in emergencies. The experimental results indicate that in the first phase, the GTJM model has an average improvement of 7% in F1 value compared to other baseline models. In the second phase, the accuracy of the sequential relationship identification reaches 92%. The feasibility of the proposed method is verified through real cases. The study could provide auxiliary decision support for emergency disposal in gas accidents.