Knowledge graph generation typically relies either on predefined ontologies or on schema-free extraction. Ontology-driven pipelines enforce consistent typing but require costly schema design and maintenance, whereas schema-free methods often produce fragmented graphs with weak global organization, especially in long technical documents with dense, context-dependent information. We propose TRACE-KG (Text-dRiven schemA for Context-Enriched Knowledge Graphs), a framework that jointly constructs a context-enriched knowledge graph and an induced schema without assuming a predefined ontology. TRACE-KG captures conditional relations through structured qualifiers and organizes entities and relations using a data-driven schema that serves as a reusable semantic scaffold while preserving full traceability to the source evidence. Experiments show that TRACE-KG produces structurally coherent, traceable knowledge graphs and offers a practical alternative to both ontology-driven and schema-free construction pipelines.

Persistent Large Language Model (LLM) agents expose a critical governance gap in memory management. Standard Retrieval-Augmented Generation (RAG) frameworks treat memory as passive storage, lacking mechanisms to resolve contradictions, enforce privacy, or prevent outdated information ("zombie memories") from contaminating the context window. We introduce MemArchitect, a governance layer that decouples memory lifecycle management from model weights. MemArchitect enforces explicit, rule-based policies, including memory decay, conflict resolution, and privacy controls. We demonstrate that governed memory consistently outperforms unmanaged memory in agentic settings, highlighting the necessity of structured memory governance for reliable and safe autonomous systems.

Diversity can be broadly defined as the presence of meaningful variation across elements, which can be viewed from multiple perspectives, including statistical variation and geometric structural richness in the dataset. Existing diversity metrics, such as feature-space dispersion and metric-space magnitude, primarily capture distributional variation or entropy, while largely neglecting the geometric structure of datasets. To address this gap, we introduce a framework based on topological data analysis (TDA) and persistence landscapes (PLs) to extract and quantify geometric features from data. This approach provides a theoretically grounded means of measuring diversity beyond entropy, capturing the rich geometric and structural properties of datasets. Through extensive experiments across diverse modalities, we demonstrate that our proposed PLs-based diversity metric (PLDiv) is powerful, reliable, and interpretable, directly linking data diversity to its underlying geometry and offering a foundational tool for dataset construction, augmentation, and evaluation.

Despite an extensive body of literature on trust in technology, designing trustworthy AI systems for high‐stakes decision domains remains a significant challenge. Widely used system design guidelines and tools are rarely attuned to domain‐specific trustworthiness principles. In this study, we introduce a design framework to address this gap within intelligence analytic tasks, called the Principles‐based Approach for Designing Trustworthy, Human‐centered AI using the MAST Methodology (PADTHAI‐MM). PADTHAI‐MM builds on the Multisource AI Scorecard Table (MAST), an AI decision support system evaluation tool designed in accordance to the U.S. Intelligence Community's standards for system trustworthiness. We demonstrate PADTHAI‐MM in our development of the Reporting Assistant for Defense and Intelligence Tasks (READIT), a research platform that leverages data visualizations and natural language processing‐based text analysis to emulate AI‐enabled intelligence reporting aids. To empirically assess the efficacy of PADTHAI‐MM, we developed two versions of READIT for comparison: a “High‐MAST” version, which incorporates AI contextual information and explanations, and a “Low‐MAST” version, designed to be akin to inscrutable “black box” AI systems. Through an iterative design process guided by stakeholder feedback, our multidisciplinary design team developed prototypes that were evaluated by experienced intelligence analysts. Results substantially supported the viability of PADTHAI‐MM in designing for system trustworthiness in this task domain. We also explored the relationship between analysts' MAST ratings and three theoretical categories of information known to impact trust: process , purpose , and performance . Overall, our study supports the practical and theoretical viability of PADTHAI‐MM as an approach to designing trustable AI systems.

High-quality training data is the foundation of machine learning and artificial intelligence, shaping how models learn and perform. Although much is known about what types of data are effective for training, the impact of the data's geometric structure on model performance remains largely underexplored. We propose that both the richness of representation and the elimination of redundancy within training data critically influence learning outcomes. To investigate this, we employ persistent homology to extract topological features from data within a metric space, thereby offering a principled way to quantify diversity beyond entropy-based measures. Our findings highlight persistent homology as a powerful tool for analyzing and enhancing the training data that drives AI systems.

Explainable Artificial Intelligence (XAI) can enhance trust in AI by offering cues that support human reasoning of AI behavior. Yet its effects on trust evolution remain unclear, especially when AI makes errors. This study examines how explanations of AI predictions influence human trust in AI-assisted decision-making under varying error severities. We tested two XAI visualizations, two AI error types, and three explanation strategies in simulated baggage screening tasks through an online study. Responses from 280 participants show that XAI representation significantly affects human compliance with AI during errors, while AI error type further shapes compliance after AI errors. AI Error type also impacts verification behaviors during AI errors, such as requesting explanations or ground truth. Moreover, strategies for conveying XAI influence perceived trust in AI, highlighting important implications for generalizing XAI effects beyond lab-based trust research.

As crowdsourcing emerges as an efficient and cost-effective method for obtaining labels for machine learning datasets, it is important to assess the quality of crowd-provided data to improve analysis performance and reduce biases in subsequent machine learning tasks. Given the lack of ground truth in most cases of crowdsourcing, we refer to data quality as the annotators’ consistency and credibility. Unlike the simple scenarios where kappa coefficient and intraclass correlation coefficient usually can apply, online crowdsourcing requires dealing with more complex situations. We introduce a systematic method for evaluating data quality and detecting spamming threats via variance decomposition, and we classify spammers into three categories based on their different behavioral patterns. A spammer index is proposed to assess entire data consistency, and two metrics are developed to measure crowd workers’ credibility by utilizing the Markov chain and generalized random effects models. Furthermore, we demonstrate the practicality of our techniques and their advantages by applying them to a face verification task using both simulated and real-world data collected from two crowdsourcing platforms.

The Multisource AI Scorecard Table (MAST) is a checklist tool to inform the design and evaluation of trustworthy AI systems based on the U.S. Intelligence Community's analytic tradecraft standards. In this study, we investigate whether MAST can be used to differentiate between high and low trustworthy AI-enabled decision support systems(AI-DSSs). Evaluating trust in AI-DSSs poses challenges to researchers and practitioners. These challenges include identifying the components, capabilities, and potential of these systems, many of which are based on the complex deep learning algorithms that drive DSS performance and preclude complete manual inspection. Using MAST, we developed two interactive AI-DSS testbeds. One emulated an identity-verification task in security screening, and another emulated a text-summarization system to aid in an investigative task. Each testbed had one version designed to reach low MAST ratings, and another designed to reach high MAST ratings. We hypothesized that MAST ratings would be positively related to the trust ratings of these systems. A total of 177 subject-matter experts were recruited to interact with and evaluate these systems. Results generally show higher MAST ratings for the high-MAST compared to the low-MAST groups, and that measures of trust perception are highly correlated with the MAST ratings. We conclude that MAST can be a useful tool for designing and evaluating systems that will engender trust perceptions, including for AI-DSS that may be used to support visual screening or text summarization tasks. However, higher MAST ratings may not translate to higher joint performance, and the connection between MAST and appropriate trust or trustworthiness remains an open question.

Data augmentation that introduces diversity into the input data has long been used in training deep learning models. It has demonstrated benefits in improving robustness and generalization, practically aligning well with other regularization strategies such as dropout and weight decay. However, the underlying mechanism of how diverse training data contributes to model improvements remains unknown. In this paper, we investigate the impact of data diversity on the weight space of deep neural networks using Random Matrix Theory. Through spectral analysis and comparing models trained with data augmentation, dropout, and weight decay, we reveal that increasing data diversity alters the weight spectral distribution similarly to other regularization techniques, while displaying a pattern more closely aligned with dropout than with weight decay. Building on these insights, we propose a metric to explain and compare the benefits of diversity introduced by traditional data augmentations and those achieved through synthetic data.

As machine learning models continue to swiftly advance, calibrating their performance has become a major concern prior to practical and widespread implementation. Most existing calibration methods often negatively impact model accuracy due to the lack of diversity of validation data, resulting in reduced generalizability. To address this, we propose a calibration method that incorporates synthetic data without compromising accuracy. We derive the expected calibration error (ECE) bound using the Probably Approximately Correct (PAC) learning framework. Large language models (LLMs), known for their ability to mimic real data and generate text with mixed class labels, are utilized as a synthetic data generation strategy to lower the ECE bound and improve model accuracy on real test data. Additionally, we propose data generation mechanisms for efficient calibration. Testing our method on four different natural language processing tasks, we observed an average up to 34% increase in accuracy and 33% decrease in ECE.

This study tests the Multisource AI Scorecard Table for evaluating the trustworthiness of AI-enabled decision support systems. Forty participants reviewed one of four system descriptions, either a low MAST rating or a high MAST rating, and either a face-matching system or a text summarization system. Participants rated the system and completed trust and credibility questionnaires. Results show that MAST items are correlated with validated trust and credibility items, suggesting that the tool can be useful, but more so for text summarization based systems.