Large language models (LLMs) are increasingly used to generate natural language explanations of predictive machine learning models, yet their outputs often emphasize plausibility over faithfulness to the underlying model logic. This study applies counterfactual testing to evaluate the fidelity of LLM explanations in a structured setting: decision tree models predicting customer churn (i.e., when customers stop using a company’s product or service-a classic benchmark problem in predictive modeling research). We design two counterfactual tests across four prompting strategies that vary in whether dataset and/or model information are provided. Results from 1,200 generated explanations show that supplying model context substantially improves faithfulness whereas dataset context provides little benefit, and LLMs often broaden explanations beyond the model’s true decision path. Notably, even when given full model logic, LLMs remain predisposed toward generating plausible rather than strictly faithful explanations. These findings highlight both the promise and the limits of LLMs as interpreters of predictive models and introduce counterfactual testing as a potential framework for evaluating explanation faithfulness in data-heavy contexts.
This paper presents an LLM-enhanced ontology framework for cybersecurity in Internet of Medical Things (IoMT) environments, integrating hardware performance counter (HPC) features, device metadata, and known vulnerabilities to enable reasoning-driven, explainable anomaly detection. The study introduces a hybrid human-LLM ontology development workflow that accelerates ontology construction and refinement while preserving conceptual rigor and semantic consistency. It further investigates how large language models (LLMs) can augment ontology engineering through a comparative evaluation of three state-of-the-art models: ChatGPT-o3, ChatGPT-4o, and Gemini 2.5 Pro, assessed across structural and semantic dimensions. Using the proposed LLM-enhanced ontology, we achieve 84.29% anomaly (e.g., malware) classification accuracy from HPC data while mapping detected anomalies to relevant vulnerabilities and device properties. Experimental results demonstrate that LLMs significantly improve ontology quality and reasoning coherence, with Gemini 2.5 Pro achieving the best balance between expansion and consistency.
Educational institutions lack a systematic understanding of how generative AI policies are implemented across computing courses, and keyword-based methods alone do not capture their nuance and complexity. This study presents a syllabus-based framework for analyzing generative AI policy adoption practices and trends in computing education. Policyrelated text is identified through keyword-based retrieval with contextual expansion and classified into policy types using a combination of rule-based inference and embedding-based semantic similarity. The resulting policy categories (No Policy, Permitted, Restricted, Discouraged, and Prohibited), together with extracted course metadata, enable analysis of adoption patterns over time and across course levels. Using 148 course syllabi from Spring 2024 to Fall 2025, results show that adoption increased from $\mathbf{12.8 \%}$ to $\mathbf{58.8 \%}$ over four semesters, with similar adoption rates between undergraduate $(40.4 \%)$ and graduate $(39.6 \%)$ courses. Instructor concerns are identified via semantic matching, with primary emphasis on over-dependence $(72.9 \%)$, proper citation $(57.6 \%)$, and appropriate use $(52.5 \%)$. This work provides a validated methodology ($90.56 \%$ accuracy) for characterizing generative AI policy adoption in higher education.
In large and complex systems such as the Circular Factory, vast amounts of heterogeneous and uncertain data are generated. Classical Bayesian Networks (BNs) provide a principled framework for reasoning under uncertainty, but their reliance on discrete Conditional Probability Tables (CPTs) limits their applicability in continuous domains. In the context of semantic computing, where heterogeneous sensor data and knowledge graph information must be integrated, this limitation prevents probabilistic reasoning from being consistently applied to continuous quantities and measurement-based information. This paper introduces a framework for continuous Bayesian Networks that replaces discrete CPTs with functional dependencies and integrates the Guide to the Expression of Uncertainty in Measurement (GUM) as a standardized approach for analytical uncertainty propagation. The proposed method enables probabilistic reasoning over continuous variables while explicitly accounting for measurement and parameter uncertainty in a mathematically transparent way. To complement the analytical formulation, Monte Carlo (MC) simulations with Gaussian Mixture Models (GMMs) are employed to handle non-linear and non-Gaussian cases, allowing for numerical validation of the GUM approximation. We present illustrative examples comparing both approaches and discuss their agreement and limitations. The results demonstrate that GUM-based inference provides accurate and computationally efficient uncertainty estimates in locally linear regions, while Monte Carlo methods serve as a reliable benchmark for more complex, non-linear dependencies. This framework thus lays the foundation for robust and interpretable uncertainty reasoning in environments such as the Circular Factory. Furthermore, it provides a bridge between probabilistic reasoning and semantic representations by enabling continuous uncertainty information to be embedded within knowledge-based systems, thus advancing uncertainty-aware semantic computing.
Enterprise Knowledge Graphs have become essential for unifying heterogeneous data and enforcing semantic governance. However, the construction of their underlying ontologies remains a resource-intensive, manual process that relies heavily on domain expertise. This paper introduces OntoEKG, a LLM-driven pipeline designed to accelerate the generation of domain-specific ontologies from unstructured enterprise data. Our approach decomposes the modelling task into two distinct phases: an extraction module that identifies core classes and properties, and an entailment module that logically structures these elements into a hierarchy before serialising them into standard RDF. Addressing the significant lack of comprehensive benchmarks for end-to-end ontology construction, we adopt a new evaluation dataset derived from documents across the Data, Finance, and Logistics sectors. Experimental results highlight both the potential and the challenges of this approach, achieving a fuzzy-match F1-score of 0.724 in the Data domain while revealing limitations in scope definition and hierarchical reasoning.
