Large language models (LLMs) can convert long documents into structured research variables. Auditability becomes difficult when evidence selection and model scoring are not stored. We study an inspectable scoring workflow that converts archived annual-report Evidence Packs into a five-dimension digital-transformation disclosure score. The design combines report selection, PDF-to-text normalization, bounded paragraph retrieval, and DeepSeek rubric scoring through a strict JavaScript Object Notation (JSON) interface; the original canonical prompt/run binding is not available. We evaluate three measurement questions and one secondary utility check: reproducible Evidence Pack construction, LLM–human agreement on shared Evidence Packs, ordinal error/confusion diagnostics, and non-causal panel-use stress tests. The evaluation panel contains 476 firm-year observations from 64 firms for 2013–2024. The post-validation score shows high internal coherence for the additive rubric (Cronbach’s α = 0.893; first principal component explains 0.704 of standardized dimensional variance), substantial within-firm variation (mean within-firm standard deviation (SD) 2.901; 92.19% non-zero), and strong LLM–human agreement on shared Evidence Packs (total-score Spearman 0.910, Pearson 0.921, mean absolute error (MAE) 1.273; dimension-level weighted kappa 0.665–0.823). Human–Human total-score reliability, measured by the intraclass correlation coefficient (ICC), is ICC(2,1) = 0.811. Baseline comparisons show lower error than frozen lexicon-count and firm-grouped term frequency–inverse document frequency (TF-IDF) Ridge alternatives (MAE = 1.273 vs . 6.387 and 2.737). A 96-record rule-based retrieval audit shows high mean dimension coverage (0.936) but incomplete heuristic paragraph recall@30 (micro recall = 0.419); four audited zero-hit packs are false empty (4.17%). The main reliability claim is therefore bounded to shared archived evidence. Calibration, error-distribution, and confusion-grid analyses indicate that disagreement is concentrated near adjacent rubric levels, with extreme mismatches rare, supporting score ordering conditional on shared archived Evidence Packs. Secondary fixed-effects and event-time checks show only that the score can enter panel workflows; pre-entry differences and year-effect sensitivity require non-causal interpretation. The main contribution is an evidence-bounded audit template rather than an exact endpoint-rerun claim.
Internet of Things (IoT) applications are widely employed in public transportation systems. However, the development of software for such systems—often characterized by high mobility and harsh operational environments—can be challenging and time-consuming. Model-driven engineering (MDE), which has proven effective in many domains, can also support the design and implementation of IoT-based public transportation systems (IPTSs). However, existing studies on the use of MDE techniques and approaches for the development of IPTSs mostly consider only forward engineering (FE), which limits the coverage of existing applications and cannot support synchronization between the created programs and the corresponding application models. Therefore, this study introduces a model-driven round-trip engineering (RTE) methodology that combines both FE and reverse engineering (RE) of IPTSs, in which synchronization between instance models created using a domain-specific modeling language, called DSML4PT, and the underlying code is achieved automatically through a dedicated toolchain. When using the DSML4PT language, legacy IPTS programs can be reverse engineered in an environment called RE4PT. Models extracted from existing programs can be automatically reprocessed using DSML4PT. Evaluation results clearly show that using RE4PT enables model-code synchronization and supports the integration of existing IPTS applications into the RTE, regardless of whether they were previously generated using a modeling language or even no models at all. Considering eight different real case studies, the proposed methodology reduced RE time by approximately 75%, a reduction that a paired statistical comparison of the case-level averages confirms is unlikely to be due to chance. Moreover, the qualitative assessment conducted with the participation of domain experts specifically highlighted the RE features addressing ontological, visualization, scaling, conformance and automation aspects, and also confirmed the usability and efficiency of the proposed RTE environment for IPTS development overall.
The volume of sentiment-rich text data produced on a daily basis by social network sites such as Twitter is very large. However, traditional approaches to sentiment analysis, especially approaches based on the Bag-of-Words (BoW) model have a difficult time modeling more subtle thematic content structure and interrelationships in this data. The approaches tend to narrow down text to an easy word frequency level, leading to the loss of valuable semantic information required for accurate sentiment classification. This problem is even more evident when one has to work with large complex data. Furthermore, being high-dimensional feature spaces, BoW methods produce represented spaces that can saturate traditional classification models, decreasing their level of accuracy and reliability. The article overcomes these drawbacks of traditional topics modeling by considering using Latent Dirichlet Allocation (LDA) as an intricate subject topic extraction method in English-language data tweets sentiment examination. LDA can reveal latent thematic structures in the tweets that enhance representation of features to be used in sentiment classification. The research methodology will consist of preprocessing the Twitter Sanders file, creating feature sets of different granularity (E40, E100, E300, E500), and testing them in Decision Trees (DT), Naive Bayes (NB) and K-Nearest Neighbors (KNN). Such distinction of sets of features allows investigating the issue of influence of feature granularity on the performance of a classifier in more detail. Findings indicate that LDA-based features substantially outperform BoW-based features in sentiment classification across all evaluation metrics. Decision Tree and Naive Bayes classifiers achieved the highest accuracy, precision, and recall using LDA features, with optimal performance observed at the E300 configuration. KNN behaved better with the smaller feature sets and poorly with the larger features sets. These improvements have also been confirmed using statistical tests and this includes paired t-tests, Wilcoxon signed-rank tests and are shown to be statistically significant across the board. These findings establish LDA as an effective and computationally efficient feature extraction technique for sentiment classification in social media data. This study contributes a systematic methodology for enhancing sentiment analysis accuracy and provides a foundation for future work integrating LDA with advanced machine learning algorithms.
In aviation and other safety-critical domains, computer vision systems must operate with high reliability, but are often trained on noisy and inconsistent bounding box annotations. NoisyBox is introduced as an open dataset for evaluating classification robustness under noisy bounding box annotations in fine-grained, safety-critical visual domains. The dataset consists of cockpit instrument images collected from flight simulation software and labeled by eight annotators, each contributing a single annotation per image. Due to lack of redundancy and expert oversight, the dataset contains significant bounding box noise. A separate expert-labeled dataset serves as the gold standard for benchmarking purposes. Evaluation of standard classification models revealed substantial sensitivity to annotation quality. Models trained on gold-standard data showed sharp performance degradation when tested on NoisyBox, with average F1 scores dropping from 0.98 to 0.38 (VGG16) and from 0.99 to 0.48 (ResNet50). Traditional ensemble methods such as majority voting were rendered ineffective by structured spatial noise (F1: 0.42), as geometric distortions led to correlated errors across models, failing to surpass the most robust single backbone (ResNet50, F1: 0.48). In contrast, a class-specific expert model selection approach provided a genuine performance gain increasing the average F1 score to 0.52 which demonstrates the benefit of backbone specialization for different classes. To address structured spatial noise, a classifier-agnostic low-supervision coverage-quality analysis framework is evaluated to identify bounding box types—full, partial, none, and multi-object. By leveraging intermediate feature representations rather than raw logits, the approach achieved a 0.92 macro-averaged F1 with 80% labeled data and maintained an F1 score of 0.83 with just 5% supervision. However, the 1% labeled setting should be interpreted cautiously because minority coverage categories become extremely sparse under this condition. NoisyBox provides a realistic and challenging benchmark for studying the impact of annotation quality and supports the development of robust learning strategies in domains where repeated labeling or expert annotation is impractical.
The capacity to anticipate and identify anomalous behavior remains fundamental to the safety and operational reliability of autonomous mobile systems. Yet temporal modeling approaches currently available tend to demand substantial computational resources or intricate parameter tuning, which meaningfully constrains their adoption on resource-limited edge devices. To address this gap, we introduce Temporal Naive Bayes (TNB), a lightweight and highly interpretable extension of the classical Naive Bayes classifier, developed specifically for sequential sensor data. TNB embeds a first-order autoregressive process directly into the feature likelihood estimation, thereby capturing temporal dependencies while incurring minimal computational overhead. The approach is validated across four scenarios of increasing complexity—from idealized synthetic data through to real-time deployment in the CARLA simulator and a real-world smartphone inertial measurement unit (IMU) dataset. Across these settings, TNB proves effective at isolating critical temporal dynamics, outperforming standard baselines in both multiclass maneuver recognition and anomaly detection tasks. An open-source pipeline accompanies this work, illustrating TNB’s practical value as a lightweight, real-time diagnostic tool for autonomous navigation.
