Context: Indicator systems are pivotal in assessing software quality, conducting economic analysis, and various other domains. However, their validation is frequently hindered by the absence of explicit expected outputs, posing a significant challenge to their reliability and effectiveness. Objective: This paper aims to address this challenge by proposing a novel validation method for indicator systems based on metamorphic testing (MT). The method seeks to eliminate the oracle problemby generating follow-up test cases and verifying their consistency through designed logical constraints, known as metamorphic relations, between inputs and outputs. Methods: To validate the proposed method, we conducted empirical research on three diverse cases: software quality evaluation, red wine quality assessment, and ecological and economic benefit evaluation. We designed metamorphic relations specific to each case and applied our method to identify inconsistencies and potential errors in the indicator systems. Results: Our experiments successfully identified errors in real-world indicator systems, demonstrating the method’s capability to detect flaws that traditional methods might overlook. Furthermore, we performed mutation testing, achieving an average mutation score of 0.83 on the mutation dataset, which significantly outperforms the traditional statistical analysis method with a maximum score of 0.65. Conclusion: This paper presents a generalized solution for the validation of indicator systems lacking clear expectations, offering substantial application value in the fields of software engineering and decision support. The proposed method not only increases confidence in the reliability of indicator systems but also opens up promising research avenues for further improving the accuracy and efficiency of validation processes in complex domains.

Traditionally, aligning interface fields when integrating test equipment into simulation platforms involves large-scale fields, heterogeneous naming schemes, complex unit and coordinate constraints, and difficult multi-source field composition. Recent advances in large language models (LLMs) provide feasible approaches to address these challenges. In this paper, we propose LLM-MSIARG (Large Language Model-based Multi-Stage Interface Adaptation Rule Generation), a multi-stage framework for generating interface adaptation rules. It consists of four stages: schema analysis, logic construction, specification writing, and logic auditing, connected by structured intermediate states to form a traceable and auditable lightweight workflow. To overcome issues such as key information omission in long contexts and hallucinations in LLMs, we design and employ source-side chunking, target-side batching, and cross-chunk anchor aggregation mechanisms. Experimental results on the interconnection task between DJI Payload SDK and the MAVLink simulation plugin show that, when using a 35B model, LLM-MSIARG achieves an overall adaptation accuracy of 89.83% and an overall adaptation rule correctness of 74.50%. The proposed method can output field dependencies, transformation logic, validity constraints, and risk warnings for engineering implementation, providing a feasible approach for integrating heterogeneous equipment into simulation platforms.

Evaluation indicator selection is a key step in constructing an effectiveness evaluation indicator system and directly affects the quality of subsequent evaluation results. Existing autoencoder-based indicator selection methods mainly learn indicator contributions from reconstruction errors. In effectiveness evaluation scenarios, however, they still suffer from relatively high redundancy in selected indicators, insufficient stability of selection results, and limited adaptation to the construction requirements of evaluation indicator systems. To address these issues, this paper proposes a robust fractal autoencoder-based indicator selection method adapted to evaluation indicator systems. Building on the global reconstruction, local reconstruction, weight exponentiation, and dynamic-window mechanisms of the Robust Fractal Autoencoder (RFAE), the proposed method introduces indicator-selection-oriented constraints. These constraints regulate indicator selection weights from three perspectives: indicator redundancy, capability-dimension coverage, and sample discriminability. As a result, the selected indicators better satisfy the construction requirements of a refined, stable, reasonably covered, and interpretable effectiveness evaluation indicator system. Experiments based on multi-agent adversarial simulation data show that the proposed method can be used to construct a core indicator subset with low redundancy, reasonable coverage, and favorable sample discriminability, thereby providing a feasible approach for the automated construction of effectiveness evaluation indicator systems for complex systems.

Compared with general images, objects in remote sensing (RS) images typically exhibit a conspicuous diversity due to their arbitrary orientations. However, many of the prevalent detectors generally apply an inflexible strategy in setting the angles of anchor, ignoring the fact that the number of possible orientations is predictable. Consequently, their processes integrate numerous superfluous angular considerations and hinder their efficiency. To deal with this situation, we propose a dynamic density-driven estimation network (DDE-Net). We design three core modules in DDE-Net: a density-map and mask generation module (DGM), mask routing prediction module (MRM), and spatial-balance calculation module (SCM). DGM is designed for the generation of a density map and mask, which can extract salient features. MRM is for the prediction of object orientation and corresponding weights, which are used to calculate feature maps. SCM is used to affine transform the convolution kernel, which applies an adaptive weighted compute mechanism to enhance the average feature, so as to balance the spatial difference to the rotation feature extraction. A broad array of experimental evaluations have conclusively shown that our methodology outperforms existing state-of-the-art detectors on common aerial object datasets (DOTA and HRSC2016).

EEG-based emotion recognition is a task that uses scalp-EEG data to classify the emotion states of humans. The study of EEG-based emotion recognition can contribute to a large spectrum of application fields including healthcare and human–computer interaction. Recent studies in neuroscience reveal that the brain regions and their interactions play an essential role in the processing of different stimuli and the generation of corresponding emotional states. Nevertheless, such regional interactions, which have been proven to be critical in recognizing emotions in neuroscience, are largely overlooked in existing machine learning or deep learning models, which focus on individual channels in brain signals. Motivated by this, in this paper, we present RGNet, a model that is designed to learn the regional level representation of EEG signal for accurate emotion recognition. Specifically, after applying preprocessing and feature extraction techniques on raw signals, RGNet adopts a novel region-wise encoder to extract the features of channels located within each region as input to compute the regional level features, enabling the model to effectively explore the regional functionality. A graph is then constructed by considering each region as a node and connections between regions as edges, upon which a graph convolutional network is designed with spectral filtering and learned adjacency matrix. Instead of focusing on only the spatial proximity, it allows the model to capture more complex functional relationships. We conducted experiments from the perspective of region division strategies, region encoders and input feature types. Our model has achieved 98.64% and 99.33% for Deap and Dreamer datasets, respectively. The comparison studies show that RGNet outperforms the majority of the existing models for emotion recognition from EEG signals.

Three-dimensional point cloud registration, which aims to find the transformation that best aligns two point clouds, is a widely studied problem in computer vision with a wide spectrum of applications, such as underground mining. Many learning-based approaches have been developed and have demonstrated their effectiveness for point cloud registration. Particularly, attention-based models have achieved outstanding performance due to the extra contextual information captured by attention mechanisms. To avoid the high computation cost brought by attention mechanisms, an encoder–decoder framework is often employed to hierarchically extract the features where the attention module is only applied in the middle. This leads to the compromised effectiveness of the attention module. To tackle this issue, we propose a novel model with the attention layers embedded in both the encoder and decoder stages. In our model, the self-attentional layers are applied in the encoder to consider the relationship between points inside each point cloud, while the decoder utilizes cross-attentional layers to enrich features with contextual information. Extensive experiments conducted on public datasets prove that our model is able to achieve quality results on a registration task.

With the rapid development of military simulation technology, simulation systems for different levels of strategy, campaign, tactics, and technology have emerged. This manuscript intends to use model mapping transformation and machine learning technology to automatically reconstruct the low-resolution simulation model from the high-resolution simulation model data. The premise of learning is to have rich and effective learning data. Therefore, this project uses the existing data cultivation technology/learning data generation technology based on a large sample simulation experiment to generate the simulation data for machine learning through the continuous iterative process of scene design, high-resolution simulation scenario construction, simulation experiment design, and sample generation, efficient parallel simulation, data analysis, and application.

Cruise missile route planning aims to improve the penetration capability and survivability of cruise missiles, and ensure the accuracy of hits, which plays a significant part in ensuring the effective completion of combat missions. The essence of the cruise missile route planning is to determine a flight route under the given constraints, so that the cruise missile can reach the target position safely and meet the condition of satisfying the maneuvering characteristics of the cruise missile to the maximum extent. In this paper, a cruise missile route planning space model is established, and three swarm intelligence algorithms, including grey wolf optimizer (GWO), firefly algorithm and particle swarm optimization (PSO), are applied to the cruise missile route planning problem. To verify the effectiveness and compare the performance of these swarm intelligence algorithms in this problem, relevant simulation experiments is designed and implemented.

In the combat simulation field, how to construct numerous corresponding simulation samples based on the simulation deduction scene, which are used to optimize the combat plan or choose a better one, is an urgent problem to be solved. Current sample generation methods, such as the generative adversarial network, cannot solve the problem of simulation sample generation very well, so a new approach is needed to generate simulation samples. A simulation sample generation technology based on multiple variable points is proposed to solve this problem. This technology can automatically create simulation samples and their corresponding scripts through rule-based means according to a given series of variable points. To verify the effectiveness of the technology, the system effectiveness was compared between the baseline version and the simulation samples generated based on variable points through the AHP, and the combat effectiveness of the generated samples improved by nearly 10

This paper proposes an approach to construct effectiveness evaluation model semi-automatically based on system architecture such as Department of Defense Architecture Framework (DoDAF). The effectiveness evaluation index system is firstly generated based on architecture models including Capability Dependencies (CV-4), Capability to Operational Activities Mapping (CV-6), Operational Activity to Systems Traceability Matrix (SV-5b) and System Measures Matrix (SV-7) in the DoDAF, using the transformation technology QVT. Then the effectiveness calculation model, which is based on analytic hierarchy process (AHP), is semi-automatically generated based on the evaluation tool developed by the authors' team. Based on the effectiveness evaluation index system and the calculation model, the effectiveness of the system described in the architecture can be evaluated in the supporting of the corresponding data, which can be from simulation results or experts. Because the effectiveness evaluation model is constructed based on the system architecture, The proposed approach can ensure that the constructed effectiveness evaluation model including the index system and the calculation model has its source and is objective, distinguishing from experience or emotion. Besides, the proposed approach can improve the efficiency and decrease the mistakes during the process of constructing effectiveness model.

To facilitate system architecture development using DoDAF, we propose an architecture development methodology based on DoDAF 2.0 in this paper. First, based on experiences in practice, gaps between Operational Viewpoint (OV), Capability Viewpoint (CV) and System/Service Viewpoint (SV/SvcV) are analyzed. Second, to bridge the gaps, two architecture viewpoints, i.e. Information activity Viewpoint (IaV) and Capability effect Viewpoint (CeV) proposed in our previous work, are introduced. In addition, a meta-model is provided to define key architecture elements and their relationships in these six viewpoints (i.e. OV, CV, IaV, CeV, SV and SvcV). Third, via analysis of element associations in different viewpoints and models, we give a suggested process of developing system architecture using the six viewpoints and their models, including model development steps, inputs and outputs of each step. Finally, a case study is conducted by applying our methodology to describe architecture of a defense system. Results show that our methodology can effectively facilitate system architecture development and help architects (especially for information system architects) to develop architecture from OV, CV to SV and SvcV.

In the model driven system engineering, architecture plays an important role in the quality of the developed complex system. In some extend, the quality of the architecture determines the developed system's quality. The system's architecture is usually described from different views, including static views and dynamic views, like DoDAF (Department of Defense Architecture). Then how to ensure the consistency of the models from different views becomes a great challenge. This paper proposes a predicate logic based approach for checking the consistency of the system models described by UML models. The case study of the consistency checking of the class diagram and state machine diagram models of a flight control system shows the feasibility and effectiveness of the proposed approach.

The complex information systems like military information systems have many complex characteristics, such as large-scale, complex interacted, dynamical changing of the structure, lots of concurrent events, and running in many different scenarios like jamming and damaging. DEVS and its extensions have been proven to be promising Modeling and Simulation tools for different systems, but the current DEVS extensions cannot perfectly support modeling and simulation of the complex characteristics of the information systems. So an extended DEVS formalism based on the current DEVS extensions is proposed for the purpose in this paper. The algorithms and the M&S environment of the extended DEVS are also proposed. And the case study in this paper shows the feasibility and effectiveness of the research.

To reduce the cost and risk of the development of system of systems (SoS) by pre-evaluation before the SoS is built, a model-driven approach is proposed to evaluate the SoS based on its architecture, especially focused on measures of performance and effectiveness. In order to implement the pre-evaluation, the system architecture needs to be transformed to the simulation model of the system under the directing of the evaluation requirement model, then the evaluation can be done based on the evaluation model and the data from simulation. The architecture, evaluation requirement model, simulation model and evaluation model involved in the above process constitute the model system (a set of models related with each other) of our approach. Through the modeling activities based on the model system, we can implement the systematically pre-evaluation of SoS. The model system and its meta-model, which are the core of our approach, are detailed after the introduction of the framework of our approach. Then the transformations from DoDAF architecture to evaluation requirement model and simulation model are studied for accelerating the evaluation process. The case study shows that our approach can not only ensure the standardization and systematization of the evaluating process, but can improve the evaluating efficiency and creditability based on the proposed transformation method.

In order to verify and evaluate the C4ISR systems before they are built, this paper proposes an approach to make the architecture developed by DoDAF executable. The model transformation technologies in model driven architecture are used to transform the architecture products, such as composite structure diagram, state machine diagram, activity diagram and sequence diagram, to single Simulink models and comprehensive Simulink models. The proposed approach can also effectively reuse the existing simulation blocks in the Simulink library to strengthen the ability of the generated Simulink model from architecture. Through the execution of the generated Simulink models, the data from the simulation can be used for the verificaiton and evaluation of the CRISR architecture. The case study shows the feasibility of the proposed approach.

One effective way to reduce the cost and risk of development of complex systems,such as C4ISR and space maintenance systems,is the simulation and validation of the system architecture.The mapping method from activity model,rule model,dynamic model and data model of the system architecture to Simulink simulation model is proposed,and the transformation between them is automated based on the model transformation technique of the model driven architecture,which can improve the effectiveness of the construction of the simulation model,and can ensure the consistence between architecture and the simulation model.The case study of the simulation and validation of the architecture of the Hubble space telescope servicing system shows that the proposed method can not only validate the consistence of the architecture,but also pre-evaluate the performance of the system.

The execution of web applications always involves many levels (Operating system, Java virtual machine, Web server, Database server, application itself). Each level can produce its corresponding logs, which record the information of the level's running state and history. Based on the multi-level logs generated by the running web systems, this paper proposes a comprehensive method to evaluate and analyze the online web applications' performance. The advantages of this method are that we can not only use it in the testing stage, but also we can discover the online web applications' performance problems not found in the performance testing stage, and can analyze the reasons of the problems effectively. Case study shows that it is an effective method for performance evaluation and analysis.