Deep learning has become a pivotal technique for building digital twin models of physical fields. However, the prohibitive computational and memory demands of these models hinder their deployment on resource-constrained edge platforms for real-time applications. Structured channel pruning offers a direct path to model lightweighting while preserving hardware compatibility. Nevertheless, existing pruning methods fail to preserve spatial gradient fidelity and suffer irreversible accuracy loss due to disjoint pruning and training workflows in physical field regression. To address these challenges, we propose a novel progressive single-stage channel pruning framework grounded in Taylor expansion. This approach employs a regression-aware channel importance criterion based on the first-order Taylor expansion of the regression loss, which explicitly quantifies each channel’s functional sensitivity to local spatial variations in the predicted field. This is coupled with a progressive single-stage pruning paradigm that integrates channel removal directly into the training loop under an arithmetic progression schedule, allowing the network to co-evolve with the pruning mask and smoothly adapt to increasing sparsity without requiring a separate fine-tuning phase. Extensive experiments on digital twin datasets, covering aerial vehicle temperature, pressure, and stress fields, demonstrate that our method not only enhances the accuracy of physical field predictions but also reduces model parameters and storage footprint by over 50% and decreases computational load by more than 20%. This work provides an ultra-low-latency and energy-efficient solution for digital twin computing of physical fields on edge devices, thereby supporting enhanced operational autonomy in aerial vehicles.
Efficient onboard multi-field sparse reconstruction is essential for the autonomous operation of aerospace vehicles. While existing deep learning models exhibit promise for single-field reconstruction, deploying multiple independent models leads to prohibitive model size growth and fails to exploit cross-field correlations, particularly under few-shot conditions. To address these challenges, we first propose a lightweight multi-task Fourier neural operator (MTL-FNO), an end-to-end joint training framework based on hard parameter sharing. In each layer, the parameters are divided into shared and task-specific components to capture common features across fields while preserving task-specific characteristics. Moreover, the task-specific fine-tuning parameters are implemented as low-rank terms, achieving substantial model compression. Second, to address the difficulty of co-optimizing shared and task-specific parameters along with their real and imaginary parts, we revisit the FNO's spectral weight from a polar-form perspective and devise a physically meaningful decoupled optimization scheme. Specifically, we apply polar decomposition to slice-wise disentangle the spectral weight into a unitary tensor encoding phase information and a positive semi-definite tensor characterizing amplitude. By decoupling the optimization of phase and amplitude, our method can effectively mitigate tasks conflict. Meanwhile, to preserve unitary geometric fidelity during training, the Cayley transform is introduced to reparameterize the unitary tensor, converting the constrained optimization problem to an unconstrained one. Finally, the effectiveness of the proposed method under few-shot conditions is validated on two representative engineering cases. Results show that MTL-FNO achieves accuracy comparable to or even surpassing that of standard FNO, while reducing total model size by 76
Reconstructing high-fidelity three-dimensional (3D) physical fields from sparse sensor measurements is an important task in aerospace engineering. However, traditional reconstruction methods, such as 3D interpolation, often suffer from insufficient accuracy. Furthermore, the sparsity of sensor measurements combined with a data-scarce regime increases the risk of overfitting and leads to weak physical consistency in standard deep learning models. To address these challenges, we propose a Gradient-Aware Adaptive Physical Regularization (GA-PR) framework. This framework integrates a 3D deep learning backbone, a composite physical loss function, and a gradient-aware masking strategy. Instead of applying uniform constraints, the proposed method utilizes gradient information to partition the physical domain during training. This mechanism enforces high fidelity in featurerich regions while applying smoothness constraints only to the background, effectively balancing noise reduction and feature preservation. Evaluation on UAV pressure (N = 190) and satellite thermal fields (N = 145) indicates that GA-PR consistently outperforms both interpolation and global regularization baselines. Under extreme sparsity (sampling ratio ≈ 0.06%), the proposed method achieves a 5.4% reduction in Root Mean Square Error (RMSE) compared to the data-driven baseline. More importantly, it demonstrates an improved capability to recover local extrema, such as pressure suction peaks and thermal hotspots, which are critical for engineering safety analysis.
Accurate reconstruction of temperature field of heat-source systems is crucial for thermal monitoring and reliability assessment in engineering applications such as electronic devices and aerospace structures. However, the high cost of measurement acquisition and the substantial distributional shifts in temperature field across varying conditions present significant challenges for developing reconstruction models with robust generalization capabilities. Existing deep neural networks (DNNs)-based methods typically formulate Temperature Field Reconstruction of Heat Source Systems (TFR-HSS) as a one-to-one regression problem based solely on target sparse measurements, without effectively leveraging reference simulation data that implicitly encode thermal knowledge. To address this limitation, this work proposes IPTR, an implicit physics-guided temperature field reconstruction framework that introduces sparse monitoring-temperature field pair from reference simulations as priors to enrich physical understanding. To integrate both reference and target information, a dual physics embedding module is developed, consisting of two complementary branches: an implicit physics-guided branch employing cross-attention to distill latent physics from the reference data, and an auxiliary encoding branch based on Fourier layers to capture the spatial characteristics of the target observation. The fused representation is then decoded to reconstruct the full temperature field. Extensive experiments across benchmark evaluation, cross-scenario generalization, and real-world case study demonstrate that IPTR consistently achieves state-of-the-art performance, outperforming UNet-based and operator-learning methods, including those with explicit physics-informed guidance, reducing average error by approximately 43% across diverse scenarios.
Recent advances in neural portrait animation have demonstrated remarked potential for applications in virtual avatars, telepresence, and digital content creation. However, traditional explicit warping approaches often struggle with accurate motion transfer or recovering missing regions, while recent attention-based warping methods, though effective, frequently suffer from high complexity and weak geometric grounding. To address these issues, we propose SynergyWarpNet, an attention-guided cooperative warping framework designed for high-fidelity talking head synthesis. Given a source portrait, a driving image, and a set of reference images, our model progressively refines the animation in three stages. First, an explicit warping module performs coarse spatial alignment between the source and driving image using 3D dense optical flow. Next, a reference-augmented correction module leverages cross-attention across 3D keypoints and texture features from multiple reference images to semantically complete occluded or distorted regions. Finally, a confidence-guided fusion module integrates the warped outputs with spatially-adaptive fusing, using a learned confidence map to balance structural alignment and visual consistency. Comprehensive evaluations on benchmark datasets demonstrate state-of-the-art performance.
While deep learning is prominent in hyperspectral image analysis, its application to hyperspectral target detection (HTD) faces severe data limitations that only target reference and unlabeled in-scene data are available. The absence of negative labels causes label asymmetry, challenging standard learning paradigms. To address this issue, we analyse the problem formulation of positive-unlabeled (PU) learning for HTD, mining latent targets from unlabeled data. Specifically, applying PU learning to HTD is complicated by combined problems of positive-unlabeled imbalance and target-background imbalance within the unlabeled data, exacerbated by the spectral mixture property of hyperspectral imagery. To bridge this gap, this work proposes a physics-informed dual-imbalance-aware PU learning method for HTD (PU-HTD), which integrates balanced PU learning with physics-constrained subspace representation autoencoder (SRA). First, SRA serves as the backbone, which aims to separate target signals from complex background by spectral signature decoupling. Simultaneously, PU learning is performed for target-background boundary discrimination learning. Second, we propose a dual-imbalance-aware PU loss for optimization stability enhancement and balanced learning, which integrates Taylor-stabilized approximation (TSA) and confidence-aware adaptive perturbation (CAP). Finally, the effectiveness is demonstrated on typically defined detection scenes.
In practical engineering, massive data are commonly utilized for data prediction and system reliability analysis. However, the presence of inevitable data noise can render reliability analysis results inaccurate, thereby necessitating the data uncertainty quantification. Moreover, current studies struggle to ensure both prediction accuracy and uncertainty quality simultaneously, which complicates the uncertainty quantification. To address these problems, this paper proposes a generic quality and accuracy driven uncertainty quantification framework based on deep learning methods. The proposed quality and accuracy driven sampling loss function builds the bridge between the significance level and quantile level, and constrains the mean prediction, upper and lower interval limits, significantly improving prediction accuracy and interval quality. The randomly sampled significance level is regarded as an input feature to derive the prediction interval at an aleatory confidence level, avoiding inaccuracies in reliability analysis. Additionally, the proposed framework is a universal one, applicable to any modeling method, and possesses high engineering practicability. Two numerical examples and one practical engineering case are adopted to verify the effectiveness. Results demonstrate our proposed method achieves higher prediction accuracy and uncertainty quality compared to other methods. This advancement offers credible information essential for interval reliability analysis and system health condition assessment.
Infrared monitoring is critical for developing high-fidelity thermal-field digital twins in aircraft electronic systems, yet its accuracy is severely constrained by low-quality IR(infrared) images with inherent noise and bias. To overcome this limitation, this paper proposes a novel multi-source fusion framework that strategically enhances IR image quality through controlled noise injection and sensor-data integration. The spatial-domain fusion of point-temperature sensor data with raw IR images to correct spatial biases. Then the frequency-domain fusion of diverse artificially noised and original IR images to suppress stochastic noise. Furthermore, sensor-guided adaptive thresholds optimization are proposed to enhance fusion precision. This counter-intuitive noise-augmentation strategy reconstructs high-quality thermal images through multi-information fusion, achieving measurable enhancements in infrared data quality. The fusion method provides a deeper understanding of the underlying physical phenomena, facilitating effective integration of information. Experimental validation on aircraft electronic circuit boards demonstrated reductions in RMSE (more than 23%) and PIQE (more than 1.6%), alongside PSNR gains (more than 38%), confirming markedly improved thermal accuracy, robustness across operational scenarios, and enhanced accuracy, effectiveness and robustness for infrared monitoring. The framework provides a generalizable solution for thermal monitoring with IR images in complex electronic mechanical systems.
Fluid-structure interaction (FSI) in multibody systems, a non-negligible phenomenon in engineering applications, has been extensively studied via traditional experimental and simulation methods with high cost and time consumption. Deep learning has shown promise in improving computing efficiency while ensuring modelling accuracy in FSI analysis. However, its current capabilities are limited when it comes to constructing multi-object coupling systems with dynamic boundaries. In this paper, we propose a novel FSI hybrid neural network solver integrated by an innovative fluid deep learning model and the structural motion equations for the vortex-induced vibration (VIV) modelling of two tandem cylinders. This well-designed solver, in sequence-to-point manner, can precisely predict the subsequent flow field state by coupling the historical multi-time fluid sequences and the current structural responses, moreover, derives the structural state at the next time based on the interaction forces. Therein, the fluid deep learning model consists of a wall shear stress model and an innovative flow field model with U-shaped architecture jointing the Fourier neural operator and modified convolution long-short term memory model. Two models effectively capture coupling interaction forces, and the latter has higher accuracy in modelling instantaneous flow fields compared with baseline Convolutional Neural Networks-based models with similar parameters. Compared to FSI benchmark case, the proposed FSI model demonstrates superior accuracy and robustness in constructing the nonlinear complex multi-vibration systems. And its prediction speed realises an improvement of over 1000 times than that of the numerical simulation. Significantly, the proposed FSI neural model has substantial potential for advancing FSI modelling of flexible structures featuring pronounced nonlinear deformation boundaries.
Deep learning (DL) methodology contributes a lot to the development of hyperspectral image (HSI) analysis community. However, it also makes HSI analysis systems vulnerable to adversarial attacks. To this end, we propose a masked spatial-spectral autoencoder (MSSA) in this article under self-supervised learning theory, for enhancing the robustness of HSI analysis systems. First, a masked sequence attention learning (MSAL) module is conducted to promote the inherent robustness of HSI analysis systems along spectral channel. Then, we develop a graph convolutional network (GCN) with learnable graph structure to establish global pixel-wise combinations. In this way, the attack effect would be dispersed by all the related pixels among each combination, and a better defense performance is achievable in spatial aspect. Finally, to improve the defense transferability and address the problem of limited labeled samples, MSSA employs spectra reconstruction as a pretext task and fits the datasets in a self-supervised manner. Comprehensive experiments over three benchmarks verify the effectiveness of MSSA in comparison with the state-of-the-art hyperspectral classification methods and representative adversarial defense strategies.
Deep neural networks (DNNs) have attained remarkable success in aerial detection tasks, yet they remain susceptible to adversarial samples, posing a significant challenge for their practical applications. While numerous transferable attacks have been proposed, they frequently overlook the essential balance between attack effectiveness and the feasibility of physical implementation. In this article, we concentrate our efforts on adversarial attacks against aerial detection, crafting transferable adversarial patches that can be implemented in the physical world. To this end, we introduce localized pattern corruptions, such as light spots and shadows, around the target during the training phase. These corruptions could pull the image distributions closer to the decision boundaries of the surrogate model, thereby enhancing the transferability of patches. In addition, we avoid directly optimizing adversarial patterns with traditional gradient-based techniques. Instead, we opt to update the weights of a specialized generator, which employs multilayer perceptrons (MLPs) as its core component for mapping purposes. To verify the effectiveness of our method, we conduct experiments across both the digital and physical domains. The results reveal that our approach outperforms state-of-the-art methods in terms of attack performance.
Estimating the full state of physical systems, including thermal and flow status, from sparse measurements of limited sensors is a critical technology for perception and control. Neural networks have been used in recent studies to reconstruct the global field in a supervised learning paradigm. However, these studies encounter two major challenges: the lack of interpretability of black-box models and performance bottleneck caused by network structure and parameter optimization. This paper proposes a hybrid method based on proper orthogonal decomposition (POD) and deep neural networks (DNNs) to further enhance the interpretability and accuracy of flow and heat field reconstruction. The key idea is to leverage the inherent data modes extracted by POD that capture essential features in physical fields, and formulate the reconstruction problem as finding an optimal linear combination of dominant POD modes. To reduce the error introduced by underfitting and model structure, this paper estimates the coefficients of POD modes by establishing and solving a linear optimization problem that minimizes the gap between the recovered field and the exact measurements, rather than employing regression models. However, the underdetermined issue cased by the sparse measurements restricts the optimization problem to obtain a proper solution. To alleviate this problem, this paper presents to utilize the powerful non-linear approximation ability of DNNs to produce a reference field as auxiliary observations, which combines exact measurements to jointly constrain the optimization problem solving. Finally, the global physical field is reconstructed by superposing dominant POD modes weighted with the solved coefficients. By combining with POD technology, the proposed method can also improve the performance of neural networks on reconstruction problems with large-scale and irregular domains. The experiments conducted on the fluid and thermal benchmarks demonstrate that the proposed method can significantly boost neural network reconstruction performance and outperform existing POD-based methods.
Temperature reconstruction is vital for ensuring system reliability in electronic component design. However, current approaches struggle to effectively explore system information and physical relationships, thereby limiting their performance. This paper presents deep learning surrogate models for precise temperature field reconstruction, showcasing their effective discernment of system distribution laws. However, the scarcity of high-quality training data poses a significant challenge, often leading to issues like overfitting and compromised precision. To address this problem, the paper proposes an adaptive multi-source information fusion method (MFIF) for integrating physical information from various data sources in the frequency domain. By leveraging frequency domain analysis, a deeper understanding of underlying physical phenomena is achieved, facilitating effective integration of information. Furthermore, by utilizing deep surrogate models and high-quality training samples, the developed multi-source frequency fusion method enables the creation of a multi-source fusion driven deep learning method for temperature field reconstruction. The proposed method enhances the robustness, accuracy, and effectiveness of aircraft temperature field reconstruction in orbit. Experimental results demonstrate a substantial decrease in both noise and errors, while the Signal-to-Noise Ratio can be improved by up to more than 86%.
Convolutional neural networks (CNNs) have been demonstrated their powerful ability to extract discriminative features for hyperspectral image classification. However, general deep learning methods for CNNs ignore the influence of complex environmental factor which enlarges the intra-class variance and decreases the inter-class variance. This multiplies the difficulty to extract discriminative features. To overcome this problem, this work develops a novel deep intrinsic decomposition with adversarial learning, namely AdverDecom, for hyperspectral image classification to mitigate the negative impact of environmental factors on classification performance. First, we develop a generative network for hyperspectral image (HyperNet) to extract the environmental-related feature and category-related feature from the image. Then, a discriminative network is constructed to distinguish different environmental categories. Finally, a environmental and category joint learning loss is developed for adversarial learning to make the deep model learn discriminative features. Experiments are conducted over three commonly used real-world datasets and the comparison results show the superiority of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu-sw/Adversarial Learning Intrinsic Decomposition for the sake of reproducibility.
Various uncertainties like random input and data noise in aerospace engineering affect flight vehicle safety, making uncertainty quantification important for improving system reliability. Polynomial chaos expansion (PCE) is a versatile method for quantifying stochastic systems’ uncertainty caused by random input, while it cannot quantify data uncertainty caused by noise. To solve this problem, a novel uncertainty quantification method, learnable quantile PCE (LQPCE), is proposed for interval reliability analysis. Unlike the traditional PCE, the multi-dimensional orthonormal basis is constructed for combination input rather than only random variables, and the expansion coefficients are parameterized to learnable weights, ensuring the LQPCE model can capture data noise’s features. The LQPCE Pinball loss function is constructed to learn parameterized expansion coefficients by quantile level iterative sampling, achieving accurate prediction and data uncertainty quantification. The quantified data uncertainty is used for interval reliability analysis, calculating more reliable results about the influence of input uncertainty on stochastic systems’ reliability. Three numerical examples and one engineering application validate the LQPCE method’s effectiveness. The results show that the LQPCE method can quantify data uncertainty accurately and obtain credible interval reliability.
Few-shot segmentation is a task that aims to generalize well to segment novel categories in images leveraging only a few annotated samples. Most existing methods adopt the prototype learning architecture, where support prototype vectors are expanded and concatenated with query features to perform conditional segmentation. However, such framework may potentially focus more on query features and neglect the similarity between support and query features. Moreover, it is valuable to mine information and latent categories in the background region to improve method generalization. To address these issues, this paper proposes a contrastive enhancement approach that can mine latent prototypes from background regions and leverage latent classes to raise the utilization of similarity information between prototype and query features. Specifically, a latent prototype sampling module is proposed to generate pseudo-mask and novel prototypes from the region with same category. It is based on feature similarity in high-level features, which is cost-effective for conducting end-to-end learning. This module does not require prior knowledge of object categories and has no strong dependence on clustering numbers like cluster-based method. Besides, a contrastive enhancement module is developed to drive models to provide different predictions with the same query features. It introduces the latent prototypes to regularize the decoder to concentrate more on similarity between support and query features. The proposed modules can be used as auxiliary modules to flexibly integrate into other baselines for better segmentation performance. Extensive experiments show our approach remarkably improves the performance of state-of-the-art methods for 1-shot and 5-shot segmentation, especially outperforming baseline by 5.9% and 7.3% for 5-shot task on Pascal-5i and COCO-20i.
The dissection of hyperspectral images into intrinsic components through hyperspectral intrinsic image decomposition (HIID) enhances the interpretability of hyperspectral data, providing a foundation for more accurate classification outcomes. However, the classification performance of HIID is constrained by the model's representational ability. To address this limitation, this study rethinks hyperspectral intrinsic image decomposition for classification tasks by introducing deep feature embedding. The proposed framework, HyperDID, incorporates the Environmental Feature Module (EFM) and Categorical Feature Module (CFM) to extract intrinsic features. Additionally, a Feature Discrimination Module (FDM) is introduced to separate environment-related and category-related features. Experimental results across three commonly used datasets validate the effectiveness of HyperDID in improving hyperspectral image classification performance. This novel approach holds promise for advancing the capabilities of hyperspectral image analysis by leveraging deep feature embedding principles. The implementation of the proposed method could be accessed soon at https://github.com/shendu-sw/HyperDID for the sake of reproducibility.
Due to the powerful ability in capturing the global information, Transformer has become an alternative architecture of CNNs for hyperspectral image classification. However, general Transformer mainly considers the global spectral information while ignores the multiscale spatial information of the hyperspectral image. In this paper, we propose a multiscale spectral-spatial convolutional Transformer (MultiscaleFormer) for hyperspectral image classification. First, the developed method utilizes multiscale spatial patches as tokens to formulate the spatial Transformer and generates multiscale spatial representation of each band in each pixel. Second, the spatial representation of all the bands in a given pixel are utilized as tokens to formulate the spectral Transformer and generate the multiscale spectral-spatial representation of each pixel. Besides, a modified spectral-spatial CAF module is constructed in the MultiFormer to fuse cross-layer spectral and spatial information. Therefore, the proposed MultiFormer can capture the multiscale spectral-spatial information and provide better performance than most of other architectures for hyperspectral image classification. Experiments are conducted over commonly used real-world datasets and the comparison results show the superiority of the proposed method.
Perception of the full state is an essential technology to support the monitoring, analysis, and design of physical systems, one of whose challenges is to recover global field from sparse observations. Well-known for brilliant approximation ability, deep neural networks have been attractive to data-driven flow and heat field reconstruction studies. However, limited by network structure, existing researches mostly learn the reconstruction mapping in finite-dimensional space and has poor transferability to variable resolution of outputs. In this paper, we extend the new paradigm of neural operator and propose an end-to-end physical field reconstruction method with both excellent performance and mesh transferability named RecFNO. The proposed method aims to learn the mapping from sparse observations to flow and heat field in infinite-dimensional space, contributing to a more powerful nonlinear fitting capacity and resolution-invariant characteristic. Firstly, according to different usage scenarios, we develop three types of embeddings to model the sparse observation inputs: MLP, mask, and Voronoi embedding. The MLP embedding is propitious to more sparse input, while the others benefit from spatial information preservation and perform better with the increase of observation data. Then, we adopt stacked Fourier layers to reconstruct physical field in Fourier space that regularizes the overall recovered field by Fourier modes superposition. Benefiting from the operator in infinite-dimensional space, the proposed method obtains remarkable accuracy and better resolution transferability among meshes. The experiments conducted on fluid mechanics and thermology problems show that the proposed method outperforms existing POD-based and CNN-based methods in most cases and has the capacity to achieve zero-shot super-resolution.