The success of modern machine learning hinges on access to high-quality training data. In many real-world scenarios, such as acquiring data from public repositories or sharing across institutions, data is naturally organized into discrete datasets that vary in relevance, quality, and utility. Selecting which repositories or institutions to search for useful datasets, and which datasets to incorporate into model training are therefore critical decisions, yet most existing methods select individual samples and treat all data as equally relevant, ignoring differences between datasets and their sources. In this work, we formalize the task of dataset selection: selecting entire datasets from a large, heterogeneous pool to improve downstream performance under resource constraints. We propose Dataset Selection via Hierarchies (DaSH), a dataset selection method that models utility at both dataset and group (e.g., collections, institutions) levels, enabling efficient generalization from limited observations. Across two public benchmarks (Digit-Five and DomainNet), DaSH outperforms state-of-the-art data selection baselines by up to 26.2% in accuracy, while requiring significantly fewer exploration steps. Ablations show DaSH is robust to low-resource settings and lack of relevant datasets, making it suitable for scalable and adaptive dataset selection in practical multi-source learning workflows.

In many supervised learning applications, the response consists of both continuous and binary outcomes. Studies have shown that jointly modeling such mixed-type responses can substantially improve predictive performance compared to separate analyses. But outliers pose a new challenge to the existing likelihood-based modeling approaches. In this paper, we propose a new robust joint modeling framework for data with both continuous and binary responses. It is based on the density power divergence (DPD) loss function with the ℓ_1 regularization. The proposed framework leads to a sparse estimator that simultaneously predicts continuous and binary responses in high-dimensional input settings while down-weighting influential outliers and mislabeled samples. We also develop an efficient proximal gradient algorithm with Barzilai-Borwein spectral step size and a robust information criterion (RIC) for data-driven selection of the penalty parameters. Extensive simulation studies under a variety of contamination schemes demonstrate that the proposed method achieves lower prediction error and more accurate parameter estimation than several competing approaches. A real case study on wafer lapping in semiconductor manufacturing further illustrates the practical gains in predictive accuracy, robustness, and interpretability of the proposed framework.

The manufacturing industrial internet (MII) is transforming traditional factories into data-driven environments. However, the dynamically varying contexts of the MII, caused by adjustment of process parameters, equipment degradation, and customized specifications, challenge the deployed machine learning models used in process modeling, variation analysis, and anomaly detection. To address this, we propose a novel approach for robust machine learning pipeline selection and adaptation in varying industrial contexts. We introduce a weighted ensemble mechanism based on Bayesian latent space model recommender systems, optimizing sparse ensemble weights across pipelines while incorporating uncertainty quantification. This enables data-driven decision making under uncertainty by automatically selecting and adapting optimal pipelines, reducing manual intervention and improving computational efficiency. We validate our methodology using real-world data from two manufacturing processes (fused deposition modeling and aerosol jet printing) and one chemometric data set (from Tecator). Results demonstrate that our approach achieves superior and more robust performance across data sets compared with traditional single-pipeline recommenders, highlighting the importance of uncertainty quantification in improving pipeline selection accuracy and robustness. History: Bianca Maria Colosimo served as the senior editor for this article. Funding: This work was supported by the National Science Foundation [Awards DMS-2413701, DMS-2124535, CMMI-2331985, and Grant CMMI-2430998] and the American Heart Association Collaborative Science [Award 23CSA1052735]. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://codeocean.com/capsule/4351450/tree/v1 and in the Supplemental Material to this article (available at https://doi.org/10.1287/ijds.2024.0052 ).

The advancement of Artificial Intelligence (AI) models heavily relies on large high-quality datasets. However, in advanced manufacturing, collecting such data is time-consuming and labor-intensive for a single enterprise. Hence, it is important to establish a context-aware and privacy-preserving data sharing system to share small-but-high-quality datasets between trusted stakeholders. Existing data sharing approaches have explored privacy-preserving data distillation methods and focused on valuating individual samples tied to a specific AI model, limiting their flexibility across data modalities, AI tasks, and dataset ownership. In this work, we propose a performance-oriented representation learning (PORL) framework in a Directed Graph Neural Network (DiGNN). PORL distills raw datasets into privacy-preserving proxy datasets for sharing and learns compact meta data representations for each stakeholder locally. The meta data will then be used in DiGNN to forecast the AI model performance and guide the sharing via graph-level supervised learning. The effectiveness of the PORL-DiGNN is validated by two case studies: data sharing in the semiconducting manufacturing network between similar processes to create similar quality defect models; and data sharing in the design and manufacturing network of Microbial Fuel Cell anodes between upstream (design) and downstream (Additive Manufacturing) stages to create distinct but related AI models. Note to Practitioners-To accelerate AI adoption in advanced manufacturing, there is an urgent need for data sharing among participants to prepare high-quality datasets and improve AI model performance. This work proposes a dataset-sharing framework that lays the foundation for future data exchange and trade. Current approaches lack the flexibility to support sharing across diverse context and may expose the value of the data prematurely. To address these challenges, we introduce a performance-oriented representation learning framework that generates data for sharing and valuation to secure the value and preserve the private information, and then utilizes graph-based supervised learning to guide the sharing decisions for data receivers. The framework's effectiveness and generalizability are demonstrated through two real-world manufacturing dataset-sharing case studies. Industrial participants can use this framework to rank datasets from others based on their predicted utility for specific downstream AI tasks.

Knowledge distillation (KD) and model averaging (MA) are prominent techniques for enhancing the efficiency and effectiveness of deep neural networks (DNNs). MA produces an average model by averaging multiple checkpoints along the trajectory of the stochastic gradient descent (SGD), which tends to converge toward the flatter side of the local minimum (Fig. 2). However, MA operates offline and does not influence the specific local minimum to which the network converges. By contrast, KD transfers knowledge from a teacher model to a student model, guiding the student toward a better local minimum. Despite this, KD can still result in a poor convergence location within the minimum if the teacher over/under-regularizes the student model's behavior. By combining KD and MA, we aim to leverage KD for a favorable local minimum and MA for a robust convergence location within the local minimum. However, our results revealed that these two techniques were incompatible. This study empirically analyzed the causes of this incompatibility and proposed an annealing knowledge distillation (AKD) scheme to address it. Building on MA and AKD, we introduced a general self-distillation framework referred to as self-distillation with model averaging (SDMA). To the best of our knowledge, SDMA is the first general distillation framework that aims to achieve a favorable local minimum and a robust convergence location. Extensive experiments demonstrated that SDMA significantly improved the generalization performance of DNNs.

Operational hazards in Manufacturing Industrial Internet (MII) systems generate severe data outliers that cripple traditional statistical analysis. This paper proposes a novel robust regression method, DPD-Lasso, which integrates Density Power Divergence with Lasso regularization to analyze contaminated data from AI resilience experiments. We develop an efficient iterative algorithm to overcome previous computational bottlenecks. Applied to an MII testbed for Aerosol Jet Printing, DPD-Lasso provides reliable, stable performance on both clean and outlier-contaminated data, accurately quantifying hazard impacts. This work establishes robust regression as an essential tool for developing and validating resilient industrial AI systems.

Vision-language pre-training (VLP) methods have significantly advanced cross-modal tasks in recent years. However, image-text retrieval still faces two critical challenges: inter-modal matching deficiency and intra-modal fine-grained localization deficiency. These issues significantly impede the accuracy of image-text retrieval. To address these challenges, we propose a novel dual-stage training framework. In the first stage, we employ Soft Label Distillation (SLD) to align the contrastive relationships between images and texts by mitigating the overfitting problem caused by hard labels. In the second stage, we introduce Spatial Text Prompt (STP) to enhance the model's visual grounding capabilities by incorporating spatial prompt information, thereby achieving more precise fine-grained alignment. Extensive experiments on standard datasets show that our method outperforms state-of-the-art approaches in image-text retrieval.The code and supplementary files can be found at https://github.com/Leon001211/DSSLP.

In robotic operations, heterogeneous computation tasks and sensor configurations pose significant challenges to analyze different modalities of data for data sharing and collaborative learning in robotic Artificial Intelligence (AI) tasks. The lack of historical data in new scenarios or new computation tasks complicates model training and limits the applicability of existing AI methodologies. Current transfer learning approaches heavily rely on static feature extraction, which fail to dynamically adjust to specific feature relationships between different samples or modalities. In the literature, these methods struggle to capture inter-modal associations effectively, resulting in insufficient information sharing and poor modeling performance. Motivated by these challenges, this paper proposes a Dynamic Cross-Attention Feature Fusion (DCAF) approach to map the features from one robotic AI task to another. By calculating attention weights tailored to each target domain sample, DCAF extracts the most relevant source domain features and generates dynamic fused representations. The proposed approach enables sample-specific feature selection and fine-grained domain alignment, effectively enhancing the modeling performance compared with traditional transfer learning and model training based on the local data source. It is particularly suited for a new robotic AI training task with limited sample size and new data modalities. Experimental results for feature fusion from a robotics anomaly detection dataset to a position accuracy modeling data set demonstrate the effectiveness of DCAF, providing an efficient solution for domain adaptation and multimodal fusion.

In Manufacturing Industrial Internet, Deep Neural Networks (DNNs) are widely used to improve production efficiency and quality via computation services. The resilience of DNNs in inference tasks is easily affected by data quality, computation pipeline, and cyber-physical layer of manufacturing. To create a proactive-adaptive resilient Artificial Intelligence (AI) models, it is important to predict the AI model performance based on online streaming data, such as runtime metrics of computation tasks. The class imbalance (i.e., a small percentage of AI performance degradation under hazardous conditions) and time-varying distribution are commonly encountered, which makes the AI model performance prediction difficult. We propose a contrastive ensemble active learning (ConEAL) framework that couples a contrastive representation learner with a contextualbandit ensemble active-learning policy proposed in our earlier work. The representation learning module standardizes predictors and creates stochastic views for each sample to learn more stable representation space. By employing a staged loss function to train the encoder, the representation space adapts to the data stream as new classes emerge. Based on the projected representation, ConEAL ensembles different sampling agentsand adaptively weights them with a composite reward in order to acquire informative samples to reduce the annotation efforts. Validation is performed based on an AI resilience experiment in the Fog-Cloud Manufacturing Industrial Internet testbed, where ConEAL achieves better prediction performance of abnormal AI model performance than benchmark methods while requiring fewer labeled samples.

Artificial intelligence (AI) systems have been increasingly adopted in the Manufacturing Industrial Internet (MII). Investigating and enabling the AI resilience is very important to alleviate profound impact of AI system failures in manufacturing and Industrial Internet of Things (IIoT) operations, leading to critical decision making. However, there is a wide knowledge gap in defining the resilience of AI systems and analyzing potential root causes and corresponding mitigation strategies. In this work, we propose a novel framework for investigating the resilience of AI performance over time under hazard factors in data quality, AI pipelines, and the cyber-physical layer. The proposed method can facilitate effective diagnosis and mitigation strategies to recover AI performance based on a multimodal multi-head self latent attention model. The merits of the proposed method are elaborated using an MII testbed of connected Aerosol Jet Printing (AJP) machines, fog nodes, and Cloud with inference tasks via AI pipelines.

As image–text matching (a critical task in the field of computer vision) links cross-modal data, it has captured extensive attention. Most of the existing methods intended for matching images and texts explore the local similarity levels between images and sentences to align images with texts. Even though this fine-grained approach has remarkable gains, how to further mine the deep semantics between data pairs and focus on the essential semantics in data remains to be quested. In this work, a new semantic filtering and adaptive approach (FAAR) was proposed to ease the above problem. To be specific, the filtered attention (FA) module selectively focuses on typical alignments with the interference of meaningless comparisons eliminated. Next, the adaptive regulator (AR) further adjusts the attention weights of key segments for filtered regions and words. The superiority of our proposed method was validated by a number of qualitative experiments and analyses on the Flickr30K and MSCOCO data sets.

Digital twin (DT) models are computational models that can effectively represent different assets and processes in the manufacturing environment. Moreover, the DT models can support intelligent automation by integrating with the digital foundation and the data analytics provided by the cyber-physical system (CPS) in an industrial environment. To properly model a physical process, a DT model should be updated online to closely and timely model the underlying process and reduce modeling uncertainty in the CPS. However, most DT models are created offline and implemented online, which cannot be easily updated by using online data from heterogeneous product designs or manufacturing processes. This limitation arises from existing online learning methods, which are typically designed for identical structures, while real manufacturing CPS involves personalized designs and diverse processes. More importantly, there are limited samples for the same product design or manufacturing process due to manufacturing personalization, which slows down the online updating of DT models. In this article, the authors investigated online DT model updating based on data collected from different product designs and/or processes. The authors proposed a mutual active learning framework to identify informative samples from different designs or processes for online DT model updating. Specifically, by properly balancing the gradient-based features of the DT models and the similarity among these heterogeneous designs or processes, the proposed method can effectively query the most informative samples among heterogeneous processes to update the corresponding DT model in a timely manner. The advantages of the proposed method are illustrated by an engineering-driven statistical DT model for an additive manufacturing process (i.e., fused deposition modeling).

Knowledge Distillation (KD) enhances the generalization ability of a student model by transferring knowledge from a teacher model. However, literature suggests that the student may struggle to match the teacher’s predictions perfectly, even with sufficient capacity. A distillation scheme imposing strong constraints on the student may lead to over-regularization and performance degradation. On the other hand, a scheme with low supervision intensity can alleviate over-regularization but may lead to the student acquiring less potentially beneficial knowledge. Determining the optimal distillation scheme often requires significant effort and experimentation. Furthermore, the fact that the optimal distillation strategy may vary at different stages of the distillation process is also often overlooked.In this work, we introduce PRKD (Progressively Relaxed Knowledge Distillation), an advanced distillation framework that dynamically adjusts the supervision intensity from the teacher throughout the training process. Specifically, PRKD is built upon a gradual transition of the distillation scheme, progressing iteratively from a strong to a weak configuration. Through this transition, the student preserves essential knowledge from the teacher while simultaneously alleviating the potential for excessive regularization. Our comprehensive experiments on diverse datasets demonstrate the efficacy of PRKD, consistently surpassing other state-of-the-art methods.

An Industrial Cyber-physical System (ICPS) provides a digital foundation for data-driven decision-making by artificial intelligence (AI) models. However, the poor data quality (e.g., inconsistent distribution, imbalanced classes) of high-speed, large-volume data streams poses significant challenges to the online deployment of offline-trained AI models. As an alternative, updating AI models online based on streaming data enables continuous improvement and resilient modeling performance. However, for a supervised learning model (i.e., a base learner), it is labor-intensive to annotate all streaming samples to update the model. Hence, a data acquisition method is needed to select the data for annotation to ensure data quality while saving annotation efforts. In the literature, active learning methods have been proposed to acquire informative samples. Different acquisition criteria were developed for exploration of under-represented regions in the input variable space or exploitation of the well-represented regions for optimal estimation of base learners. However, it remains a challenge to balance the exploration-exploitation trade-off under different online annotation scenarios. On the other hand, an acquisition criterion learned by AI adapts itself to a scenario dynamically, but the ambiguous consideration of the trade-off limits its performance in frequently changing manufacturing contexts. To overcome these limitations, we propose an ensemble active learning method by contextual bandits ( CbeAL ). CbeAL incorporates a set of active learning agents (i.e., acquisition criteria) explicitly designed for exploration or exploitation by a weighted combination of their acquisition decisions. The weight of each agent will be dynamically adjusted based on the usefulness of its decisions to improve the performance of the base learner. With adaptive and explicit consideration of both objectives, CbeAL efficiently guides the data acquisition process by selecting informative samples to reduce the human annotation efforts. Furthermore, we characterize the exploration and exploitation capability of the proposed agents theoretically. The evaluation results in a numerical simulation study and a real case study demonstrates the effectiveness and efficiency of CbeAL in manufacturing process modeling of the ICPS.

Non-intrusive load identification plays a crucial role in developing a green and low-carbon energy supply and demand mechanism. Among the various load identification technologies, the low-frequency electrical signal load identification stands out due to its ability to discern user’s electricity consumption habits without increasing the user’s hardware cost, thus presenting promising application prospects. However, the challenge lies in the scarcity of available data and the uneven distribution of samples, resulting in reduced accuracy. In this paper, we present a non-intrusive load state identification method based on the combination of LSTM model and multi-head attention model. By fusing those two techniques, the model can extract and utilize richer and more essential intrinsic features in electrical signals. The use of focal loss can balance the weight importance of different states with significantly different sample numbers. The experimental results on two open data-set show that the proposed algorithm can considerably improve the identification accuracy compared to other NILM methods, and the method also has good practical value in real application.

The recommendation system has been widely used in life, which greatly facilitates people's life. The traditional recommendation method is mainly used to analyze the interaction between users and items. analyze the history of users and items, and get only the users' preferences for items in the past. The serialization recommendation system analyzes the sequence of users interacting with objects in a recent period of time. To consider the relevance of the user's before and after behavior, can obtain the user's preference for items in the short term. However, the serialization method emphasizes the user's connection with the item in the short term. Ignoring the relationship between the properties of objects. In view of the above problems, the convolution serialization recommendation of fusion time characteristics and user preferences is proposed Convolutional Embedding Recommendation with Time and User Preference. CERTU model. The model is able to analyze the diversity relationships between items, thus capturing the user's dynamic preference for items over time. Otherwise, the model further considers the influence of a single item and multiple items present in the item sequence on the next item recommendation. The experimental results show that the No. The CERTU model outperforms the current baseline method.

Artificial intelligence (AI) has played an important role for data-driven decision making in complex engineering problems. However, there has been a huge waste of efforts to configure AI methods (e.g., to select preprocessing and modeling methods, etc.), catering to different contexts (e.g., data analytics objectives, data distributions, etc.). In current practice, data scientists need to manually configure the AI methods in trial-and-errors according to a specific context, including determining the different options of the pipeline components and evaluating the advantages and limitations of an AI method. In this article, we propose a local low-rank response imputation (Lori) method, which will automatically configure AI methods to specific contexts by completing a sparse context-pipeline response matrix. Different from the traditional recommendation systems, Lori performs multivariate partition of the entire context-pipeline response matrix based on the principal Hessian directions of the low-rank imputed response matrix. Thus, the partitioned local low-rank response matrices can be closely modeled to automatically match the AI methods with the datasets. A small-scale and a large-scale case studies in three manufacturing processes demonstrated the merits of the proposed Lori method.

Fog Manufacturing applies both Fog and Cloud Computing collaboratively in Smart Manufacturing to create an interconnected network through sensing, actuation, and computation nodes. Fog Manufacturing has become a promising research component to be integrated into the existing Smart Manufacturing paradigm and provides reliable and responsive computation services. However, Fog nodes' relatively limited communication bandwidth and computation capabilities call for reduced data communication load and computation time latency for modeling. There has long been a lack of an integrated framework to automatically reduce manufacturing data and perform computationally efficient modeling/machine learning. This research direction is increasingly important as both the computational demands and Fog/networked Manufacturing become prevalent. This paper proposes an integrated and distributed framework for data reduction and modeling of multiple systems in a Smart Manufacturing network considering the system similarities. A simulation study and a Fog Manufacturing testbed for ingot growth manufacturing validated that the proposed framework significantly reduces the sample size used for improved computational runtime metrics while outperforming various other data reduction methods in modeling performance.