Laser powder bed fusion (LPBF) is a promising additive manufacturing technique that suffers from quality assurance concerns. Predicting melt pools from process parameters is crucial for assessing quality prior to manufacturing but remains a difficult problem because of the complex physical processes underlying LPBF. Quantum computers present a new computing paradigm, providing a new approach to information processing using quantum entanglement and superposition. This paper presents a practical demonstration of a hybrid quantum-classical model that leverages quantum computing to improve process parameter feature extraction with a quantum feature encoder. To make the quantum approach computationally feasible for large datasets, we first employ a clustering algorithm to reduce the number of expensive quantum computations. These quantum features are then processed by a classical neural network to predict the melt pool morphology, allowing for more accurate predictions of melt pools. We demonstrate the method using a quantum simulator, analyze the effect of measurement shot noise on the predictive performance of the network, and verify the results using quantum hardware. Finally, by examining which quantum features are most important, we provide insights that can inform the future design of more effective quantum encoding circuits. Ultimately, the performance improvement over purely classical networks validates the hybrid approach, demonstrating an engineering application of quantum computing using noisy and intermediate scale quantum (NISQ) devices.

Truly autonomous mobile robots must detect both known and unknown objects. This article proposes a fast, real-time open set object detector (OSOD) that enables mobile robots to identify both known and unknown objects. By leveraging a small YOLO model with pseudolabels provided by a multimodal large language model (LLM), we develop an effective open set object detector for edge devices. We replace the traditional human-in-the-loop for interpreting novel objects with a multimodal LLM, which automatically provides semantic information (name and properties) from images, automating the information learning process. Once a mobile robot acquires the semantics of an unknown object from an LLM, a vision-language model classifies repeated instances of the object, reducing the number of slow LLM queries. Meanwhile, an object's properties provided by an LLM allow a mobile robot to operate more effectively around the new object. To facilitate incremental learning, images and labels of novel objects are stored as they are encountered. Once a sufficient number of instances of a novel object are compiled, the original object detector is retrained to include the new object in the set of known objects. Demonstrated with real mobile robots in an academic office building, the incremental learning method showcases how mobile robots can learn to detect novel objects without human intervention. Source code is available at the URL provided in Note 2.

This paper describes the development and validation of a hybrid electromagnetic energy harvester capable of simultaneously scavenging energy from two different energy sources: low-speed natural wind and traffic-induced bridge vibrations. The harvester employs a cantilever structure designed to exhibit significant vibrations due to aeroelastic instability even at low wind speeds and dynamic excitation from traffic-induced bridge vibrations through an electromagnetic transduction mechanism. It is noted from the laboratory tests using a shaker and wind tunnel that the electrical load resistance significantly influences vibration amplitude, generated power, and the onset velocity of galloping in the harvester. Wind tunnel test results also reveal the necessity of adjusting load resistance according to wind speed for optimal power output. This study provides approximate analytical solutions for the optimal external load resistance and corresponding maximum power under wind and base excitation separately. These solutions are validated through comparisons with numerical simulations and wind tunnel tests. Additionally, under hybrid excitations, particularly at low wind speeds and accelerations, a notable enhancement in average power output is observed. Combining wind with vibration-only or base excitation with galloping-only harvesting significantly improves power output, ensuring enhanced energy availability even if one source is interrupted. Field tests on the bridge confirm the effectiveness of the present harvester in generating electricity from both wind-induced galloping and bridge vibrations, demonstrating its capability to harness energy from two complementary sources. Overall, this hybrid harvester presents a promising solution for maximizing energy capture from ambient natural sources, contributing to advancements in sustainable energy generation.

Monitoring human activity in indoor environments is important for applications such as facility management, safety assessment, and space utilization analysis. While mobile robot teams offer the potential to actively improve observation quality, existing multi-robot monitoring and active perception approaches typically rely on coverage or visitation based objectives that are weakly aligned with the accuracy requirements of human-centric monitoring tasks. In this work, we formulate cooperative active observation as a decentralized control problem in which multiple robots adjust their motion to directly optimize monitoring accuracy under partial observability. We propose a learning-based framework for cooperative policies from decentralized observations using multi-agent reinforcement learning (MARL), supported by an architecture that handles variable numbers of humans and temporal dependencies. Simulation results across diverse indoor environments and monitoring tasks show that the proposed approach consistently outperforms classical coverage, persistent monitoring, and learning-free multi-robot baselines, while remaining robust to changes in the number of observed humans.

Although most of the research on large language models (LLMs) focuses on their development and validation against datasets, significant gaps remain in their application to real-world knowledge-intensive tasks. This research addresses key challenges in using LLMs for extracting and synthesizing knowledge from unstructured sources, focusing on applications where the validity and consistency of the results are critical. We propose Ontology-based Adaptive Knowledge System (OAKS) as a holistic approach to manage the complexities of acquiring unstructured knowledge, varying user expertise, and dynamic query formulation. This research provides practical value for enabling the domain user community to leverage their technical documentation and expertise and accelerate ongoing working projects through improved literature review and cross-disciplinary insight discovery. Validated through empirical studies, our findings offer insight into best practices for the deployment of OAKS, bridging the gaps between AI capabilities and real-world needs in knowledge acquisition and research.

Predicting crowd flow patterns in a physical space can be useful for infrastructure management and safety planning. A simple representation of individuals in Euclidean space is insufficient for representing people's spatial distribution and movements over time. This paper describes a spatiotemporal graph formulation, namely crowd mobility graphs (CMGraphs), to represent the spatiotemporal data. The CMGraphs model employs dynamic node features that store temporal crowd flow information, while the time-invariant edges represent spatial connectivity of locations of interests in the surrounding space. The spatiotemporal formulation using the CMGraphs allows for crowd flow prediction. Specifically, graph neural network is used to aggregate neighborhood nodal information on CMGraphs to capture spatial connectivity. Subsequently, recurrent neural network is employed to generate future sequences of crowd flow. An experiment is conducted using a publicly available video dataset at a train station to demonstrate the effectiveness of the proposed CMGraph formulation for crowd flow forecasting.

Laser powder bed fusion (LPBF) is a popular additive manufacturing process with many advantages compared with traditional (subtractive) manufacturing. However, ensuring the quality of LPBF parts remains a challenge in the manufacturing industry. This work proposes the use of unsupervised learning, specifically, the k-means clustering method, to identify unique melt pool shapes produced during LPBF manufacturing. Melt pools are a key process signature in LPBF and can assist in the evaluation of process quality. k-means is employed multiple times sequentially to produce clusters of melt pools, and the silhouette value is used to identify the optimal number of clusters. The clusters produced by k-means are used as labels to train a deep neural network to classify the melt pool shapes. By inputting the melt pool image and the corresponding LPBF machine process parameters into the neural network, the neural network identifies the melt pool shape to aid human analysis and provide insight into part quality. The trained neural network is interpreted using explainable artificial intelligence (XAI) methods to investigate the relationships between process parameters and the melt pool shape. Using layer-wise relevance propagation, the process parameters that most significantly influence the melt pool shapes are identified. The relationship between process parameters and melt pool shapes can be useful for selecting the process parameters to produce the desired melt pool shapes. In summary, this study describes an approach that combines unsupervised machine learning and XAI methods to effectively enable the analysis and interpretation of melt pools.

The interrupting swap-allowed blocking job shop problem (ISBJSSP) is a complex scheduling problem that is able to model many manufacturing planning and logistics applications realistically by addressing both the lack of storage capacity and unforeseen production interruptions. Subjected to random disruptions due to machine malfunction or maintenance, industry production settings often choose to adopt dispatching rules to enable adaptive, real-time re-scheduling, rather than traditional methods that require costly re-computation on the new configuration every time the problem condition changes dynamically. To generate dispatching rules for the ISBJSSP problem, we introduce a dynamic disjunctive graph formulation characterized by nodes and edges subjected to continuous deletions and additions. This formulation enables the training of an adaptive scheduler utilizing graph neural networks and reinforcement learning. Furthermore, a simulator is developed to simulate interruption, swapping, and blocking in the ISBJSSP setting. Employing a set of reported benchmark instances, we conduct a detailed experimental study on ISBJSSP instances with a range of machine shutdown probabilities to show that the scheduling policies generated can outperform or are at least as competitive as existing dispatching rules with predetermined priority. This study shows that the ISBJSSP, which requires real-time adaptive solutions, can be scheduled efficiently with the proposed method when production interruptions occur with random machine shutdowns.

Laser powder bed fusion (LPBF) has shown enormous potential for metal additive manufacturing in recent years. However, the relationship between the LPBF process parameters and part quality is not yet fully understood. Some LPBF machines now use cameras to monitor melt pools during manufacturing. Machine learning techniques have been proposed to analyze the melt pool data and to evaluate the quality of the manufacturing process. However, these machine learning techniques often appear as a black box and the underlying decisions made by the machine learning models are unknown. This paper proposes a neural network to classify the melt pool shapes using melt pool images and process parameters as model inputs. With both process parameters and the melt pool image being included, an explainable artificial intelligence (XAI) approach is developed to interpret the neural network and understand the relationships between the melt pool shape and the process parameters. Specifically, layer-wise relevance propagation (LRP) is used to reveal the relevance of process parameters in the neural network’s decision-making. Using LRP, relationships between the process parameters and melt pool shapes are revealed without explicit knowledge of the underlying physics. These relationships can potentially be used to adjust the process parameters and improve the quality of LPBF manufactured parts. This paper demonstrates how neural networks and XAI can effectively identify relationships between process parameters and LPBF melt pools.

Crowd congestion is one of the main causes of modern public safety issues such as stampedes. Conventional crowd congestion monitoring using closed-circuit television (CCTV) video surveillance relies on manual observation, which is tedious and often error-prone in public urban spaces where crowds are dense, and occlusions are prominent. With the aim of managing crowded spaces safely, this study proposes a framework that combines spatial and temporal information to automatically map the trajectories of individual occupants, as well as to assist in real-time congestion monitoring and prediction. Through exploiting both features from CCTV footage and spatial information of the public space, the framework fuses raw CCTV video and floor plan information to create visual aids for crowd monitoring, as well as a sequence of crowd mobility graphs (CMGraphs) to store spatiotemporal features. This framework uses deep learning-based computer vision models, geometric transformations, and Kalman filter-based tracking algorithms to automate the retrieval of crowd congestion data, specifically the spatiotemporal distribution of individuals and the overall crowd flow. The resulting collective crowd movement data is then stored in the CMGraphs, which are designed to facilitate congestion forecasting at key exit/entry regions. We demonstrate our framework on two video data, one public from a train station dataset and the other recorded at a stadium following a crowded football game. Using both qualitative and quantitative insights from the experiments, we demonstrate that the suggested framework can be useful to help assist urban planners and infrastructure operators with the management of congestion hazards.

In high seismic risk regions, it is important for city managers and decision makers to create programs to mitigate the risk for buildings. For large cities and regions, a mitigation program relies on accurate information of building stocks, that is, a database of all buildings in the area and their potential structural defects, making them vulnerable to strong ground shaking. Structural defects and vulnerabilities could manifest via the building's appearance. One such example is the soft‐story building—its vertical irregularity is often observable from the facade. This structural type can lead to severe damage or even collapse during moderate or severe earthquakes. Therefore, it is critical to screen large building stock to find these buildings and retrofit them. However, it is usually time‐consuming to screen soft‐story structures by conventional methods. To tackle this issue, we used full image classification to screen them out from street view images in our previous study. However, full image classification has difficulties locating buildings in an image, which leads to unreliable predictions. In this paper, we developed an automated pipeline in which we segment street view images to identify soft‐story buildings. However, annotated data for this purpose is scarce. To tackle this issue, we compiled a dataset of street view images and present a strategy for annotating these images in a semi‐automatic way. The annotated dataset is then used to train an instance segmentation model that can be used to detect all soft‐story buildings from unseen images.

Globally, civil infrastructure and our built environment is being embedded with dense sensing networks. These networks form the technological foundation of "smart cities" and enable a new, data-driven era of infrastructure engineering for increased lifecycle performance. But smart cities require more than simply embedding IT hardware into existing urban infrastructure, or the application of artificial intelligence to analyze municipal data streams. This paper discusses the set of physical infrastructures, digital technologies, regulations and policies, financing mechanisms, community outreach, businesses and business models, partnerships, institutions, and other engagement mechanisms that must be established in concert with each other to provide a high quality of life in smart and sustainable cities. Specifically, the role of infrastructure life-cycle evaluation is studied in the context of smart city traffic management systems. Results show that smart city traffic management can result in significant reductions in life cycle impacts associated with urban mobility and transportation.

Load rating is a widely used approach for evaluating the load-carrying capacity of bridges in an effort to ensure safe bridge operation under expected traffic loads. Load rating often relies on simplified analytical models including empirically derived model parameters that do not reflect bridge-specific information resulting in conservative ratings. To reduce this conservatism, this study proposes a novel data-driven framework that utilizes long-term bridge response data to extract bridge-specific model parameters that can be used within in the Load and Resistance Factor Rating (LRFR) process. The data-driven LRFR (DD-LRFR) framework is empowered by a cyber-physical system (CPS) architecture that uses Internet connectivity to integrate measured bridge responses with truck weights measured by a weigh-in-motion (WIM) station. The CPS architecture uses computer vision of camera images to confirm trucks observed at a WIM station are identical to those observed at a bridge. Bridge response and axle weight data are then used to extract probabilistic models of dynamic load allowances and unit influence lines. The DD-LRFR method is validated using a 20-mile (32.2-km) segment of the I-275 northbound highway in Michigan that is monitored continuously by the CPS architecture. Six girders associated with two bridges along I-275 are rated using the proposed DD-LRFR methodology with rating factors compared to those obtained using conventional and refined load rating methods. The DD-LRFR method yields inventory- and operational-level rating factors that are less conservative than those from the approximate LRFR method and comparable to those using finite element modeling of the bridge.

Laser Powder Bed Fusion (LPBF) is one of the most promising forms of Additive Manufacturing (AM), allowing easily customized metal manufactured parts. Industry use is currently limited due to the often unknown and unreliable part quality, which is largely caused by the complex relationships between process parameters that include laser power, laser speed, scan strategy, and other machine settings. Melt pools can be monitored with a camera aligned co-axially with the laser to monitor part quality. However, the number of images acquired can be large, exceeding hundreds of thousands for a single part. This paper investigates how the K-Means algorithm, an unsupervised machine learning method, can be used to cluster images of melt pools based on their shape, including undesirable anomalous melt pools. Another unsupervised learning method in this paper is the U-Net autoencoder, which identifies anomalous melt pools by identifying images with a large reconstruction loss. The K-Means clustering or autoencoder provides labels that can be used for training a convolutional neural network image classifier. The image classifier can then be used to identify anomalous melt pools during the LPBF process. This paper provides a first step for real-time process control of the LPBF process by demonstrating how anomalous melt pools can be automatically identified in real-time.

Satellite and street view images are widely used in various disciplines as a source of information for understanding the built environment. In natural hazard engineering, high-quality building inventory data sets are crucial for the simulation of hazard impacts and for supporting decision-making. Screening the building stocks to gather the information for simulation and to detect potential structural defects that are vulnerable to natural hazards is a time-consuming and labor-intensive task. This paper presents an automated method for extracting building information through the use of satellite and street view images. The method is built upon a novel transformer-based deep neural network we developed. Specifically, a multidomain learning approach is employed to develop a single compact model for multiple image-based deep learning information extraction tasks using multiple data sources (e.g., satellite and street view images). Our multidomain Vision Transformer is designed as a unified architecture that can be effectively deployed for multiple classification tasks. The effectiveness of the proposed approach is demonstrated in a case study in which we use pretrained models to collect regional-scale building information that is related to natural hazard risks.

While the increasing deployment of sensors for structural health monitoring has significantly enhanced the monitoring and management of structures, the volume and the variety of data collected have raised the importance of data management. This chapter describes a cloud-based cyber infrastructure platform that builds upon information modeling, NoSQL databases, and cloud computing to address the scalability and flexibility issues for the management of sensor data and engineering information. The platform-neutral interfaces allow application services to be built using a variety of devices—from mobile device, web browser to high-performance computers. The platform is demonstrated for the monitoring of the bridge structures along the I-275 corridor in the State of Michigan. The results show that the platform can greatly facilitate data utilization and data integration.

The articles in this special section focus on the concept of Society 5.0. The five articles provide a glimpse of the human-centric applications and approaches in our continuously evolving digital society. With the rapid developments and proliferation of IoT, cyber-physical systems, AI, autonomous and mobility technologies, cyber (virtual) space and physical (real) space are converging. Such advancements in information and communication technology (ICT) have brought progress in industrial automation and economic growth. As things, people, and devices are increasingly connected, another digital revolution that focuses on human, social, and environmental needs is underway. Having its origin in Japan, Society 5.0 attempts to bring social and environmental concerns into the midst of a digital society. Conceptually, Society 5.0 is defined as “a human-centered (smart) society that balances economic advancement with the resolution of social problems by a system that highly integrates cyberspace and physical space.” a For a human-centered society, technologies can contribute significant value to many fields, such as mobility, healthcare and caregiving, manufacturing and infrastructures, food and agriculture, energy and climate, and disaster prevention. All the previous are in complete accordance with IEEE’s core purpose to foster technological innovation and excellence for the benefit of humanity.

The intensity of many natural hazards, such as hurricanes, floods, tornadoes, etc., are increasing as a consequence of climate change. This increase in intensity coupled with the increase in population density, particularly along the coasts, is only magnifying the impact of such events. In order to quantify and mitigate the risk due to the hazards and to prepare for the potential impacts in a region, it is necessary to collect the information of existing buildings that are pertinent to natural hazard analysis and risk management. Gathering the building information in a region- or city-scale is a laborious and expensive undertaking. This paper presents a framework for regional scale building information generation/gathering to support regional hazard analysis. In this framework, different types of data are acquired from multiple sources (satellite and street view images, property tax assessment data, etc.) and are fused to semantically profile each building in a city. Specifically, deep learning technique is employed to extract building information from street or satellite images. A novel data mining tool is developed to overcome the data scarcity issue, quantify the uncertainty and enrich the data repository. With this framework, building inventories of cities are created to provide the data needed for disaster and risk management planning and simulations.

Segmentation of additive manufacturing (AM) defects in X-ray Computed Tomography (XCT) images is challenging, due to the poor contrast, small sizes and variation in appearance of defects. Automatic segmentation can, however, provide quality control for additive manufacturing. Over recent years, three-dimensional convolutional neural networks (3D CNNs) have performed well in the volumetric segmentation of medical images. In this work, we leverage techniques from the medical imaging domain and propose training a 3D U-Net model to automatically segment defects in XCT images of AM samples. This work not only contributes to the use of machine learning for AM defect detection but also demonstrates for the first time 3D volumetric segmentation in AM. We train and test with three variants of the 3D U-Net on an AM dataset, achieving a mean intersection of union (IOU) value of 88.4%.

Additive manufacturing (AM) provides design flexibility and allows rapid fabrications of parts with complex geometries. The presence of internal defects, however, can lead to deficit performance of the fabricated part. X-ray Computed Tomography (XCT) is a non-destructive inspection technique often used for AM parts. Although defects within AM specimens can be identified and segmented by manually thresholding the XCT images, the process can be tedious and inefficient, and the segmentation results can be ambiguous. The variation in the shapes and appearances of defects also poses difficulty in accurately segmenting defects. This paper describes an automatic defect segmentation method using U-Net based deep convolutional neural network (CNN) architectures. Several models of U-Net variants are trained and validated on an AM XCT image dataset containing pores and cracks, achieving a best mean intersection over union (IOU) value of 0.993. Performance of various U-Net models is compared and analyzed. Specific to AM porosity segmentation with XCT images, several techniques in data augmentation and model development are introduced. This work demonstrates that U-Net can be effectively applied for automatic segmentation of AM porosity from XCT images with high accuracy. The method can potentially help improve quality control of AM parts in an industry setting.