Reinforcement Learning (RL) algorithms, crucial for robotic tasks in sparse reward settings, often require extensive interactions with the environment to learn effectively. Hindsight Experience Replay (HER) addresses this challenge by introducing virtual goals, thereby enhancing sample efficiency by learning from transitions in which the original goal was not achieved. However, randomly sampled additional goals in HER may hinder learning due to their irrelevance. To mitigate this, we propose the improved Priority-based Hindsight experience replay method to prioritize additional goals, improving learning speed and robustness across various robotic tasks. Our improved Priority-based Hindsight Experience Replay method samples with higher probability on the trajectories with temporal difference error and reducible loss, prioritizing those with greater learning potential. Through empirical validation, strategies taken by improved Priority-based Hindsight experience replay method have been demonstrated to enhance training performance and sample efficiency, offering promising avenues for advancing RL in robotic control tasks.
Gaining competitive advantage and market share is essential for business continuity and growth. However, under conditions of increased uncertainty (e.g. economic, geopolitical, technological, and environmental risks), the risk of supply chain disruptions also increases. Thus, market concentration and the risk of supply chain disruption are closely related, and how to balance supply chain resilience (information and technology sharing for risk dispersion) and competitiveness (price, quality, and delivery advantages) is an important issue. This study is to construct a conceptual model that captures phenomena such as supply fluctuations (disruptions and delays) and changes in trading relationships (overlaps and concentrations) that occur in supply chain networks (SCNs), and to visualize how supply chain disruption and market concentration affect SCNs by constructing two types of conceptual models: the SCNs disruption model and the SCNs overlap model. Although, it is very important to analyze SCNs with large and complex structures in more detail, this study considers it is also important to construct simple models, and combining them, quickly analysis.
A point-based deep peeling method applicable to processing three-dimensional (3D) scanned point clouds is proposed in this paper, which has been originally used in the field of polygon processing. With this method, the task of photorealistic, i.e., opaque visualization of a 3D scanned point cloud, is achieved by extracting visible points and making noise points transparent, i.e., invisible. Additionally, this method does not result in loss of clarity and can finely visualize the detailed structure of the object. It allows the method to achieve high-precision visualization of 3D scanned point clouds. By taking advantage of the low color stability of the noise, our method ensures that the visualized object itself is made opaque and visible while the noise points are made invisible. The effectiveness of transparentizing noise is demonstrated using actual 3D scanned point cloud data for a high-value cultural heritage object, and the proposed method may be a ideal solution of opaque visualization of 3D scanned point clouds.
Image captioning is a cross-modal task that combines computer vision and natural language processing. The model is required to generate an appropriate caption for the given image. To address this challenge, we proposed a Residual Gated Transformer, RGFormer, as an enhancement of Transformer architecture. The model based on CLIP and RGFormer, CRM, is then proposed for image captioning. CRM utilizes the encoder of CLIP to extract image features as a prefix to the caption. The prefix is projected into language space using RGFormer, a lightweight mapping network, and then fed into GPT-2 to generate captions. CLIP was trained on an extensive dataset comprising image-text pairs, which contains rich visual and semantic information and is exceptionally well-suited for vision-language tasks. The core idea of CRM is to reduce the disparity between visual and textual representations by using RGFormer to accomplish the cross-modal task. CRM could generate meaningful captions for diverse and large-scale datasets in a short training time without additional annotations or pre-training. Quantitative evaluation experiments show that CRM achieves results comparable to some advanced models on the COCO Caption dataset more efficiently.
Industrial Metaverse plays an increasingly important role in Intelligent Manufacturing Training, improving the user immersion as well as reducing the cost and risks in training. Action recognition technology can effectively detect human actions, help construct the industrial metaverse space of Human-in-the-loop, and empower the low-intrusion, personalized intelligent manufacturing training system. Intelligent Manufacturing Training-Worker Action Recognition and Evaluation (IMTware) is researched to improve the human-manchine interaction and intelligent guidance in industrial training. In terms of action recognition, IMTware extracts the skeletal point sequences of human practice training actions in the video via OpenPose network and inputs them into ST-GCN network for action recognition. In terms of action evaluation, a method combining Dynamic Time Warping (DTW) keyframe matching with cosine similarity solving is proposed to compute the action similarity between the skeletal point sequences and the standard skeletal point sequences, and evaluate them according to the similarity to give personalized correction suggestions. Subsequently, the performance of the system is validated using overhead crane command actions training in converter steelmaking as an example, achieving satisfactory results.
