
Despite the broad application of deep reinforcement learning (RL), transferring and adapting the policy to unseen but similar environments is still a significant challenge. Recently, the language-conditioned policy is proposed to facilitate policy transfer through learning the joint representation of observation and text that catches the compact and invariant information across environments. Existing studies of language-conditioned RL methods often learn the joint representation as a simple latent layer for the given instances (episode-specific observation and text), which inevitably includes noisy or irrelevant information and cause spurious correlations that are dependent on instances, thus hurting generalization performance and training efficiency. To address this issue, we propose a conceptual reinforcement learning (CRL) framework to learn the concept-like joint representation for language-conditioned policy. The key insight is that concepts are compact and invariant representations in human cognition through extracting similarities from numerous instances in real-world. In CRL, we propose a multi-level attention encoder and two mutual information constraints for learning compact and invariant concepts. Verified in two challenging environments, RTFM and Messenger, CRL significantly improves the training efficiency (up to 70%) and generalization ability (up to 30%) to the new environment dynamics.
Geo-distributed ML training can benefit many emerging ML scenarios (e.g., large model training, federated learning) with multi-regional cloud resources and wide area network. However, its efficiency is limited due to 2 challenges. First, efficient elastic scheduling of multi-regional cloud resources is usually missing, affecting resource utilization and performance of training. Second, training communication on WAN is still the main overhead, easily subjected to low bandwidth and high fluctuations of WAN. In this paper, we propose a framework, Cloudless-Training, to realize efficient PS-based geo-distributed ML training in 3 aspects. First, it uses a two-layer architecture with control and physical training planes to support elastic scheduling and communication for multi-regional clouds in a serverless maner.Second, it provides an elastic scheduling strategy that can deploy training workflows adaptively according to the heterogeneity of available cloud resources and distribution of pre-existing training datasets. Third, it provides 2 new synchronization strategies for training partitions among clouds, including asynchronous SGD with gradient accumulation (ASGD-GA) and inter-PS model averaging (MA). It is implemented with OpenFaaS and evaluated on Tencent Cloud. Experiments show that Cloudless-Training can support general ML training in a geo-distributed way, greatly improve resource utilization (e.g., 9.2%-24.0% training cost reduction) and synchronization efficiency (e.g., 1.7x training speedup over baseline at most) with model correctness guarantees.
In order to study the effect of solid particles of different shapes in the gas pipelines on the erosion wear charac-teristics of the elbow and predict the wear distribution of the inner wall of the elbow after being impacted,computa-tional fluid dynamics(CFD)is used to calculate the gas flow.The discrete element method(DEM)is used to cal-culate the particle motion,and the non-spherical particles are described by the super-ellipsoids model.The shear impact energy model(SIEM)is used to calculate elbow wear.The simulation results show that,under the same gas velocity,although the difference of the movement of particles of different shapes in the elbow is not evident,the corresponding wear condition of the inner wall of the elbow is significantly different from each other:the closer the particle shape is to the square,the higher the maximum wear rate in the elbow,and the wider the range of serious wear.But the position of the maximum wear rate will tend to be stable.The probability of sliding friction of prolate ellipsoid particles in the elbow is higher,while the probability of rolling friction of oblate ellipsoid particles is high-er,which causes the total wear rate of prolate ellipsoid particles on the elbow to be higher than that of the oblate el-lipsoid.Comparing the influence of particle shape on the wear of the elbow under different gas velocities,it can be found that larger gas velocity only homogeneously increases the wear of the elbow and the maximum erosion zone will not change.
In this paper,the shock vibration characteristic and the fatigue life of the sampling mechanism of sludge drying moisture detection device are investigated based on the discrete element method and finite element method(DEM-FEM).The impact load distribution under different particle sizes and the impact characteristic under different re-ceiving angles are analyzed.Furthermore,the modal vibration patterns and amplitude-frequency responses under different excitations are studied,and the fatigue life distribution and safety factor distribution under different loads are analyzed.The results show that particle size and receiving angle have significant influence on impact character-istics,and the avoidance between the motor frequency and the structural inherent frequency during the advance of the sampling mechanism reaches 24%.Under normal light loads,the number of stress cycles that can be withstood is 1×108,while the minimum safety factor is approximately 0.54 under a heavy load of 100 N.The fatigue failure and fracture tend to occur at the connecting rods.
In this paper,numerical artificial boundary conditions for the damped dispersive wave problems are designed.The matching boundary condition method is used to construct accurate local artificial boundary conditions by matc-hing the characteristic frequency-wave number relationship of damped dispersive waves.The matching boundary conditions take a linear combination form of atomic displacement and velocity near the artificial boundary,and the combination coefficients are determined by matching the frequency-wave number relationship.In this work,a one-dimensional infinitely long damped monoatomic chain is taken as an example,and the frequency-wave number rela-tionship of damped dispersive waves is established with frequency rather than wave number as the independent vari-able.The proposed matching boundary conditions can effectively deal with damped waves with different wave speeds and different spatial attenuation rates.Both reflection coefficient analysis and numerical examples verify the validity of the artificial boundary conditions.Matching boundary conditions are compact in form,low cost in compu-tation,and efficient in absorption,and can be applied to molecular dynamics simulation and multi-scale calculation of crystals.