Pre-trained language models (PLMs) have been successfully used to build high-performance ranking models for large-scale information retrieval systems. However, traditional PLM-based ranking approaches face two key challenges: (1) these models use both sparse and dense content (such as the query/title and content of documents) as inputs, which may require different attention allocations; and (2) traditional PLM-based ranking approaches have identified multiple objectives to gauge user satisfaction with ranking results, but integrating these objectives into the end-to-end training process and the subsequent feature updates and iterations usually involves significant computational resource overhead. In this paper, we propose a novel PLM-based ranking approach M2oE Rank, Multi-objective Mixture-of-Experts (MoE) enhanced Ranking. Specifically, M2oERank lever-ages a context-aware PLM-based hierarchical encoder to extract semantic relevance between the query and the document title and content, while allowing for separate dense and sparse attention for different inputs. With the extracted semantic relevance repre-sentations, multifacet user satisfaction features and task-specific annotations, M2oERank employs an MoE module to perform multi-objective pre-training of ranking models focused on user satisfaction. Finally, M2oERank uses a weight fusion module that fuses outputs from the above experts to predict ranking scores. Moreover, we present a three-stage offline training strategy and the online system workflow for deploying M2oERank at web-scale search. To demonstrate the effectiveness of our proposed approach, we conduct extensive offline and online evaluations using real-world web traffic from Baidu Search. The comparisons against numbers of advanced baselines confirmed the advantages of M2oERank in producing high-performance ranking models for web-scale search.

In this paper, we introduce the AI Search Paradigm, a comprehensive blueprint for next-generation search systems capable of emulating human information processing and decision-making. The paradigm employs a modular architecture of four LLM-powered agents (Master, Planner, Executor and Writer) that dynamically adapt to the full spectrum of information needs, from simple factual queries to complex multi-stage reasoning tasks. These agents collaborate dynamically through coordinated workflows to evaluate query complexity, decompose problems into executable plans, and orchestrate tool usage, task execution, and content synthesis. We systematically present key methodologies for realizing this paradigm, including task planning and tool integration, execution strategies, aligned and robust retrieval-augmented generation, and efficient LLM inference, spanning both algorithmic techniques and infrastructure-level optimizations. By providing an in-depth guide to these foundational components, this work aims to inform the development of trustworthy, adaptive, and scalable AI search systems.

Target search using a swarm of robots is a classic research topic that poses challenges, particularly in conducting multi-target searching in unknown environments. Key challenges include high communication cost among robots, unknown positions of obstacles, and the presence of multiple targets. To address these challenges, we propose a novel Robotic Flow Direction Algorithm (RFDA), building upon the modified Flow Direction Algorithm (FDA) to suit the characteristics of the robot’s motion. RFDA efficiently reduces the communication cost and navigates around unknown obstacles. The algorithm also accounts for scenarios involving isolated robots. The pipeline of the proposed RFDA method is outlined as follows: (1). Learning strategy: a neighborhood information based learning strategy is adopted to enhance the FDA’s position update formula. This allows swarm robots to systematically locate the target (the lowest height) in a stepwise manner. (2). Adaptive inertia weighting: An adaptive inertia weighting mechanism is employed to maintain diversity among robots during the search and avoid premature convergence. (3). Sink-filling process: The algorithm simulates the sink-filling process and moving to the aspect slope to escape from local optima. (4). Isolated robot scenario: The case of an isolated robot (a robot without neighbors) is considered. Global optimal information is only required when the robot is isolated or undergoing the sink-filling process, thereby reducing communication costs. We not only demonstrate the probabilistic completeness of RFDA but also validate its effectiveness by comparing it with six other competing algorithms in a simulated environment. Experiments cover various aspects such as target number, population size, and environment size. Our findings indicate that RFDA outperforms other methods in terms of the number of required iterations and the full success rate. The Friedman and Wilcoxon tests further demonstrate the superiority of RFDA.

Flow direction algorithm (FDA) is a new physics-based optimization algorithm for solving global optimization problems. Although the FDA has shown effectiveness in many areas, there has been a lack of rigorous theoretical guarantees. This paper first proves that FDA is globally convergent with probability 1 by establishing a Markov process model. Furthermore, to enhance the FDA’s exploration and exploitation abilities, we propose an improved FDA algorithm (IFDA) by introducing random opposition-based learning and an adaptive neighbour generation strategy. Finally, extensive experiments and statistical tests are investigated on the classical benchmark functions, CEC 2019 benchmark function, and wireless sensor network coverage optimization problem with several state-of-the-art algorithms, demonstrating the proposed algorithm’s efficiency and effectiveness.

The unbiased learning to rank (ULTR) problem has been greatly advanced by recent deep learning techniques and well-designed debias algorithms. However, promising results on the existing benchmark datasets may not be extended to the practical scenario due to the following disadvantages observed from those popular benchmark datasets: (1) outdated semantic feature extraction where state-of-the-art large scale pre-trained language models like BERT cannot be exploited due to the missing of the original text;(2) incomplete display features for in-depth study of ULTR, e.g., missing the displayed abstract of documents for analyzing the click necessary bias; (3) lacking real-world user feedback, leading to the prevalence of synthetic datasets in the empirical study. To overcome the above disadvantages, we introduce the Baidu-ULTR dataset. It involves randomly sampled 1.2 billion searching sessions and 7,008 expert annotated queries, which is orders of magnitude larger than the existing ones. Baidu-ULTR provides:(1) the original semantic feature and a pre-trained language model for easy usage; (2) sufficient display information such as position, displayed height, and displayed abstract, enabling the comprehensive study of different biases with advanced techniques such as causal discovery and meta-learning; and (3) rich user feedback on search result pages (SERPs) like dwelling time, allowing for user engagement optimization and promoting the exploration of multi-task learning in ULTR. In this paper, we present the design principle of Baidu-ULTR and the performance of benchmark ULTR algorithms on this new data resource, favoring the exploration of ranking for long-tail queries and pre-training tasks for ranking. The Baidu-ULTR dataset and corresponding baseline implementation are available at https://github.com/ChuXiaokai/baidu_ultr_dataset.

Session-based recommendation has recently attracted increasing attention from both industry and academic communities. Previous models mostly focus on designing different models to fit the observed data, which can be quite sparse in real-world scenarios. To alleviate this problem, in this paper, we propose a novel generative session-based recommendation framework. The main building block of our idea is to develop a generator to simulate user sequential behaviors, which are leveraged to train and improve the target sequential recommender model. In order to generate high quality samples, we consider two aspects: (1) the rationality as a sequence of user behaviors, and (2) the informativeness for training the target model. To satisfy these requirements, we design a doubly adversarial network. The first adversarial module aims to make the generated samples conform to the underlying patterns of the real user sequential preference (rationality requirement). The second adversarial module is targeted at widening the model experiences by generating samples which can induce larger model losses (informativeness requirement). In our model, the samples are generated based on a reinforcement learning strategy, where the reward is related with both of the above aspects. In order to stable the training process, we introduce a self-paced regularizer to learn the agent in an easy-to-hard manner. We conduct extensive experiments based on real-world datasets to demonstrate the effectiveness of our model.

Extracting query-document relevance from the sparse, biased clickthrough log is among the most fundamental tasks in the web search system. Prior art mainly learns a relevance judgment model with semantic features of the query and document and ignores directly counterfactual relevance evaluation from the clicking log. Though the learned semantic matching models can provide relevance signals for tail queries as long as the semantic feature is available. However, such a paradigm lacks the capability to introspectively adjust the biased relevance estimation whenever it conflicts with massive implicit user feedback. The counterfactual evaluation methods, on the contrary, ensure unbiased relevance estimation with sufficient click information. However, they suffer from the sparse or even missing clicks caused by the long-tailed query distribution. In this paper, we propose to unify the counterfactual evaluating and learning approaches for unbiased relevance estimation on search queries with various popularities. Specifically, we theoretically develop a doubly robust estimator with low bias and variance, which intentionally combines the benefits of existing relevance evaluating and learning approaches. We further instantiate the proposed unbiased relevance estimation framework in Baidu search, with comprehensive practical solutions designed regarding the data pipeline for click behavior tracking and online relevance estimation with an approximated deep neural network. Finally, we present extensive empirical evaluations to verify the effectiveness of our proposed framework, finding that it is robust in practice and manages to improve online ranking performance substantially.

Multivariate Time Series Classification (MTSC) is a prevalent but challenging problem in data mining. With the development of Deep Neural Networks (DNN), hundreds of deep models for MTSC have been proposed. However, most prior works only explicitly model the relationship between time series and classes and ignore the diversity of the relationship, suffering from insufficient information exploitation. In this paper, we propose a novel module named Multi-Relationship Modeling Module(MRM2) for more effective MTSC. MRM2 uses the classified labels to explicitly model not only the relationship between time series and classes, but also the relationship among time series, enabling the backbone to generate distinguishable embeddings. In addition, MRM2 is versatile because it can be combined with the existing backbones of DNN for end-to-end training. Finally, we conduct a series of ablation studies and comparative experiments on the real multivariate time series archive UEA. Experimental results indicate that MRM2 can significantly improve classification performance in most cases. Codes are available on GitHub (1).

Deep pre-trained language models (e.g., BERT) lead to remarkable headway in many Natural Language Processing tasks. Their superior capacity in perceiving textual data is also witnessed in semantic matching tasks (e.g., question answering, web search). Particularly for matching a pair of query and text candidate, the current state-of-the-arts usually rely on the semantic representations produced by BERT, and compute relevance scores with various interaction (i.e., matching) methods. However, they may 1) miss fine-grained phrase-level interaction between the input query and candidate context or 2) lack a thoughtful consideration of both effectiveness and efficiency. Motivated by this, we propose \hyttInteractor, a BERT-based semantic matching model with a flexible contextualized interaction paradigm. It is capable of capturing fine-grained phrase-level information in the interaction, and thus is more effective to be applied for semantic matching tasks. Moreover, we further facilitate \hyttInteractor with a novel partial attention scheme, which significantly reduces the computational cost while maintaining the high effectiveness. We conduct comprehensive experimental evaluations on three datasets. The results show that \hyttInteractor achieves superior effectiveness and efficiency for semantic matching.

Incorporating review information into the recommender system has been demonstrated to be an effective method for boosting the recommendation performance. Previous research mainly focus on designing advanced architectures to better profile the users and items. However, the review information in realities can be highly sparse and imbalanced, which poses great challenges for effective user/item representations and satisfied performance enhancement. To alleviate this problem, in this paper, we propose to improve review-based recommendation by counterfactually augmenting the training samples. We focus on a common setting --- feature-aware recommendation, and the main building block of our idea lies in the counterfactual question: "what would be the user's decision if her feature-level preference had been different?''. When augmenting the training samples, we actively change the user preference (also called intervention), and predict the user feedback on the items based on pre-trained recommender models. Instead of changing the user preference in a random manner, we design a learning-based method to discover the samples which are more effective for model optimization. In order to improve the sample qualities, we propose two strategies --- constrained feature perturbation and frequency-based sampling --- to equip our model. Since the sample generation model can be not perfect, we theoretically analyze the relation between the model prediction error and the number of generated samples. As a byproduct, our framework can explain the user pair-wise preference, which is complementary to the traditional point-wise explanations. Extensive experiments demonstrate that our model can significantly improve the performance of the state-of-the-art methods.

Objective To investigate the anti-inflammatory effects and mechanisms of pulsed electromagnetic field (PEMF) in the treatment of osteoarthritis (OA) in the destabilization of the medial meniscus (DMM) mice. Design Ten-week-old male wild-type (WT), interleukin (IL)-6−/− and tumor necrosis factor (TNF)-α−/− mice undergoing DMM surgery were randomly divided into 2 groups ( n = 10 each): mice with PEMF exposure and mice with sham PEMF exposure. PEMF (75 Hz, 3.8 mT, 1 h/day) or sham PEMF was applied for 4 weeks. Pain behavior of mice, histological assessment of cartilage and synovium, micro-CT (computed tomography) analysis of bone, real-time polymerase chain reaction, and immunohistochemical staining of cartilage were performed. Results After DMM surgery, PEMF had a beneficial effect on pain, cartilage degeneration, synovitis, and trabecular bone microarchitecture in WT mice; these protective effects were reduced in IL-6−/− and TNF-α−/− mice. In addition, PEMF downregulated IL-6 and TNF-α expression in cartilage. PEMF also ameliorated cartilage matrix, chondrocyte apoptosis, and autophagy, while deletion of IL-6 or TNF-α suppressed the effects. Conclusions PEMF attenuates structural and functional progression of OA through inhibition of TNF-α and IL-6 signaling. The protective effects of PEMF on chondrocyte apoptosis and autophagy are regulated by TNF-α and IL-6 signaling.

Post-click conversion, as a strong signal indicating the user preference, is salutary for building recommender systems. However, accurately estimating the post-click conversion rate (CVR) is challenging due to the selection bias, i.e., the observed clicked events usually happen on users' preferred items. Currently, most existing methods utilize counterfactual learning to debias recommender systems. Among them, the doubly robust (DR) estimator has achieved competitive performance by combining the error imputation based (EIB) estimator and the inverse propensity score (IPS) estimator in a doubly robust way. However, inaccurate error imputation may result in its higher variance than the IPS estimator. Worse still, existing methods typically use simple model-agnostic methods to estimate the imputation error, which are not sufficient to approximate the dynamically changing model-correlated target (i.e., the gradient direction of the prediction model). To solve these problems, we first derive the bias and variance of the DR estimator. Based on it, a more robust doubly robust (MRDR) estimator has been proposed to further reduce its variance while retaining its double robustness. Moreover, we propose a novel double learning approach for the MRDR estimator, which can convert the error imputation into the general CVR estimation. Besides, we empirically verify that the proposed learning scheme can further eliminate the high variance problem of the imputation learning. To evaluate its effectiveness, extensive experiments are conducted on a semi-synthetic dataset and two real-world datasets. The results demonstrate the superiority of the proposed approach over the state-of-the-art methods. The code is available at https://github.com/guosyjlu/MRDR-DL.

Recently, significant progress has been made in time series classification with deep learning. However, using deep learning models to solve time series classification generally suffers from expensive calculations and difficulty of data labeling. In this work, we study self-supervised time series pre-training to overcome these challenges. Compared with the existing works, we focus on the universal and unlabeled time series pretraining. To this end, we propose a novel end-to-end neural network architecture based on self-attention, which is suitable for capturing long-term dependencies and extracting features from different time series. Then, we propose two different self-supervised pretext tasks for time series data type: Denoising and Similarity Discrimination based on DTW (Dynamic Time Warping). Finally, we carry out extensive experiments on 85 time series datasets (also known as UCR2015 [2]). Empirical results show that the time series model augmented with our proposed self-supervised pretext tasks achieves state-of-the-art / highly competitive results.

Objective. Pulsed electromagnetic field (PEMF) therapy is a potentially useful treatment for osteoarthritis (OA), but its effectiveness is still controversial. This study aimed to examine the effects of PEMF therapy and PEMF parameters on symptoms and quality of life (QOL) in patients with OA. Methods. Cochrane Central Register of Controlled Trials, PubMed, CINAHL, EMBASE, PEDro, clinical trial registers, and reference lists were searched until April 2019. This study examined randomized, placebo-controlled trials, patients with OA, symptom and/or QOL related outcomes, and articles published in English. Two authors extracted data and completed quality assessment. Results. Sixteen studies were included in our systematic review, while 15 studies with complete data were included in the meta-analysis. Our primary outcome was the standardized mean difference, which was equal to the treatment effect in the PEMF group minus the treatment effect in the placebo group divided by the pooled standard deviation. For pain, the standardized mean difference was 1.06 (95% CI = 0.61 to 1.51), for stiffness 0.37 (95% CI = 0.07 to 0.67), for function 0.46 (95% CI = 0.14 to 0.78), and for QOL 1.49 (95% CI = -0.06 to 3.04). PEMF parameters did not influence symptoms. Conclusions. Compared with placebo, there was a beneficial effect of PEMF therapy on pain, stiffness, and physical function in patients with OA. Duration of treatment may not be a critical factor in pain management. Further studies are required to confirm the effects of PEMF therapy on QOL. Impact. Our study suggests that PEMF therapy has clinically significant effects on pain in patients with OA. The current evidence was limited to the short-term effects of PEMF therapy.

Pulsed electromagnetic field (PEMF) and whole body vibration (WBV) interventions are expected to be important strategies for management of osteoarthritis (OA). The aim of the study was to investigate the comparative effectiveness of PEMF versus WBV on cartilage and subchondral trabecular bone in mice with knee OA (KOA) induced by surgical destabilization of the medial meniscus (DMM). Forty 12‐week‐old male C57/BL mice were randomly divided into four groups ( n = 10): Control, OA, PEMF, and WBV. OA was induced (OA, PEMF, and WBV groups) by surgical DMM of right knee joint. Mice in PEMF group received 1 h/day PEMF exposure with 75 Hz, 1.6 mT for 4 weeks, and the WBV group was exposed to WBV for 20 min/day with 5 Hz, 4 mm, 0.3 g peak acceleration for 4 weeks. Micro‐computed tomography (micro‐CT), histology, and immunohistochemistry analyses were performed to evaluate the changes in cartilage and microstructure of trabecular bone. The bone volume fraction (BV/TV), trabecular thickness (Tb.Th), and trabecular number (Tb.N) increased, and bone surface/bone volume (BS/BV) decreased by micro‐CT analysis in PEMF and WBV groups. The Osteoarthritis Research Society International (OARSI) scores in PEMF and WBV groups were significantly lower than in the OA group. Immunohistochemical results showed that PEMF and WBV promoted expressions of Aggrecan, and inhibited expressions of IL‐1β, ADAMTS4, and MMP13. Superior results are seen in PEMF group compared with WBV group. Both PEMF and WBV were effective, could delay cartilage degeneration and preserve subchondral trabecular bone microarchitecture, and PEMF was found to be superior to WBV. Bioelectromagnetics. 2020;41:298–307 © 2020 Bioelectromagnetics Society

Incorporating temporal information into recommender systems has recently attracted increasing attention from both the industrial and academic research communities. Existing methods mostly reduce the temporal information of behaviors to behavior sequences for subsequently RNN-based modeling. In such a simple manner, crucial time-related signals have been largely neglected. This paper aims to systematically investigate the effects of the temporal information in sequential recommendations. In particular, we firstly discover two elementary temporal patterns of user behaviors: "absolute time patterns'' and "relative time patterns'', where the former highlights user time-sensitive behaviors, e.g., people may frequently interact with specific products at certain time point, and the latter indicates how time interval influences the relationship between two actions. For seamlessly incorporating these information into a unified model, we devise a neural architecture that jointly learns those temporal patterns to model user dynamic preferences. Extensive experiments on real-world datasets demonstrate the superiority of our model, comparing with the state-of-the-arts.

In this paper, we propose a new maximum entropy clustering algorithm by modified grey wolf optimizer (GWO), which modify the traditional GWO from twofold: First, nonlinear decay factor is constructed, which leads to flexible refined search; Second, proportional weights, which render the positions of distinct grey wolves, are adjusted adaptively according to the social hierarchy of them. Based on these two modifications, the rectified GWO could greatly improve both the accuracy and convergence rate. Experimental results on 12 benchmark functions demonstrate the effectiveness of those modifications. Furthermore, we utilize modified GWO to maximum entropy based fuzzy clustering problems. Experiments on 5 real datasets indicate the high performance and efficiency of the proposed approach.

OBJECTIVE:The aim of the study was to assess the effects of elastic taping on pain, physical function, range of motion, and muscle strength in patients with knee osteoarthritis.DESIGN:We searched the PubMed, the Cochrane Central Register of Controlled Trials, Web of Science, Physiotherapy Evidence Database, Scopus, EMBASE, OVID, CNKI, and WANFANG to identify relevant randomized controlled trials. The primary outcome measures were pain and physical function. The secondary outcome measures were range of motion and muscle strength.RESULTS:Eleven randomized controlled trials involving 490 patients with knee osteoarthritis were included. A statistically significant difference was detected in pain (standardized mean difference = -0.78, 95% confidence interval = 1.07 to -0.50, P < 0.00001), physical function (standardized mean difference = 0.73, 95% confidence interval = -1.03 to -0.43, P < 0.00001), range of motion (mean difference = 2.04, 95% confidence interval = 0.14 to 3.94, P = 0.04), and quadriceps muscle strength (mean difference = 2.42, 95% confidence interval = 1.09 to 3.74, P = 0.0004). No significant differences were found for the hamstring muscle strength.CONCLUSIONS:Elastic taping has significant effects on pain, physical function, range of motion, and quadriceps muscle strength in patients with knee osteoarthritis. The current evidence is insufficient to draw conclusions on the effects of elastic taping combined with other physiotherapy for knee osteoarthritis. Further studies are needed to investigate the long-term effects of elastic taping combined with other physiotherapy compared with elastic taping alone for knee osteoarthritis.

Recent years have witnessed the popularity of game recommendation. Different from the other recommendation scenarios, the user and item properties in game recommendation usually exhibit highly dynamic properties, and may influence each other in the user-item interaction process. For taming such characters, so as to design a high quality recommender system tailored for game recommendation, in this paper, we design a dynamic graph convolutional network to highlight the user/item evolutionary features. More specifically, the graph neighbors in our model are not static, they will be adaptively changed at different time. The recently interacted users or items are gradually involved into the aggregation process, which ensures that the user/item embeddings can evolve as the time goes on. In addition, to timely match the changed neighbors, we also update the convolutional weights in a RNN-manner. By these customized strategies, our model is expected to learn more accurate user behavior patterns in the field of game recommendation. We conduct extensive experiments on real-world datasets to demonstrate the superiority of our model.

Image captioning (IC) is a commonly-used technique for generating textual image description, which finds its applications on semantic image retrieval and multi-modal image understanding, among many others. This paper focuses on an important IC method specialized for generating aesthetic descriptions of images, i.e., aesthetic image captioning (AIC). Despite some effectiveness of initial work on AIC, their performances are inherently limited due to a lack of consideration of user preferences on aesthetics and better aesthetic feature, making it unusable for real-world applications where human users present a large variation on evaluating visual aesthetics of images. To tackle this, we propose a novel personalized aesthetic image caption (PAIC) approach for capturing and incorporating user preferences for AIC tasks. Our approach mainly contains Aesthetic feature Extraction Network(AEN), User Encoder network(UEN) and a personalized image caption model. AEN is designed to extract more expressive feature, UEN is introduced for learning the user vector from the limited information in our AVA-PCap dataset. Personalized image caption model is constructed to generate the caption when given the user id and photo pairs. The experimental results show that our methods outperform baselines by 10% , which is encouraging for a first step towards personalized aesthetic image caption.