Multi-task learning has been widely used in many applications to enable more efficient learning by sharing part of the architecture across multiple tasks. However, a major challenge is the gradient conflict when optimizing the shared parameters, where the gradients of different tasks could have opposite directions. Directly averaging those gradients will impair the performance of some tasks and cause negative transfer. Different from most existing works that manipulate gradients to mitigate the gradient conflict, in this paper, we address this problem from the perspective of architecture learning and propose a Conflict-Noticed Architecture Learning (CoNAL) method to alleviate the gradient conflict by learning architectures. By introducing purely-specific modules specific to each task in the search space, the CoNAL method can automatically learn when to switch to purely-specific modules in the tree-structured network architectures when the gradient conflict occurs. To handle multi-task problems with a large number of tasks, we propose a progressive extension of the CoNAL method. Extensive experiments on computer vision, natural language processing, and reinforcement learning benchmarks demonstrate the effectiveness of the proposed methods.

Designing neural network architecture for embedded devices is practical but challenging because the models are expected to be not only accurate but also enough lightweight and robust. However, it is challenging to balance those trade-offs manually because of the large search space. To solve this problem, we propose an Effective, Efficient, and Robust Neural Architecture Search (E2RNAS) method to automatically search a neural network architecture that balances the performance, robustness, and resource consumption. Unlike previous studies, the objective function of the proposed E2RNAS method is formulated as a multi-objective bi-level optimization problem with the upper-level subproblem as a multi-objective optimization problem that considers the performance, robustness, and resource consumption. To solve the proposed objective function, we integrate the multiple-gradient descent algorithm, a widely studied gradient-based multi-objective optimization algorithm, with the bi-level optimization. Experiments on benchmark datasets show that the proposed E2RNAS method can find robust architecture with low resource consumption and comparable classification accuracy.

In domain adaptation, where the feature distributions of the source and target domains are different, various distance-based methods have been proposed to handle the domain shift by minimizing the discrepancy between the source and target domains. These methods use hand-crafted bottleneck networks, which might hinder the alignment of hidden feature representations extracted from both domains. In this paper, we propose a new method called Alignment Architecture Search with Population Correlation (AASPC) to automatically learn the architecture of the bottleneck network that can align the source and target domains. The proposed AASPC method introduces a new similarity function called Population Correlation (PC) to measure the domain discrepancy. The proposed AASPC method leverages PC to learn the alignment architecture and domaininvariant feature representation. Experiments on several benchmark datasets, including Office-31, Office-Home, and VisDA2017, show the effectiveness of the proposed AASPC method.

In recent years, Multi-Task Learning (MTL) has attracted much attention due to its good performance in many applications. However, many existing MTL models cannot guarantee that their performance is no worse than their single-task counterparts on each task. Though some works have empirically observed this phenomenon, little work aims to handle the resulting problem. In this paper, we formally define this phenomenon as negative sharing and define safe multi-task learning where no negative sharing occurs. To achieve safe multi-task learning, we propose a Deep Safe Multi-Task Learning (DSMTL) model with two learning strategies: individual learning and joint learning. We theoretically study the safeness of both learning strategies in the DSMTL model to show that the proposed methods can achieve some versions of safe multi-task learning. Moreover, to improve the scalability of the DSMTL model, we propose an extension, which automatically learns a compact architecture and empirically achieves safe multi-task learning. Extensive experiments on benchmark datasets verify the safeness of the proposed methods.

Meta learning with multiple objectives can be formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possible conflicting targets for the meta learner. However, existing studies either apply an inefficient evolutionary algorithm or linearly combine multiple objectives as a single-objective problem with the need to tune combination weights. In this paper, we propose a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, neural architecture search, domain adaptation, and multi-task learning.

In Domain Adaptation (DA), where the feature distributions of the source and target domains are different, various distance-based methods have been proposed to minimize the discrepancy between the source and target domains to handle the domain shift. In this paper, we propose a new similarity function, which is called Population Correlation (PC), to measure the domain discrepancy for DA. Base on the PC function, we propose a new method called Domain Adaptation by Maximizing Population Correlation (DAMPC) to learn a domain-invariant feature representation for DA. Moreover, most existing DA methods use hand-crafted bottleneck networks, which may limit the capacity and flexibility of the corresponding model. Therefore, we further propose a method called DAMPC with Neural Architecture Search (DAMPC-NAS) to search the optimal network architecture for DAMPC. Experiments on several benchmark datasets, including Office-31, Office-Home, and VisDA-2017, show that the proposed DAMPC-NAS method achieves better results than state-of-the-art DA methods.

Today there are abounding collected data in cases of various diseases in medical sciences. Physicians can access new findings about diseases and procedures in dealing with them by probing these data. Clinical data is a collection of large and complex datasets that commonly appear in multidimensional data formats. It has been recognized as a big challenge in modern data analysis tasks. Therefore, there is an urgent need to find new and effective techniques to deal with such huge datasets. This paper presents an application of a new visual data mining platform for visual analysis of the stroke data for predicting the levels of risk to those people who have the similar characteristics of the stroke patients. The visualization platform uses a hierarchical clustering algorithm to aggregate the data and map coherent groups of data-points to the same visual elements - curved 'super-polylines' that significantly reduces the visual complexity of the visualization. On the other hand, to enable users to interactively manipulate data items (super-polylines) in the parallel coordinates geometry through the mouse rollover and clicking, we created many 'virtual nodes' along the multi-axis of the visualization based on the hierarchical structure of the value range of selected data attributes. The experimental result shows that we can easily verify research hypothesis and reach to the conclusion of research questions through human-data & human-algorithm interactions by using this visual platform with a fully transparency manner of data processing.

With the development of mobile internet technology, there have a enormous pontential in mobile advertisement. But how we use this recources becomes a big problem. Fortinately, we can use a recommendation to recommend the advertise for the peolpe the may like it. This is called a excatly advertising putting. It can help the people to get the really information they want, also it can cut down the cost of the company, the can get the consumer approach from which platform, and pay for the cost in datadflow. And the collected data can help the company analysis the user's distribution so that the can improve the production and advertising. So Exactly advertising is one of the most important thing, the effect of advertising, usually measure by clicking and conversion rate in each link, most advertising system by advertising effect data return as the delivery efficiency measure standard to carry out optimization through exposure or click. But how we can trace the user behavior and predicted the advertisement conversion rate. Tecent use the pCVR(Predicted Conversion Rate), to help advertisers tracking advertising.This topic based on the mobile App advertising as the research object, to predict the probability of App ad Click after the activated which is a given advertising, the user and the context condition of advertising is the probability of click after activation. We will try to use KNN, random forest, User-Based top-N recommendation, Time Series model to set up a predict model and verification it in the last for this problem.

Developing a verbal reasoning question recommendation system is an ideal way to help the GRE® test takers improve their verbal reasoning abilities by practicing questions more efficiently. As there are a great number of verbal reasoning practice questions and limited practice time for test takers, it is impossible to practice all kinds of questions at the same time. Personalized referral systems should be built based on the characteristics of specific respondents, and forming professional recommendation systems for different questions. Based on the examinee's current practicing accuracy and fallible difficulties, we propose an End-to-end Tag-based Recommendation System (ETRS) for task takers to optimize practice effect. Code of this paper can be found on https://github.com/Oliver-Q/ETRS-for-Verbal-Reasoning-Questions.