Transfer learning (TL) is an information reuse learning tool, which can help us learn better classification effect than traditional single task learning, because transfer learning can share information within the task-to-task model. Most TL algorithms are studied in the field of data improvement, doing some data extraction and transformation. However, it ignores that existing the additional information to improve the model's accuracy, like Universum samples in the training data with privileged information. In this article, we focus on considering prior data to improve the TL algorithm, and the additional features also called privileged information are incorporated into the learning to improve the learning paradigm. In addition, we also carry out the Universum samples which do not belong to any indicated categories into the transfer learning paradigm to improve the utilization of prior knowledge. We propose a new TL Model (PU-TLSVM), in which each task with corresponding privileged features and Universum data is considered in the proposed model, so as to apply tasks with a priori data to the training stage. Then, we use Lagrange duality theorem to optimize our model to obtain the optimal discriminant for target task classification. Finally, we make a lot of predictions and tests to compare the actual effectiveness of the proposed method with the previous methods. The experiment results indicate that the proposed method is more effective and robust than other baselines.

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications.

Universum, as third class that does not belong to the positive class and negative class, allows to incorporate the prior knowledge into the learning process. A lot of reaserchers confirmed that Universum is helpful in the supervised and semi-supervised learning. Moreover, Universum has already been introduced into the support vector machine (SVM) to enhance the generalisation performance. The twin support vector machine (TSVM) serves as an updated classification algorithm based on SVM, having a fast calculation speed. So we introduce Universum into TWSVM to improve the generalisation performance in the same way. Furthermore, in order to make the generalisation performance better in complex environment, an adaptive robust Adaboost-based twin support vector machine with universum learning (ARABUTWSVM) is put up in this paper. First of all, universum learning is used in TSVM to settle a matter of universum data. In order to make our method more robust, we use various loss functions selected by adaptive parameter θ and an update metric induced by correntropy method for distance measure. Further, so as to make further improvement of the learning effect of our method, the Adaboost method is embeded to ARABUTWSVM. The comprehensive experimentation has been performed on various datasets compared with previous universum learning methods, which show that ARABUTWSVM is an adaptive and robust method.