The labeling process for supervised learning is costly and time-consuming, and is often impractical to scale due to real-world constraints. Active learning (AL) addresses this challenge by strategically selecting representative and informative data points to reduce labeling efforts. This paper focuses on an AL scenario in which only a very limited number of labels can be acquired. We propose an algorithm operating in two phases: (1) an exploration phase that prioritizes representative and diverse data points using density-driven criteria, and (2) an exploitation phase that combines predictive uncertainty with density weighting to select informative samples from densely populated regions. This enhances both representativeness and informativeness. Our results demonstrate significant improvements in model quality compared to other algorithms typically employed for this scenario, across various scenarios involving imbalanced data in classification tasks and skewness in regression tasks. Through this work, we aim to provide a new algorithm for this scenario and investigate general principles for AL. While most AL studies focus on either classification or regression, our work applies the algorithms to both. Therefore, we can analyze the differences between classification and regression problems and their effects on AL strategies. Furthermore, we explore different categories of AL criteria and their effectiveness in the low-budget regime. These results also provide insight into the cold-start problem, which involves selecting an initial labeled set and is faced by many model-based AL methods.

In recent years, the rapid growth of Internet of Things (IoT) devices, the popularity of social media platforms, and user interactions in various online environments, such as streaming platforms and mobile applications, have led to a significant increase in the amount of data generated. However, as this data is raw and unlabeled, its value for training supervised machine learning models is still limited. The challenge is further complicated by the costly and time-consuming process of manual labeling. Active learning (AL) provides a solution by selectively labeling small but highly informative and representative subset of data points to enable better generalization and improve model performance on unseen data for both classification and regression. The use of AL techniques spans both classification and regression frameworks, each facing different challenges such as binary or multi-class settings, balanced and imbalanced datasets, and the presence of outliers. Such diversity necessitates fair and representative metrics tailored to each AL scenario. Despite this need, some evaluation metrics remain underexplored, and many studies consider only limited perspectives, often evaluating performance without accounting for the representativeness of the selected labeled data. Furthermore, some studies have used inappropriate or limited metrics. This motivates our investigation into fairer evaluation metrics for AL algorithms. In this study, we review current AL evaluation metrics, highlight underexplored but emerging ones, and outline key research questions. We also aim to highlight previously unexplored evaluation metrics that could provide valuable insights into evaluating active learners from perspectives not addressed by traditional metrics.

Pretrained cell segmentation models have simplified and accelerated microscopy image analysis, but they often perform poorly on challenging datasets. Although these models can be adapted to new datasets with only a few annotated images, the effectiveness of fine-tuning depends critically on which images are selected for annotation. To address this, we propose MDMR (Maximum Diversity Minimum Redundancy), a novel algorithm that selects the most informative subset of images by explicitly balancing diversity and redundancy in feature space. We evaluate MDMR under an extremely low annotation budget of just two images per dataset for fine-tuning the pretrained Cellpose Cyto2 model on four different 2D datasets from the Cell Tracking Challenge. MDMR consistently outperforms six competitive active learning and subset selection methods and approaches the performance of fully-supervised finetuning. The results show that explicitly balancing diversity and redundancy enables stable and annotation-efficient adaptation of pretrained cell segmentation models. Code is publicly available at: . ### Competing Interest Statement The authors have declared no competing interest. Ministry of Culture and Science of the State of North Rhine-Westphalia, Germany, NW21-059B

Recently, it has been challenging to generate enough labeled data for supervised learning models from a large amount of free unlabeled data due to the high cost of the labeling process. Here, the active learning technique provides a solution by annotating a small but highly informative set of unlabeled data. This ensures high generalizability in space and improves classification performance with test data. The task is more challenging when the query budget is small, the data is imbalanced, multiple classes are present, and no predefined knowledge is available. To address these challenges, we present a novel active learner geometrically based on principal component analysis (PCA) and linear discriminant analysis (LDA). The proposed active learner consists of two phases: The PCA-inspired exploration phase, in which regions with high variances are explored, and the LDA-inspired exploitation phase, in which boundary points between classes are selected. The proposed geometric strategy improves the search capabilities of the active learner, allowing it to explore the space of minority classes even with multiple minority classes and a small query budget. Experiments on synthetic and real binary and multi-class imbalanced data show that the proposed algorithm has significant advantages over multiple known active learners.

Recently, the massive growth of IoT devices and Internet data, which are widely used in many applications, including industry and healthcare, has dramatically increased the amount of free unlabeled data collected. However, this unlabeled data is useless if we want to learn supervised machine learning models. The expensive and time-consuming cost of labeling makes the problem even more challenging. Here, the active learning (AL) technique provides a solution by labeling small but highly informative and representative data, which guarantees a high degree of generalizability over space and improves classification performance with data we have never seen before. The task is more difficult when the active learner has no predefined knowledge, such as initial training data, and when the obtained data is incomplete (i.e., contains missing values). In previous studies, the missing data should first be imputed. Then, the active learner selects from the available unlabeled data, regardless of whether the points were originally observed or imputed. However, selecting inaccurate imputed data points would negatively affect the active learner and prevent it from selecting informative and/or representative points, thus reducing the overall classification performance of the prediction models. This motivated us to introduce a novel query selection strategy that accounts for imputation uncertainty when querying new points. For this purpose, we first introduce a novel multiple imputation method that considers feature importance in selecting the most promising feature groups for missing values estimation. This multiple imputation method provides the ability to quantify the imputation uncertainty of each imputed data point. Furthermore, in each of the two phases of the proposed active learner (exploration and exploitation), imputation uncertainty is taken into account to reduce the probability of selecting points with high imputation uncertainty. We tested the effectiveness of the proposed active learner on different binary and multiclass datasets with different missing rates.

Despite the availability and ease of collecting a large amount of free, unlabeled data, the expensive and time-consuming labeling process is still an obstacle to labeling a sufficient amount of training data, which is essential for building supervised learning models. Here, with low labeling cost, the active learning (AL) technique could be a solution, whereby a few, high-quality data points are queried by searching for the most informative and representative points within the instance space. This strategy ensures high generalizability across the space and improves classification performance on data we have never seen before. In this paper, we provide a survey of recent studies on active learning in the context of classification. This survey starts with an introduction to the theoretical background of the AL technique, AL scenarios, AL components supported with visual explanations, and illustrative examples to explain how AL simply works and the benefits of using AL. In addition to an overview of the query strategies for the classification scenarios, this survey provides a high-level summary to explain various practical challenges with AL in real-world settings; it also explains how AL can be combined with various research areas. Finally, the most commonly used AL software packages and experimental evaluation metrics with AL are also discussed.

Despite the availability of a large amount of free unlabeled data, collecting sufficient training data for supervised learning models is challenging due to the time and cost involved in the labeling process. The active learning technique we present here provides a solution by querying a small but highly informative set of unlabeled data. It ensures high generalizability across space, improving classification performance with test data that we have never seen before. Most active learners query either the most informative or the most representative data to annotate them. These two criteria are combined in the proposed algorithm by using two phases: exploration and exploitation phases. The former aims to explore the instance space by visiting new regions at each iteration. The second phase attempts to select highly informative points in uncertain regions. Without any predefined knowledge, such as initial training data, these two phases improve the search strategy of the proposed algorithm so that it can explore the minority class space with imbalanced data using a small query budget. Further, some pseudo-labeled points geometrically located in trusted explored regions around the new labeled points are added to the training data, but with lower weights than the original labeled points. These pseudo-labeled points play several roles in our model, such as (i) increasing the size of the training data and (ii) decreasing the size of the version space by reducing the number of hypotheses that are consistent with the training data. Experiments on synthetic and real datasets with different imbalance ratios and dimensions show that the proposed algorithm has significant advantages over various well-known active learners.

The population initialization step is a common step in the majority (or even all) of evolutionary algorithms (EAs). There are many population initialization techniques. Due to the limited population size and the high dimensionality of many problems, there is little chance to cover the promising regions in the search space. From different perspectives, this paper compares the stochastic and deterministic population initialization techniques through comparing five of the well-known population initializers: Random number generator (RNG), Latin Hypercube, Sobol, Halton, and Kronecker. Due to the presence of many constraints in real-world applications, in this paper, we are focusing only on single-objective constrained optimization problems. Specifically, the goal is to investigate if there is a significant difference between these population initialization methods. In this paper, we explain theoretically and mathematically these different population initialization techniques. Moreover, different illustrative examples and visualizations are introduced to explain the behavior of each technique and compare different techniques from different perspectives. The results show that due to the high uniformity of the low-discrepancy sequences such as the Halton and Sobol sequences, the generated points using these sequences are more evenly distributed over the space than RNG, which is the commonly used technique for initializing the populations in EAs. Practically, using a set of benchmark functions, we investigate the use of each population initialization technique for initializing different population-based evolutionary algorithms. The results of our experiments prove that with sufficient numbers of iterations, the EAs are not sensitive to the initialization methods and there are no significant differences between the mentioned population initialization methods. Further, the low discrepancy methods enhance the exploration ability of EAs in early iterations.

Optimization algorithms are widely employed for finding optimal solutions in many applications. Stochastic optimization algorithms including nature-inspired optimization algorithms are simple and easy to implement, and this is the reason why there is a growing interest in this research area. Recently, many nature-inspired optimization algorithms have been proposed for solving many optimization problems. Moreover, with the aim of improving the performance of optimization algorithms, some modifications were applied such as combining different algorithms and employing some sampling techniques for replacing critical parameters in the optimization algorithms. This paper compares five different widely used PSO-style optimization algorithms to investigate if there is a significant difference between them or not. Theoretically, we explain different PSO-style algorithms and discuss the similarities and differences between them. Practically, a number of experiments were conducted to compare these algorithms. Theoretical analysis and practical results prove that there is not any significant difference between the PSO-style algorithms regarding their performance.

Classification techniques have been applied to many applications in various fields of sciences. There are several ways of evaluating classification algorithms. The analysis of such metrics and its significance must be interpreted correctly for evaluating different learning algorithms. Most of these measures are scalar metrics and some of them are graphical methods. This paper introduces a detailed overview of the classification assessment measures with the aim of providing the basics of these measures and to show how it works to serve as a comprehensive source for researchers who are interested in this field. This overview starts by highlighting the definition of the confusion matrix in binary and multi-class classification problems. Many classification measures are also explained in details, and the influence of balanced and imbalanced data on each metric is presented. An illustrative example is introduced to show (1) how to calculate these measures in binary and multi-class classification problems, and (2) the robustness of some measures against balanced and imbalanced data. Moreover, some graphical measures such as Receiver operating characteristics (ROC), Precision-Recall, and Detection error trade-off (DET) curves are presented with details. Additionally, in a step-by-step approach, different numerical examples are demonstrated to explain the preprocessing steps of plotting ROC, PR, and DET curves.

The parameters of support vector machines (SVMs) such as the penalty parameter and the kernel parameters have a great impact on the classification accuracy and the complexity of the SVM model. Therefore, the model selection in SVM involves the tuning of these parameters. However, these parameters are usually tuned and used as a black box, without understanding the mathematical background or internal details. In this paper, the behavior of the SVM classification model is analyzed when these parameters take different values with balanced and imbalanced data. This analysis including visualization, mathematical and geometrical interpretations and illustrative numerical examples with the aim of providing the basics of the Gaussian and linear kernel functions with SVM. From this analysis, we proposed a novel search algorithm. In this algorithm, we search for the optimal SVM parameters into two one-dimensional spaces instead of searching into one two-dimensional space. This reduces the computational time significantly. Moreover, in our algorithm, from the analysis of the data, the range of kernel function can be expected. This also reduces the search space and hence reduces the required computational time. Different experiments were conducted to evaluate our search algorithm using different balanced and imbalanced datasets. The results demonstrated how the proposed strategy is fast and effective than other searching strategies.

The parameters of support vector machines (SVMs) such as kernel parameters and the penalty parameter have a great influence on the accuracy and complexity of the classification models. In the past, different evolutionary optimization algorithms were employed for optimizing SVMs; in this paper, we propose a social ski-driver (SSD) optimization algorithm which is inspired from different evolutionary optimization algorithms for optimizing the parameters of SVMs, with the aim of improving the classification performance. To cope with the problem of imbalanced data which is one of the challenging problems for building robust classification models, the proposed algorithm (SSD-SVM) was enhanced to deal with imbalanced data. In this study, eight standard imbalanced datasets were used for testing our proposed algorithm. For verification, the results of the SSD-SVM algorithm are compared with grid search, which is a conventional method of searching parameter values, and particle swarm optimization (PSO). The experimental results show that the SSD-SVM algorithm is capable of finding near-optimal values of SVMs parameters. The results also demonstrated high classification performance compared to the PSO algorithm.

Classificationtechniqueshavebeenappliedtomanyapplicationsinvariousfieldsofsciences.Thereareseveralwaysofevaluatingclassificationalgorithms.Theanalysisofsuchmetricsanditssignificancemustbeinterpretedcorrectlyforevaluatingdifferentlearningalgorithms.Mostofthesemeasuresarescalarmetrics andsomeofthemaregraphicalmethods.Thispaperintroducesadetailedoverviewoftheclassification assessmentmeasureswiththeaimofprovidingthebasicsofthesemeasuresandtoshowhowitworkstoserveasacomprehensivesourceforresearcherswhoareinterestedinthisfield.Thisoverviewstartsbyhighlightingthedefinitionoftheconfusionmatrixinbinaryandmulti-classclassificationproblems.Manyclassificationmeasuresarealsoexplainedindetails,andtheinfluenceofbalancedandimbalanceddataoneachmetricis presented.Anillustrativeexampleisintroducedtoshow(1)howtocalculatethesemeasuresinbinaryandmulti-classclassificationproblems,and(2)therobustnessofsomemeasuresagainstbalancedandimbalanced data.Moreover,somegraphicalmeasuressuchasReceiveroperatingcharacteristics(ROC),Precision-Recall,andDetectionerrortrade-off(DET)curvesarepresentedwithdetails.Additionally,inastep-by-stepapproach, differentnumericalexamplesaredemonstratedtoexplainthepreprocessingstepsofplottingROC,PR,andDETcurves.

BACKGROUND AND OBJECTIVES:Toxicity testing is an important step for developing new drugs, and animals are widely used in this step by exposing them to the toxicants. Zebrafishes are widely used for measuring and detecting the toxicity. However, measuring and testing toxicity manually is not feasible due to the large number of embryos. This work presents an automated model to investigate the toxicity of two toxicants (3, 4-Dichloroaniline (34DCA) and p-Tert-Butylphenol (PTBP)).METHODS:The proposed model consists of two steps. In the first step, a set of features is extracted from microscopic images of zebrafish embryos using the Segmentation-Based Fractal Texture Analysis (SFTA) technique. Secondly, a novel rough set-based model using Social ski-driver (SSD) is used to find a global minimal subset of features that preserves important information of the original features. In the third step, the AdaBoost classifier is used to classify an unknown sample to alive or coagulant after exposing the embryo to a toxic compound.RESULTS:For detecting the toxicity, the proposed model is compared with (i) three deterministic rough set reduction algorithms and (ii) the PSO-based algorithm. The classification performance rate of our model was ranged from 97.1% to 99.5% and it outperformed the other algorithms.CONCLUSIONS:The results of our experiments proved that the proposed drug toxicity model is efficient for rough set-based feature selection and it obtains a high classification performance.

A supervised machine learning classifier can only be as good as the labeled training data. For this reason, there is a need for explicit human expert knowledge inside the workflow. Existing data collections often consist of classes different to the ones which are necessary for an individual application. Therefore, generating a new data set based on a predefined labeling guideline is mandatory. The aim of this work is to increase the quality of labeled data sets during their creation. We present a workflow for the labeling of unsorted data by a group of experts, including subsequent classifier training and evaluation. Even if combined with standard methods for feature extraction and classification, a performance improvement was achieved with the proposed labeling method. Furthermore, we offer access to our data set (German newspaper articles) including the labeling guideline as contribution to the research community.

Active learning receives great interest from researchers with the aim of reducing the amount of time, cost, and efforts for labeling data in many applications. Active learning aims to generate/select the smallest possible amount of training data that ensures strong classification performance in the test phase. An active learner carries out two main steps: (i) selecting a set of promising queries from unlabeled data, and (ii) annotating the selected queries. Most active learners choose either the most informative or representative instances for annotation. In this paper, we combined these two criteria for query selection. First, in the exploration phase, the proposed algorithm explores the search space and tries in each iteration to visit new regions for better exploration. This improves the capability of exploring the space of minority classes with imbalanced data. Second, in the exploitation phase, the goal is to generate a new point in an uncertain region, which is expected to be around the decision boundaries of the target functions. Some variants of the proposed algorithm do not require any labeled or unlabeled data in advance. There is only comparably few existing work which addresses this scenario. Experiments on synthetic and real datasets with different dimensions and imbalance ratios indicate that the proposed algorithm has significant advantages compared to various well-known active learners.

Inter-agent communication has been playing an important role in soccer simulation 2D since its introduction. Its primary usage has been to communicate with teammates in order to share state observations to fill gaps in the players’ world models, to announce near future actions like passes or requesting passes, as well as for sharing and synchronizing on locker room agreements. In this paper, by contrast, our focus is on the communication of the opponent team. We present an approach for wiretapping and decoding opponent communication and systematically evaluate its impact. Our main finding is that a team that wiretaps its opponent and exploits intercepted information appropriately, can boost its own playing performance significantly.

Measuring toxicity is an important step in drug development. However, the current experimental methods which are used to estimate the drug toxicity are expensive and need high computational efforts. Therefore, these methods are not suitable for large-scale evaluation of drug toxicity. As a consequence, there is a high demand to implement computational models that can predict drug toxicity risks. In this paper, we used a dataset that consists of 553 drugs that biotransformed in the liver. In this data, there are four toxic effects, namely, mutagenic, tumorigenic, irritant and reproductive effects. Each drug is represented by 31 chemical descriptors. This paper proposes two models for predicting drug toxicity risks. The proposed models consist of three phases. In the first phase, the most discriminative features are selected using rough set-based methods to reduce the classification time and improve the classification performance. In the second phase, three different sampling algorithms, namely, Random Under-Sampling, Random Over-Sampling, and Synthetic Minority Oversampling Technique (SMOTE) are used to obtain balanced data. In the third phase, the first proposed model employs the Neutrosophic Rule-based Classification System (NRCS), and the second model uses Genetic NRCS (GNRCS) to classify an unknown drug into toxic or non-toxic. The experimental results proved that the proposed models obtained high sensitivity (89-93%), specificity (91-97%), and GM (90-94%) for all toxic effects. Overall, the results of the proposed models indicate that it could be utilized for the prediction of drug toxicity in the early stages of drug development. (C) 2018 Elsevier Ltd. All rights reserved.

This paper is addressing the personal identification problem by using mobile-based keystroke dynamics of touch mobile phone. The proposed approach consists of two main phases, namely feature selection and classification. The most important features are selected using Genetic Algorithm (GA). Moreover, Bagging classifier used the selected features to identify persons by matching the features of the unknown person with the labeled features. The outputs of all Bagging classifiers are fused to determine the final decision. In this experiment, a keystroke dynamics database for touch mobile phones is used. The database, which consists of four sets of features, is collected from 51 individuals and consists of 985 samples collected from males and females with different ages. The results of the proposed model conclude that the third subset of features achieved the best accuracy while the second subset achieved the worst accuracy. Moreover, the fusion of all classifiers of all ensembles will improve the accuracy and achieved results better than the individual classifiers and individual ensembles.

Selecting the most discriminative features is a challenging problem in many applications. Bio-inspired optimization algorithms have been widely applied to solve many optimization problems including the feature selection problem. In this paper, the most discriminating features were selected by a new Chaotic Dragonfly Algorithm (CDA) where chaotic maps embedded with searching iterations of the Dragonfly Algorithm (DA). Ten chaotic maps were employed to adjust the main parameters of dragonflies’ movements through the optimization process to accelerate the convergence rate and improve the efficiency of DA. The proposed algorithm is employed for selecting features from the dataset that were extracted from the Drug bank database, which contained 6712 drugs. In this paper, 553 drugs that were bio-transformed into liver are used. This data have four toxic effects, namely, irritant, mutagenic, reproductive, and tumorigenic effect, where each drug is represented by 31 chemical descriptors. The proposed model is mainly comprised of three phases; data pre-processing, features selection, and the classification phase. In the data pre-processing phase, Synthetic Minority Over-sampling Technique (SMOTE) was used to solve the problem of the imbalanced dataset. At the features selection phase, the most discriminating features were selected using CDA. Finally, the selected features from CDA were used to feed Support Vector Machine (SVM) classifier at the classification phase. Experimental results proved the capability of CDA to find the optimal feature subset, which maximizing the classification performance and minimizing the number of selected features compared with DA and the other meta-heuristic optimization algorithms. Moreover, the experiments showed that Gauss chaotic map was the appropriate map to significantly boost the performance of DA. Additionally, the high obtained value of accuracy (81.82–96.08%), recall (80.84–96.11%), precision (81.45–96.08%) and F-Score (81.14–96.1%) for all toxic effects proved the robustness of the proposed model.