Emergent interest in genomic research has resulted in the creation of huge biological sequence databases, however search and retrieval of relevant information from these databases takes a lot of processing time, when performed conventionally as size of databases containing DNA sequences is huge. Hence, providing an efficient searching mechanism is mandatory. In this paper we present an efficient search mechanism using Hashing techniques. Initially, the data is hashed and indexed according to different window sizes. During this process, we eliminate redundancies and only record patterns with distinct elements and provide them with corresponding hash values. During the search phase, the search string is checked for the size of the window and if it exceeds the maximum limit of 4, then it is divided. The first part is considered as the search string and the search is made. After the confirmation of the index, the strings that follow the current indexed string are matched with the search string and finally the confirmation is made. The simulation results show that the current methodology provides faster results, while occupying lesser memory.

Medical claims fraud is a major contributor to increased healthcare costs, but the negative impact can be lessened through effective fraud detection. In this paper, we combine Medicare provider utilization and payment data from 2012 to 2015 with corresponding fraud labels from the List of Ex-cluded Individuals/Entities (LEIE) database. We demonstrate the effectiveness of detecting Medicare fraud with a limited number of known perpetrators, leading to severe class imbalance. For each of the three selected specialties, we use random undersampling to create four class distributions. Random Forest and Logistic Regression learners are built and evaluated based on fraud detection performance. Good fraud detection is demonstrated through the use of random under-sampling, across three selected medical specialties. Statistically significant results are seen across the class distributions, with the 80:20 distribution having the best results. Overall, Random Forest (with either 100 or 500 trees), for each class distribution across all specialties, significantly outperforms Logistic Regression, with average AUC scores of 0.881 and 0.88, respectively.

Bagging ensemble techniques have been utilized effectively by practitioners in the field of bioinformatics to alleviate the problem of class imbalance and to improve the performance of classification models. However, many previous works have used bagging only with a single arbitrary number of iterations. In this study, we raise the question of what is the impact of altering the number of iterations/ensembles on the classification performance of bagging classifiers? To answer this question, we conducted an empirical study using four different choices of number of iterations (10, 20, 50, and 100) within the bagging algorithm, across 15 different imbalanced bioinformatics datasets. Our results indicate that the choice of 50 iterations performs slightly better than all others without any exception, but the difference in performance is statistically insignificant. Thus, we recommend bagging with 10 iterations because, it achieves quality classification results; additional iterations do not significantly improve performance; and, a smaller number of iterations would be computationally less costly. The unique contribution of this work is to examine the effects of the number of iterations on the classification performance of bagging classifiers in the context of imbalanced datasets in the bioinformatics field.

Tweet sentiment classification seeks to identify the emotional polarity of a tweet. One potential way to enhance classification performance is to include emoticons as features. Emoticons are representations of faces expressing various emotions in text. They are created through combinations of letters, punctuation marks and symbols, and are frequently found within tweets. While emoticons have been used as features for sentiment classification, the importance of their inclusion has not been directly measured. In this work, we seek to determine if the addition of emoticon features improves classifier performance. We also investigate how high dimensionality impacts the addition of emoticon features. We conducted experiments testing the impact of using emoticon features, both with and without feature selection. Classifiers are trained using four different learners and either emoticons, unigrams, or both as features. Feature selection was conducted using five filter based feature rankers with four feature subset sizes. Our results showed that the choice of feature set (emoticon, unigram or both) had no significant impact in our initial tests when using no feature selection; however, with any of the tested feature selection techniques, augmenting unigram features with emoticon features resulted in significantly better performance than unigrams alone. Additionally, we investigate how the addition of emoticons changes the top features selected by the rankers.

Due to the great increase in the amount of attacks that occur in computer networks, there is an increasing dependence on network intrusion detection systems which monitor and analyze the network data to detect attacks. In recent years, machine learning methods have been used to build predictive models for network intrusion detection. These methods are able to automatically extract patterns from the network data to build detection models. Defining proper features, which help models to better discriminate between normal and attack data, is a critical task. While network attacks vary widely, they share some commonalities. Many attacks, by their nature, are repetitive and exhibit behaviors different from normal traffic. Among these commonalities are self-similarity between attack packets, periodicity and repetition characteristics seen in the attack traffic. In this paper, we study the common behaviors between two different attack types, called RUDY and DNS Amplification attacks, in order to propose new features for building predictive models by using machine learning algorithms. We collected Netflow traffic from an operational ISP network. We introduce a concept called “session” derived from Netflow which incorporates both sides of a network communication to define a network instance. Features are extracted for each session. To demonstrate how the newly defined features work for the task of intrusion detection, we use these features to build intrusion detection models for the detection of RUDY attack, DNS Amplification attack and the combination of these two attacks. To build predictive models we apply four machine learning classification algorithms: two versions of a decision tree algorithm, Naïve Bayes and 5-Nearest Neighbor (5-NN) algorithm. Our results show that the proposed features based on the attack commonalities provide very good prediction results for the detection of two studied attacks on real network traffic.

Ensemble learning is a powerful tool that has shown promise when applied towards bioinformatics datasets. In particular, the Random Forest classifier has been an effective and popular algorithm due to its relatively good classification performance and its ease of use. However, Random Forest does not account for class imbalance which is known for decreasing classification performance and increasing bias towards the majority class. In this study, we seek to determine if the inclusion of data sampling will improve the performance of the Random Forest classifier. In order to test the effect of data sampling, we used Random Undersampling along with two post-sampling class distribution ratios: 35: 65 and 50: 50 (minority: majority). Additionally, we also built inductive models with Random Forest when no data sampling technique was applied, so we can observe the true effect of the data sampling. All three options were tested on a series of fifteen imbalanced bioinformatics datasets. Our results show that data sampling does improve the classification performance of Random Forest, especially when using the 50: 50 post-sampling class distribution ratio. However, statistical analysis shows that the increase in performance is not statistically significant. Thus, we can state that while data sampling does improve the classification performance of Random Forest, it is not a necessary step as the classifier is fairly robust to imbalanced data on its own.

Compared to common DoS/DDoS attacks that are destructive and generate massive traffic, the application layer DoS attacks can be slow-and-low which means they occur at a slow rate and do not generate a massive amount of traffic. These attacks appear legitimate in terms of the protocol rules and rates. These characteristics make the detection of these attacks difficult. In this paper, we study the RUDY (R-U-DeadYet) attack which is one of the slow-and-low application layer attack types. RUDY attacks can bring down a server by creating long POST HTTP form submissions to the server at a very slow rate which results in application threads at the server side becoming stuck. The mitigation methods against RUDY attacks are mostly host-based. In this paper, we use a machine learning approach for the detection of RUDY attacks as well as determining the important features for their detection at the network level. The network level detection is scalable and it provides detection for hosts that do not have their own detection mechanism. We extract features from bi-directional instances of the network traffic. We then use an ensemble feature selection approach containing 10 different feature ranker methods in order to extract the most important features for the detection of RUDY attacks at the network level.

Defect prediction is an important process activity frequently used for improving the quality and reliability of software products. Defect prediction results provide a list of fault-prone modules which are necessary in helping project managers better utilize valuable project resources. In the software quality modeling process, high dimensionality and class imbalance are the two potential problems that may exist in data repositories. In this study, we investigate three data preprocessing approaches, in which feature selection is combined with data sampling, to overcome these problems in the context of software quality estimation. These three approaches are: Approach 1 - sampling performed prior to feature selection, but retaining the unsampled data instances; Approach 2 - sampling performed prior to feature selection, retaining the sampled data instances; and Approach 3 - sampling performed after feature selection. A comparative investigation is presented for evaluating the three approaches. In the experiments, we employed three sampling methods (random undersampling, random oversampling, and synthetic minority oversampling), each combined with a filter-based feature subset selection technique called correlation-based feature selection. We built the defect prediction models using five common classification algorithms. The case study was based on software metrics and defect data collected from multiple releases of a real-world software system. The results demonstrated that the type of sampling methods used in data preprocessing significantly affected the performance of the combination approaches. It was found that when the random undersampling technique was used, Approach 1 performed better than the other two approaches. However, when the feature selection technique was used in conjunction with an oversampling method (random oversampling or synthetic minority oversampling), we strongly recommended Approach 3.

Class imbalance is a significant challenge that practitioners in the field of bioinformatics are faced with on a daily basis. It is a phenomenon that occurs when number of instances of one class is much greater than number of instances of the other class(es) and it has adverse effects on the performance of classification models built on this skewed data. Random Forest as a robust classifier has been utilized effectively to deal with challenging characteristics of imbalanced bioinformatics datasets. In this study, we seek the answer to the question, do alterations to the bootstrapping process within Random Forest improve its classification performance? Thus, we performed an experimental study using Random Forest with four bootstrapping approaches, including two new novel bootstrapping approaches, across 15 imbalanced bioinformatics datasets. Our results demonstrate that two of the bootstrapping approaches, including one of our proposed approaches, outperform other approaches, however, this difference is statistically insignificant. We conclude that Random Forest is a robust classifier, able to handle the challenge of class imbalance, and can be slightly improved by altering bootstrapping process. To the best of our knowledge, no previous work has studied the effects of multiple bootstrapping processes on the performance of Random Forest in the domain of bioinformatics. In addition, we proposed and implemented the two innovative bootstrapping approaches evaluated in this paper.

The basic measurements for software quality control and management are the various project and software metrics collected at various states of a software development life cycle. The software metrics may not all be relevant for predicting the fault proneness of software components, modules, or releases. Thus creating the need for the use of feature (software metric) selection. The goal of feature selection is to find a minimum subset of attributes that can characterize the underlying data with results as well as, or even better than the original data when all available features are considered. As an example of inter-disciplinary research (between data science and software engineering), this study is unique in presenting a large comparative study of wrapper-based feature (or attribute) selection techniques for building defect predictors. In this paper, we investigated thirty wrapper-based feature selection methods to remove irrelevant and redundant software metrics used for building defect predictors. In this study, these thirty wrappers vary based on the choice of search method (Best First or Greedy Stepwise), leaner (Naive Bayes, Support Vector Machine, and Logistic Regression), and performance metric (Overall Accuracy, Area Under ROC (Receiver Operating Characteristic) Curve, Area Under the Precision-Recall Curve, Best Geometric Mean, and Best Arithmetic Mean) used in the defect prediction model evaluation process. The models are trained using the three learners and evaluated using the five performance metrics. The case study is based on software metrics and defect data collected from a real world software project.The results demonstrate that Best Arithmetic Mean is the best performance metric used within the wrapper. Naive Bayes performed significantly better than Logistic Regression and Support Vector Machine as a wrapper learner on slightly and less imbalanced datasets. We also recommend Greedy Stepwise as a search method for wrappers. Moreover, comparing to models built with full datasets, the performances of defect prediction models can be improved when metric subsets are selected through a wrapper subset selector.

Software defect prediction is a classification technique that utilizes software metrics and fault data collected during the software development process to identify fault-prone modules before the testing phase. It aims to optimize project resource allocation and eventually improve the quality of software products. However, two factors, high dimensionality and class imbalance, may cause low quality training data and subsequently degrade classification models. Feature (software metric) selection and data sampling are frequently used to overcome these problems. Feature selection (FS) is a process of choosing a subset of relevant features so that the quality of prediction models can be maintained or improved. Data sampling alters the dataset to change its balance level, therefore alleviating the problem of traditional classification models that are biased toward the overrepresented (majority) class. A recent study shows that another method, called boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models), is also effective for addressing the class imbalance problem. In this paper, we present a technique that uses FS followed by a boosting algorithm in the context of software quality estimation. We investigate four FS approaches: individual FS, repetitive sampled FS, sampled ensemble FS, and repetitive sampled ensemble FS, and study the impact of the four approaches on the quality of the prediction models. Ten base feature ranking techniques are examined in the case study. We also employ the boosting algorithm to construct classification models with no FS and use the results as the baseline for further comparison. The empirical results demonstrate that (1) FS is important and necessary prior to the learning process; (2) the repetitive sampled FS method generally has similar performance to the individual FS technique; and (3) the ensemble filter (including sampled ensemble filter and repetitive sampled ensemble filter) performs better than or similarly to the average of the corresponding individual base rankers.

Defect prediction is very challenging in software development practice. Classification models are useful tools that can help for such prediction. Classification models can classify program modules into quality-based classes, e.g. fault-prone (fp) or not-fault-prone (nfp). This facilitates the allocation of limited project resources. For example, more resources are assigned to program modules that are of poor quality or likely to have a high number of faults based on the classification. However, two main problems, high dimensionality and class imbalance, affect the quality of training datasets and therefore classification models. Feature selection and data sampling are often used to overcome these problems. Feature selection is a process of choosing the most important attributes from the original dataset. Data sampling alters the dataset to change its balance level. Another technique, called boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models), is found to also be effective for resolving the class imbalance problem.In this study, we investigate an approach for combining feature selection with this ensemble learning (boosting) process. We focus on two different scenarios: feature selection performed prior to the boosting process and feature selection performed inside the boosting process. Ten individual base feature ranking techniques, as well as an ensemble ranker based on the ten, are examined and compared over the two scenarios. We also employ the boosting algorithm to construct classification models without performing feature selection and use the results as the baseline for further comparison. The experimental results demonstrate that feature selection is important and needed prior to the learning process. In addition, the ensemble feature ranking method generally has better or similar performance than the average of the base ranking techniques, and more importantly, the ensemble method exhibits better robustness than most base ranking techniques. As for the two scenarios, the results show that applying feature selection inside boosting performs better than using feature selection prior to boosting.

Bioinformatics datasets contain a number of characteristics, such as noisy data and difficult to learn class boundaries, which make it challenge to build effective predictive models. One option for improving results is the use of ensemble learning methods, which involve combining the results of multiple predictive models into a single decision. Since we do not rely on a single model, we reduce the effect of any hidden bias which may reside in a single model. In this study, we investigate two ensemble learning methods, Select-Bagging and Random Forest, to find which one is better suited for classification of bioinformatics data. In addition, we examine how the choice of learning algorithms affects the classification results of the Bagging method. We conduct an empirical study using six ensemble classifiers (Random Forest and Select-Bagging utilizing five different classifiers) applied to 12 balanced datasets using three feature rankers along with four feature subset sizes. Based on our results, including statistical analysis, we recommend Random Forest as it is competitive with the best of the Select-Bagging classifiers, and does not require an additional choice of classifier, which can significantly affect classification performance. To our knowledge, this work is unique in terms of investigating the effectiveness of these two ensemble learning methods in the domain of bioinformatics as well as examining how the choice of classifier impacts classification results when using a Bagging-based ensemble learning method.

Performing sentiment analysis of tweets by training a classifier is a challenging and complex task, requiring that the classifier can correctly and reliably identify the emotional polarity of a tweet. Poor data quality, due to class imbalance or mislabeled instances, may negatively impact classification performance. Ensemble learning techniques combine multiple models in an attempt to improve classification performance, especially on poor quality or imbalanced data, however, these techniques do not address the concern of high dimensionality present in tweets sentiment data and may require a prohibitive amount of resources to train on high dimensional data. This work addresses these issues by studying bagging and boosting combined with feature selection. These two techniques are denoted as Select-Bagging and Select-Boost, and seek to address both poor data quality and high dimensionality. We compare the performance of Select-Bagging and Select-Boost against feature selection alone. These techniques are tested with four base learners, two datasets and ten feature subset sizes. Our results show that Select-Boost offers the highest performance, is significantly better than using no ensemble technique, and is significantly better than Select-Bagging for most learners on both datasets. To the best of our knowledge, this is the first study to focus on the effects of using ensemble learning in combination with feature selection for the purpose of tweet sentiment classification.

Sentiment analysis of tweets is a popular method of opinion mining social media. Many machine learning techniques exist that can improve the performance of classifiers trained to determine the sentiment or emotional polarity of a tweet; however, they are designed with different objectives and it is unclear which techniques are most beneficial. Additionally, these techniques may behave differently depending on quality of data issues, such as class imbalance, a common problem when using real world data. In an effort to determine which techniques are more important, we tested 12 techniques consisting of: eight feature selection techniques, bagging, boosting and data sampling with two post sampling class ratios. Using five base learners, we compare these techniques against each other and each base learners with no additional technique. We train and test each classifier on a balanced dataset and two imbalanced datasets with different class ratios. Additionally, we conduct statistical tests to determine if the differences observed between techniques are significant. Our results show that bagging and seven of the eight feature selection techniques significantly improve performance (compared to using no technique) on all three datasets, while boosting and data sampling are less beneficial for imbalanced tweet sentiment data. To the best of our knowledge, this is the first study comparing these three types of techniques on tweet sentiment data and the first to show that feature selection and ensemble techniques perform better than data sampling on tweet sentiment data.

Bioinformatics datasets contain many challenging characteristics, such as class imbalance, which adversely impacts the performance of supervised classification models built on these datasets. Techniques such as ensemble learning and data sampling from the domain of data mining can be deployed to alleviate the problem and to improve the classification performance. In this study, we sought to seek whether inclusion of data sampling within the ensemble framework can further improve the performance of classification models. To this end, we performed an experimental study using two newly hybrid ensemble techniques, one integrates feature selection within the boosting process and the other incorporates random under-sampling followed by feature selection within the boosting framework, two learners, three forms of feature rankers, and four feature subset sizes on 15 highly imbalanced bioinformatics datasets. Our results and statistical analysis demonstrate that the difference between the two boosting methods is statistically insignificant. Therefore, as the inclusion of data sampling has no significant positive effect on the performance of ensemble classifiers, it is not required to achieve maximum classification performance. To our knowledge, this is the first empirical study that examined the effects of data sampling, random under-sampling, to enhance classification performance of boosting algorithm for highly imbalanced bioinformatics data.

One of the major challenges when working with software metrics datasets is that some metrics may be redundant or irrelevant to software defect prediction. This may be addressed using feature (metric) selection, which chooses an appropriate subset of features for use in downstream computation. There are three major forms of feature selection: filter-based feature rankers, which uses statistical measures to assign a score to each feature and present the user with a ranked list; filter-based subset evaluation, which uses statistical measures on feature subsets to find the best choice; and wrapper-based subset selection, which builds classification models using different subsets to find the one which maximizes performance. Software practitioners are interested in which feature selection methods are best at providing the most stable feature subset in the face of changes to the data (here, the addition or removal of instances). In this study we select feature subsets using fifteen feature selection methods and then use our newly proposed Average Pairwise Tanimoto Index (APTI) to evaluate the stability of feature selection methods. We evaluate the stability of feature selection methods on a pair of subsamples generated by fixed-overlap partitions algorithm. Four different levels of overlap are considered in this study. Four software metric datasets from a real-world software project are used in this study. Results demonstrate that ReliefF (RF) is the most stable feature selection method and wrapper based feature subset selection shows least stability. In addition, as the overlap of partitions increased, the stability of the feature selection strategies increased.

Sentiment classification of tweets is used for a variety of social sensing tasks and provides a means of discerning public opinion on a wide range of topics. A potential concern when performing sentiment classification is that the training data may contain class imbalance, which can negatively affect classification performance. A classifier trained on imbalanced data may be biased in favor of the majority class. One possibile method of addressing this is to use data sampling to achieve a more balanced class distribution. In this work, we seek to observe how data sampling (using random undersampling with either a 50:50 or 35:65 positive: negative post-sampling class distribution ratio) affects the classification performance on tweet sentiment data. Our experimental results show that Random Undersampling significantly improves classification performance in comparison to not using any data sampling. Furthermore, there is no significant difference between selecting a 50:50 or 35:65 post-sampling class distribution ratio.

The main goal of software quality engineering is to produce a high-quality software product through the use of various techniques and processes. Classification models are effective tools for software quality prediction, helping practitioners to detect potentially problematic modules and eventually improve software product. However, two potential problems, high dimensionality and class imbalance, may affect the classifiers performance. In this study, we propose a data pre-processing approach, in which feature selection is combined with data sampling, to overcome these problems. We investigate two filter-based feature subsets selection techniques, i.e., correlation-based and consistency-based subset evaluation methods, and three data sampling methods, i.e., random undersampling, random oversampling, and synthetic minority oversampling. We are interested in exploring the effect of the various feature selection techniques, sampling methods, and their interactions on the performance of classification models. The empirical studies were carried out on 13 datasets from two real-world software systems. The results demonstrate that the correlation-based subset evaluation technique outperformed the consistency-based method when they were used along with a random sampling method and when the training data had a high degree of class imbalance; however, when synthetic minority oversampling was employed or when the training dataset was less imbalanced, the consistency-based technique had better performance than the correlation-based approach.

One major challenge posed by bioinformatics datasets is class imbalance which occurs when one class has many more instances than the other class(es). Its undesirable effect on the classification performance is compounded with the fact that, in general, the class with fewer instances is the class of interest. Bagging has been utilized by practitioners in the field to overcome the challenge of class imbalance and to improve the classification performance. Our motivation for this study is to investigate whether changes to the bootstrapping step of bagging classifiers can further improve their performance. Specifically, these modifications to the bootstrapping process take into account the membership of the classes. We performed an extensive empirical study utilizing four bootstrap approaches within bagging framework using three feature rankers along with four feature subset sizes and two base classifiers across 15 imbalanced bioinformatics datasets. Three of these bootstrap approaches were proposed and implemented by our research team for this study. Our results show that all new approaches improve performance over the classic bootstrap approach, with balanced bagging having the highest performance, however, observed increases in performance are not statistically significant. We recommend the balanced bootstrap approach because it shows the most improvement, in terms of frequency of having the highest performance, and it generates fully balanced bootstrap datasets that can account for the class imbalance problem. The uniqueness of this paper is proposing and implementing the three innovative bootstrapping approaches to examine the effects of these bootstrapping processes against the classic one on the performance of bagging classifiers in the domain of bioinformatics.