In 2008, Rocket Fuel's founders saw a gap in the digital advertising market. None of the existing players were building autonomous systems based on big data and artificial intelligence, but instead they were offering fairly simple technology and relying on human campaign managers to drive success. Five years later in 2013, Rocket Fuel had the best technology IPO of the year on NASDAQ, reported $240 million in revenue, and was ranked by accounting firm Deloitte as the #1 fastest-growing technology company in North America. Along the way we learned that it's okay to be bold in our expectations of what is possible with fully autonomous systems, we learned that mainstream customers will buy advanced technology if it's delivered in a familiar way, and we also learned that it's incredibly difficult to debug the complex "robot psychology" when a number of complex autonomous systems interact. We also had excellent luck and timing: as we were building the company, real-time ad impression-level auctions with machine-to-machine buying and selling became commonplace, and marketers became increasingly focused on delivering better results for their company and delivering better personalized and relevant digital experiences for their customers. The case study presentation will present a fast-paced overview of the business and technology context for Rocket Fuel at inception and at present, key learnings and decisions, and the road ahead.

When modeling a probability distribution with a Bayesian network, we are faced with the problem of how to handle continuous variables. Most previous work has either solved the problem by discretizing, or assumed that the data are generated by a single Gaussian. In this paper we abandon the normality assumption and instead use statistical methods for nonparametric density estimation. For a naive Bayesian classifier, we present experimental results on a variety of natural and artificial domains, comparing two methods of density estimation: assuming normality and modeling each conditional distribution with a single Gaussian; and using nonparametric kernel density estimation. We observe large reductions in error on several natural and artificial data sets, which suggests that kernel estimation is a useful tool for learning Bayesian models.

We present a new method for the induction of tree-structured recursive partitioning classifiers that use a neural network as the partitioning function at each node in the tree. Our technique is appropriate for pattern recognition tasks with many continuous inputs and a single multivalued nominal output. This paper presents two main contributions: 1) a novel objective function called soft entropy, which is used to train each neural net to give the optimal partitioning of the data, and 2) a novel but simple method for removing outliers called iterative re-filtering, which boosts performance on many datasets. These two ideas are presented in the context of a single learning system called SENNT-PIRE (Soft Entropy Neural Net Trees with Pruning and Iterative RE-filtering).

Data mining is an umbrella term referring to the process of discovering patterns in data, typically with the aid of powerful algorithms to automate part of the search. These methods come from disciplines such as statistics, machine learning (artiicial intelligence), pattern recognition, neural networks, and databases. Two data analysts with diierent heritages might approach a similar problem quite diier-ently. In particular, this paper shows how the same problem, prostate cancer detection, is approached by an M.D. and a data mining analyst with a background in machine learning: hypothesis testing by the former, and bagged classiication models by the latter. We then survey medical data analysis literature , describing the common statistical methods employed by physicians for clinical studies, and the advances in data mining and machine learning research that have been motivated by medical data analysis.

Successful technology becomes invisible. Few people think much about internal combustion engines while they drive to work in three-thousand-pound hunks of metal powered by them, or electricity while it enables countless parts of their modern lives. Data mining has a long way to go before it succeeds in this way -- or does it? At KDD-98, the Behind-the-Scenes Data Mining panel presented five views on this aspect of the success of data mining. George H. John, Data Mining Guru at E.piphany, moderated the panel which included Usama Fayyad, Senior Researcher at Microsoft; Elliot Fishman, Director of Product Management at DoubleClick; Gerald Fahner, Project Analyst at Fair Isaac & Co.; and Paul DuBose, CTO of Analytika. The story below is a fictional amalgam of their presentations, seen from the viewpoint of an average citizen -- a day in the life of John Q. Record.

In the feature subset selection problem, a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention, while ignoring the rest. To achieve the best possible performance with a particular learning algorithm on a particular training set, a feature subset selection method should consider how the algorithm and the training set interact. We explore the relation between optimal feature subset selection and relevance. The wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain. We compare the wrapper approach to induction without feature subset selection and to Relief, a lter approach to feature subset selection. Improvement in accuracy is achieved for some datasets for the two families of induction algorithms used: decision trees and Naive-Bayes. In addition, the feature subsets selected by the wrapper are signiicantly smaller than the original subsets used by the learning algorithms, thus producing more comprehensible models. 1.1 INTRODUCTION In supervised machine learning, an induction algorithm is typically presented with a set of training instances, where each instance is described by a vector of feature (or attribute) values and a class label. For example, in medical diagnosis problems the features might include the age, weight, and blood pressure of a patient, and the class label might indicate whether or not a physician de-Signiicant parts of this chapter are reprinted from

In the feature subset selection problem, a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention, while ignoring the rest. To achieve the best possible performance with a particular learning algorithm on a particular training set, a feature subset selection method should consider how the algorithm and the training set interact. We explore the relation between optimal feature subset selection and relevance. Our wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain. We study the strengths and weaknesses of the wrapper approach and show a series of improved designs. We compare the wrapper approach to induction without feature subset selection and to Relief, a filter approach to feature subset selection. Significant improvement in accuracy is achieved for some datasets for the two families of induction algorithms used: decision trees and Naive-Bayes.

A current collector for connection to a current rail having current conductors in its side walls. The collector has a housing from which a coupling member having laterally directed electrical contact fingers projects. An element which screens the contact fingers and provides protection against touching the live parts of the device is rotatably mounted about the coupling member.

When mining large databases, the data extraction problem and the interface between the database and data mining algorithm become important issues. Rather than giving a mining algorithm full access to a database (by extracting to a flat file or other directly-accessible data structure), we propose the SQL Interface Protocol (SIP), which is a framework for interaction between a mining algorithm and a database. The data continues to reside entirely within the database management system (DBMS), but the query interface to the database gives the data mining algorithm sufficient information to discover the same patterns it would have found with direct access to the data. This model of interaction brings several advantages; for example, it allows a mining algorithm to be parallelized automatically just by using a parallelized DBMS to answer queries. We show how two families of mining algorithms may be implemented as "SIPpers," and we discuss related work in databases that should further enhance performance in the future.

The paper reports a preliminary investigation of the use of of modern data mining tools for mortgage scoring. Using IBM's Intelligent Miner (a data mining toolbox), the authors built a model of serious delinquency on a sample of data from Mortgage Information Corporation's Loan Performance System, which contains over 20 million loans with a volume of over $1.6 trillion. Currently, two technologies prevail in mortgage scoring: logistic regression, a very old and very simple method, and neural networks, newer and more complex types of models that can be extremely difficult to interpret. The radial basis function (RBF) algorithm in Intelligent Miner combines the mathematical complexity and generality of neural networks with a comprehensible visualization that explains the RBF model. Due to the performance and understandability of the RBF model, as well as other unique technologies not described, the Intelligent Miner should be a useful tool for mortgage bankers, facilitating development of customized systems for mortgage scoring and other mortgage banking applications.

We approach the problem of stock selection from the perspective ofknowledge discovery in databases: given a database of several years ofquarterly information on over a thousand companies, discover patternsin the data that will allow one to predict which stocks are likely to haveexceptional returns in the future. The database includes measures oftrends in the stocks" prices as well as fundamental data on the companies.For this task we employed the Recon system, which is able to...

We approach stock selection for long/short portfolios from the perspective of knowledge discovery in databases and rule induction: given a database of historical information on some universe of stocks, discover rules from the data that will allow one to predict which stocks are likely to have exceptionally high or low returns in the future. Long/short portfolios allow a fund manager to independently address value-added stock selection and factor exposure, and are a popular tool in financial engineering. For stock selection we employed the Recon system, which is able to induce a set of rules to model the data it is given. We evaluate Recon's stock selection performance by using it to build equitized long/short portfolios over eighteen quarters of historical data from October 1988 to March 1993, repeatedly using the previous four quarters of data to build a model which is then used to rank stocks in the current quarter. When trading costs were taken into account, Recon's equitized long/short portfolio had a total return of 277%, significantly outperforming the benchmark (S&P500), which returned 92.5% over the same period. We conclude that rule induction is a valuable tool for stock selection.

In the feature subset selection problem a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention while ignoring the rest To achieve the best possible performance with a particular learning algorithm on a particular training set a feature subset selection method should consider how the algorithm and the training set interact We explore the relation between optimal feature subset selection and relevance Our wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain We study the strengths and weaknesses of the wrapper approach and show a series of improved designs We compare the wrapper approach to induction without feature subset selection and to Relief a lter approach to feature subset selection Signi cant improvement in accuracy is achieved for some datasets for the two families of induction algorithms used decision trees and Naive Bayes

As data warehouses grow to the point where one hundred gigabytes is considered small, the computational efficiency of data-mining algorithms on large databases becomes increasingly important. Using a sample from the database can speed up the data-mining process, but this is only acceptable if it does not reduce the quality of the mined knowledge. To this end, we introduce the "Probably Close Enough" criterion to describe the desired properties of a sample. Sampling usually refers to the use of static statistical tests to decide whether a sample is sufficiently similar to the large database, in the absence of any knowledge of the tools the data miner intends to use. We discuss dynamic sampling methods, which take into account the mining tool being used and can thus give better samples. We describe dynamic schemes that observe a mining tool's performance on training samples of increasing size and use these results to determine when a sample is sufficiently large. We evaluate these sampling methods on data from the UCI repository and conclude that dynamic sampling is preferable.

High-quality financial databases have existed for many years, but human analysts can only scratch the surface of the wealth of knowledge buried in this data. Using the rule-induction technology in the Recon data-mining system, an investment strategy based purely on the learned rules can generate significant profits.

Discusses the weight update rule in the cascade correlation neural net learning algorithm. The weight update rule implements gradient descent optimization of the correlation between a new hidden unit's output and the previous network's error. The author presents a derivation of the gradient of the correlation function and shows that his resulting weight update rule results in slightly faster training. The author also shows that the new rule is mathematically equivalent to the one presented in the original cascade correlation paper and discusses numerical issues underlying the difference in performance. Since a derivation of the cascade correlation weight update rule was not published, this paper should be useful to those who wish to understand the rule