Many organizations today are faced with the challenge of processing and distilling information from huge and growing collections of data. Such organizations are increasingly deploying sophisticated mathematical algorithms to model the behavior of their business processes to discover correlations in the data, to predict trends and ultimately drive decisions to optimize their operations. These techniques, are known collectively as analytics, and draw upon multiple disciplines, including statistics, quantitative analysis, data mining, and machine learning. In this survey paper, we identify some of the key techniques employed in analytics both to serve as an introduction for the non-specialist and to explore the opportunity for greater optimizations for parallelization and acceleration using commodity and specialized multi-core processors. We are interested in isolating and documenting repeated patterns in analytical algorithms, data structures and data types, and in understanding howthese could be most effectively mapped onto parallel infrastructure. To this end, we focus on analytical models that can be executed using different algorithms. For most major model types, we study implementations of key algorithms to determine common computational and runtime patterns. We then use this information to characterize and recommend suitable parallelization strategies for these algorithms, specifically when used in data management workloads.
Weak economic growth, increasing business complexity, and intensifying competition are among the motivations for the growing use of analytics across organizations of all types. In this business environment, enterprise leadership requires new tools and new insights to make better decisions. While analytic techniques, no matter how primitive, have been used to improve the effectiveness and efficiency of organizations for decades, if not centuries, capability and sophistication are now growing rapidly. Advanced analytics is proving to be a transformational technology for businesses, providing a source of competitive advantage and delivering significant, measurable outcomes. Risk management, sales-force productivity, workforce optimization, enterprise planning, and market intelligence are a few examples of core business processes that have significantly benefited from emerging analytical methodologies. As analytic tools used to transform business processes are integrated into enterprise IT (information technology) systems for use by senior business leaders, they are driving new innovations to address the size, speed, and complexity issues faced by large, global enterprises. This position paper explores emerging trends in the leadership application of analytics in global business organizations.
Automated procedures are described for discovering predictive rules from electronic health records. These patient records are structured, but are not collected relative to any targeted labels or study objectives. The learning methods cycle through all features, simulating labels and converting the problem from unlabeled learning to supervised classification and regression. Each feature in turn is processed as a simulated label, and a prediction is made from the remaining features. Using a decision-rule representation for knowledge extraction, machine learning techniques are applied to a large collection of electronic health records. Many rules are readily induced with significant predictive performance. By formulating the rules as queries to a web search engine, and then counting hit frequencies, we show how medical researchers can assess and rank potential for new insight among a collection of empirically strong associations.
Secure coprocessors have traditionally been used as a keystone of a security subsystem, eliminating the need to protect the rest of the subsystem with physical security measures. With technological advances and hardware miniaturization they have become increasingly powerful. This opens up the possibility of using them for non traditional use. This paper describes a solution for privacy preserving data sharing and mining using cryptographically secure but resource limited coprocessors. It uses memory light data mining methodologies along with a light weight database engine with federation capability, running on a coprocessor. The data to be shared resides with the enterprises that want to collaborate. This system will allow multiple enterprises, which are generally not allowed to share data, to do so solely for the purpose of detecting particular types of anomalies and for generating alerts. We also present results from experiments which demonstrate the value of such collaborations.
In some classification problems the feature space is heterogeneous in that the best features on which to base the classification are different in different parts of the feature space. In some other problems the classes can be divided into subsets such that distinguishing one subset of classes from another and classifying examples within the subsets require very different decision rules, involving different sets of features. In such heterogeneous problems, many modeling techniques including decision trees, rules, and neural networks evaluate the performance of alternative decision rules by averaging over the entire problem space, and are prone to generating a model that is suboptimal in any of the regions or subproblems. Better overall models can be obtained by splitting the problem appropriately and modeling each subproblem separately.This paper presents a new measure to determine the degree of dissimilarity between the decision surfaces of two given problems, and suggests a way to search for a strategic splitting of the feature space that identifies regions with different characteristics. We illustrate the concept using a multiplexor problem, and apply the method to a DNA classification problem.
Of the many types of information technology potentially useful for risk management in e-business, we are interested in decision support systems that help generate business rules that manage risks, while fulfilling other business objectives. This issue can be naturally thought of as the problem of deciding which actions to take in order to minimize the expected risk (or cost). Numerous offthe-shelf statistical and data-mining methods can be used to assess the risks associated with possible actions. Viewed this way, risk management becomes a traditional decision theoretic problem, and as such there is a large body of work in related fields such as decision theory, statistics, machine learning, and data mining that can provide a solid theoretical basis for it. One aspect of the risk management problem that has not received proper attention is the intrinsically sequential nature of this problem. That is, it is hardly satisfactory to consider the risk of a single action in isolation, but rather
The issues of cross channel integration and customer life time value modeling are two of the most important topics surrounding customer relationship management (CRM) today. In the present paper, we describe and evaluate a novel solution that treats these two important issues in a unified framework of Markov Decision Processes (MDP). In particular, we report on the results of a joint project between IBM Research and Saks Fifth Avenue to investigate the applicability of this technology to real world problems. The business problem we use as a testbed for our evaluation is that of optimizing direct mail campaign mailings for maximization of profits in the store channel. We identify a problem common to cross-channel CRM, which we call the Cross-Channel Challenge, due to the lack of explicit linking between the marketing actions taken in one channel and the customer responses obtained in another. We provide a solution for this problem based on old and new techniques in reinforcement learning. Our in-laboratory experimental evaluation using actual customer interaction data show that as much as 7 to 8 per cent increase in the store profits can be expected, by employing a mailing policy automatically generated by our methodology. These results confirm that our approach is valid in dealing with the cross channel CRM scenarios in the real world.