Models are specific units of work that have one job to perform: scoring new data to make predictions. Containers are self-contained workers that can be easily created, destroyed, and reused as needed. They are portable and easily integrate into numerous modern cloud and on-premises execution engines. SAS users can now follow a recipe to turn advanced model functions into on-demand containers such as Docker for both on-premises and cloud deployment. SAS Model Manager can be used to organize the model content from many sources, including SAS and open source, to create containers. This presentation presents the basics and shows you how to turn your SAS analytical models into modern containers.
Looking for new ways to improve your business? Try mining your own data! Event log data is a side product of information systems generated for audit and security purposes and is seldom analyzed, especially in combination with business data. Along with the cloud computing era, more event log data has been accumulated and analysts are searching for innovative ways to take advantage of all data resources in order to get valuable insights. Process mining, a new field for discovering business patterns from event log data, has recently proved useful for business applications. Process mining shares some algorithms with data mining but it is more focused on interpretation of the detected patterns rather than prediction. Analysis of these patterns can lead to improvements in the efficiency of common existing and planned business processes. Through process mining, analysts can uncover hidden relationships between resources and activities and make changes to improve organizational structure. This paper shows you how to use SAS® Analytics to gain insights from a real event log data.
It is well known that good initializations can improve the speed and accuracy of the solutions of many nonnegative matrix factorization (NMF) algorithms. Many NMF algorithms are sensitive with respect to the initialization of W or H or both. This is especially true of algorithms of the alternating least squares (ALS) type, including the two new ALS algorithms that we present in this paper. We compare the results of six initialization procedures (two standard and four new) on our ALS algorithms. Lastly, we discuss the practical issue of choosing an appropriate convergence criterion.
One of the key tasks in time series data mining is to cluster time series. However, traditional clustering methods focus on the similarity of time series patterns in past time periods. In many cases such as retail sales, we would prefer to cluster based on the future forecast values. In this paper, we show an approach to cluster forecasts or forecast time series patterns based on the Kullback-Leibler divergences among the forecast densities. We use the same normality assumption for error terms as used in the calculation of forecast confidence intervals from the forecast model. So the method does not require any additional computation to obtain the forecast densities for the Kullback-Leibler divergences. This makes our approach suitable for mining very large sets of time series. A simulation study and two real data sets are used to evaluate and illustrate our method. It is shown that using the Kullback-Leibler divergence results in better clustering when there is a degree of uncertainty in the forecasts.
Advances in computer technologies have enabled corporations to accumulate data at an unprecedented speed. Large-scale business data might contain billions of observations and thousands of features, which easily brings their scale to the level of terabytes. Most traditional feature selection algorithms are designed for a centralized computing architecture. Their usability significantly deteriorates when data size exceeds hundreds of gigabytes. High-performance distributed computing frameworks and protocols, such as the Message Passing Interface (MPI) and MapReduce, have been proposed to facilitate software development on grid infrastructures, enabling analysts to process large-scale problems efficiently. This paper presents a novel large-scale feature selection algorithm that is based on variance analysis. The algorithm selects features by evaluating their abilities to explain data variance. It supports both supervised and unsupervised feature selection and can be readily implemented in most distributed computing environments. The algorithm was developed as a SAS High-Performance Analytics procedure, which can read data in distributed form and perform parallel feature selection in both symmetric multiprocessing mode and massively parallel processing mode. Experimental results demonstrated the superior performance of the proposed method for large scale feature selection.
The invention relates to new pyridazino[4',5':3,4]-pyrrolo[2,1-a]isoquinolines of the formula (I) and the physiologically acceptable salts thereof with acids and complex-forming agents, wherein X is O, S or NHO and R1, R3, R4, R5, R6, R7, R8 and R9 are defined as in the specification, and pharmaceutical preparations containing these compounds.
Because the number of variables is often tremendous in data mining applications, variable selection or dimension reduction is essential to produce models with acceptable accuracy and generalization. After applying variable selection methods, data miners often consider only selected variables for their tasks. However, the input space represented by the rejected variables might still contain potential for positive contribution to the model. This paper demonstrates the use of projection methods to combine both the selected variable space and the rejected variable space. The new input space, which is made by adding projections of the rejected variables to the set of selected variables, reduces the loss of input variable information while keeping interpretability of important individual variables. The proposed method provides an effective means for analyzing the additional information potential that is hidden in the rejected terms. We also introduce a SAS ® Enterprise Miner™ extension node for random projection. This extension node enables easy creation of the new reduced input space.
Many customers face the issue of working with SAS®9 advanced analytics tools using data that comes from many different operational systems. The challenge presented to the customer is to extract, transform, and load the data into the analytical data structures with efficiency and scalability. This paper presents a case study describing practices to address the entire lifecycle of data access, aggregation, analysis, and deployment. A foundation discussion of how to interact with the Teradata database using SAS will be reviewed, followed by design and integration techniques available in SAS® Data Integration Studio to deliver data structures appropriate for analytical purposes. Finally, a presentation of how to leverage the data using SAS Enterprise Miner will be presented, including a discussion of techniques for managing algorithms that require large memory storage or multiple passes of the data.
In data mining, principal component analysis is a popular dimension reduction technique. It also provides a good remedy for the multicollinearity problem, but its interpretation of input space is not as good. To overcome the interpretation problem, principal components (cluster components) are obtained through variable clustering, which was implemented with PROC VARCLUS. The procedure uses oblique principal components analysis and binary iterative splits for variable clustering, and it provides non-orthogonal principal components. Even if this procedure sacrifices the orthogonal property among principal components, it provides good interpretable principal components and well-explained cluster structures of variables. However, the PROC VARCLUS implementation is inefficient to deal with high-dimensional data. We introduce the two-stage, variable clustering technique for large data sets. This technique uses global clusters, sub-clusters, and their principal components.
SAS released Enterprise Miner 5.3 in late 2007 with a veritable plethora of cool new features for data miners everywhere. Nearly every module of the software has been updated. New interactive data preparation tools make it easier to manipulate data and construct a sample for mining. For data exploration, Enterprise Miner now supports hierarchical market baskets to isolate interesting rules at different product category levels, multivariate graphical data exploration that persists a user’s interactive selections, a new scalable variable clustering node for dimension reduction, and more interactive user control over feature selection. Variable creation has been enhanced with a new interactive binning tool, an interactive rule building tool, and new transformation options. There are three new core predictive modeling techniques in Gradient Boosting, Support Vector Machines, and Partial Least Squares, along with a tool to make it easier to import models previously produced with SAS/STAT code. For model assessment, a new Cutoff node examines posterior probability distributions where users can enter cutoff values, and a new Reporter tool uses SAS ODS to produce reports spanning the entire analysis for printing and editing. The user interface is revised with more navigation controls, smarter property sheets, better graphics, and improved code editors. Users should see significant productivity gains from the software, and have even more fun data mining. INTRODUCTION SAS Enterprise Miner has been an industry-leading tool in the data mining field for nearly 10 years. This might lead you to believe that data mining products are in maintenance mode; however, that is most definitely not the case. On the contrary, the field of data mining is rapidly evolving to include new transactional and Web-based data sources; new applications such as social network analysis, rate making, and time series classification; and new modeling algorithms to detect global and local features. The latest release of Enterprise Miner contains a host of new productivity, statistical, interactive, and graphical tools designed to improve the productivity of the SAS data miner. This paper will focus on the new features in Enterprise Miner 5.3 with analytical examples. MIGRATION Before we can start data mining, we have to consider platforms and migration. Enterprise Miner 5.3 runs on SAS 9.1.3 Service Pack 4. Installation requires updates to the SAS Foundation, the SAS Analytics Platform, and the SAS Enterprise Miner client. Those users who need to preserve their Enterprise Miner 4.3 projects will find a new project conversion utility that moves all Enterprise Miner 4.3 diagrams into an Enterprise Miner 5.3 project. This function preserves the diagram structure, many of the node properties, and many of the tools results such as log and output listings, source and score code, and results tables needed for producing gains charts. The Enterprise Miner 4.3 result sets are visible inside the Enterprise Miner 5.3 Node Results window so that users can then run the diagrams in Enterprise Miner 5.3 and compare output. This will satisfy users’ needs to archive and retrieve their Enterprise Miner 4.3 results from within Enterprise Miner 5.3. Users of Enterprise Miner 5.2 will not need to perform any migration action because these projects are directly usable in Enterprise Miner 5.3. NEW FEATURES The Enterprise Miner 5.3 documentation and product literature provide a detailed list of new and enhanced features. That list is too lengthy to discuss in detail in this paper. Instead, we will focus on a few key features that will affect users in the areas of usability, graphical exploration, feature selection, variable binning, group processing and model building, and post processing. Data Mining and Predictive Modeling SAS Global Forum 2008
The widespread use of predictive analytics has enabled organizations to more accurately predict their business outcomes, improve business performance, and increase profitability. As the sheer number of these models in the overall portfolio is coupled with growing requirements to demonstrate external compliance, it is imperative that the organization implements sound model management practices. Model management is not a one-time activity but an essential business process. Models must be well developed and validated to demonstrate that they are working as expected. Outcomes analysis is also necessary to ensure that the scores derived from applying the model to new data are accurate and to verify that model performance remains satisfactory. This paper presents SAS-based strategies for effectively managing predictive models in a production environment and introduced a new product, SAS ® Model Manager 2.1 for SAS ® 9.1.3.
The widespread use of predictive analytics has enabled organizations to more accurately predict their business outcomes, improve business performance, and increase profitability. As the sheer number of these models in the overall portfolio is coupled with growing requirements to demonstrate external compliance, it is imperative that the organization implements sound model management practices. Model management is not a one-time activity but an essential business process. Models must be well developed and validated to demonstrate that they are working as expected. Outcomes analysis is also necessary to ensure that the scores derived from applying the model to new data are accurate and to verify that model performance remains satisfactory. This paper presents SAS-based strategies for effectively managing predictive models in a production environment and introduced a new product, SAS ® Model Manager 2.1 for SAS ® 9.1.3.
SAS Enterprise Miner 5.2 for SAS 9.1.3 provides many new enhancements to help both business analysts and statisticians carry out the data mining process more efficiently and with greater control and flexibility. A major focus of this release is to deliver new interactive statistical and visualization tools. The tool set has been expanded to include the new SOM/Kohonen, Decisions, and Replacement nodes. Major improvements have been made to nearly every other node. System administration has been enhanced through the use of the SAS Analytics Platform, which provides both thin-client distribution and server management functionality. Grid processing is now supported to manage the workload created by a large group of data miners. Customers will find many reasons to upgrade to SAS Enterprise Miner 5.2. INTRODUCTION SAS Enterprise Miner 5.2 is the SAS solution for data mining, providing unparalleled model development and deployment opportunities. Delivered as a distributed client-server system, Enterprise Miner is well suited for joint workgroup collaborations and large data mining applications. Enterprise Miner’s process flow diagram eliminates the need for manual coding and reduces the model development time for both business analysts and statisticians (see Figure 1). The system is customizable and extensible; users can integrate their code and build new nodes for redistribution. This paper provides an overview of the major enhancements of the latest release, Enterprise Miner 5.2 for SAS 9.1.3, delivered in November of 2005. Figure 1. SAS Enterprise Miner 5.2 Graphical User Interface. Projects are persisted on the analytical server enabling data miners to collaborate on the analyses. The process flow diagram is a self-documenting template that can be easily updated or applied to new problems and shared with other analysts. DATA VISUALIZATION AND MODIFICATIONS Data exploration and preparation are important data mining tasks used to reveal systematic patterns and derive new features to ultimately help the analyst better understand, analyze, and model the data. SAS Enterprise Miner 5.2 delivers new interactive statistical and visualization tools to help the data miner better search for trends and anomalies and prepare the data in a manner more useful for model development. Data Mining and Predictive Modeling SUGI 31
Predictive modeling tools such as SAS® Enterprise Miner™ generate score code that can be applied in business applications to produce recommendations. This paper describes a programming interface for scoring clients to invoke scoring services through message queues. The interface is applicable to both "real-time single-observation" scoring and "high-data-volume table-based batch" scoring. Readers learn how to invoke message queue-based scoring services from various scoring clients written in different programming languages such as C++, Java, and the SAS DATA step. Topics discussed include model identification, input data format, output data format, and error handling. The intended audience is data mining practitioners and IT professionals who are responsible for predictive model scoring in operational and business intelligence applications. The SAS DATA step, SAS SCL, and Java languages are used in code snippets; however, familiarity with those languages is not needed in order to grasp the intended points.
Many customers face the issue of working with SAS®9 advanced analytics tools using data that comes from many different operational systems. The challenge presented to the customer is to extract, transform, and load the data into the analytical data structures with efficiency and scalability. This paper presents a case study describing practices to address the entire lifecycle of data access, aggregation, analysis, and deployment. A foundation discussion of how to interact with the Teradata database using SAS will be reviewed, followed by design and integration techniques available in SAS® Data Integration Studio to deliver data structures appropriate for analytical purposes. Finally, a presentation of how to leverage the data using SAS Enterprise Miner will be presented, including a discussion of techniques for managing algorithms that require large memory storage or multiple passes of the data.
Many business elements are used to develop credit scorecards. Reject inference, related to the issue of sample bias, is one of the key processes required to build relevant application scorecards and is vital in creating successful scorecards. Reject inference is used to assign a target class (that is, a good or bad designation) to applications that were rejected by the financial institution and to applicants who refused the financial institution’s offer. This paper uses real-world data to present an example of using memorybased reasoning as a reject inference technique. SAS® Enterprise Miner™ software is used to perform the analysis. The paper discusses the technical concepts in reject inference and the methodology behind using memory-based reasoning as a reject inference technique. Several misclassification measures are reported to determine how well memory-based reasoning performs as a reject inference technique. In addition, a macro to determine how to pick the number of neighbors for the memory-based reasoning technique is given and discussed. This macro is implemented in a SAS® Enterprise Miner™ code node. OVERVIEW OF SCORECARDS Credit scorecard development is a method of modeling potential risk of credit applicants. It involves using different statistical techniques and past historical data to create a scorecard that financial institutions use to assess credit applicants in terms of risk. A scorecard model is built from a number of characteristic inputs. Each characteristic can have a number of attributes. In the example scorecard shown in Figure 1, age is a characteristic and “25<=AGE<33” is an attribute. Each attribute is associated with a number of scorecard points. These scorecard points are statistically assigned to differentiate risk, based on the predictive power of the variables, correlation between the variables, and business considerations.