Ansari, Suhail MD MS, MS, FRCS; Rollins, James MD; Ebraheim, Nabil A. MD Author Information

We show that the e-commerce domain can provide all the right ingredients for successful data mining. We describe an integrated architecture for supporting this integration. The architecture can dramatically reduce the pre-processing, cleaning, and data understanding effort often documented to take 80% of the time in knowledge discovery projects. We emphasize the need for data collection at the application server layer (not the web server) in order to support logging of data and metadata that is essential to the discovery process. We describe the data transformation bridges required from the transaction processing systems and customer event streams (e.g., clickstreams) to the data warehouse. We detail the mining workbench, which needs to provide multiple views of the data through reporting, data mining algorithms, visualization, and OLAP. We conclude with a set of challenges.

We show that the e-commerce domain can provide allthe right ingredients for successful data mining andclaim that it is a killer domain for data mining. Wedescribe an integrated architecture, based on our experienceat Blue Martini Software, for supporting thisintegration. The architecture can dramatically reducethe pre-processing, cleaning, and data understandingeffort often documented to take 80% of the time inknowledge discovery projects. We emphasize the needfor data collection at...