What is Data Mining? How can it be used to help managers make strategic

What is Data Mining? How can it be used to help managers make strategic decisions about business operations?

What do you understand by Data Mining (DM)? How is DM used by the managers to improve their decision-making ?

Ans.

Data mining, the extraction of hidden predictive information from large databases, is a powerful new technology with great potential to help companies focus on the most important information in their data warehouses. Data mining tools predict future trends and behaviors, allowing businesses to make proactive, knowledge-driven decisions. The automated, prospective analyses offered by data mining move beyond the analyses of past events provided by retrospective tools typical of decision support systems. Data mining tools can answer business questions that traditionally were too time consuming to resolve. They scour databases for hidden patterns, finding predictive information that experts may miss because it lies outside their expectations.

Most companies already collect and refine massive quantities of data. Data mining techniques can be implemented rapidly on existing software and hardware platforms to enhance the value of existing information resources, and can be integrated with new products and systems as they are brought on-line. When implemented on high performance client/server or parallel processing computers, data mining tools can analyze massive databases to deliver answers to questions such as, “Which clients are most likely to respond to my next promotional mailing, and why?”

This white paper provides an introduction to the basic technologies of data mining. Examples of profitable applications illustrate its relevance to today’s business environment as well as a basic description of how data warehouse architectures can evolve to deliver the value of data mining to end users.

The Foundations of Data Mining

Data mining techniques are the result of a long process of research and product development. This evolution began when business data was first stored on computers, continued with improvements in data access, and more recently, generated technologies that allow users to navigate through their data in real time. Data mining takes this evolutionary process beyond retrospective data access and navigation to prospective and proactive information delivery. Data mining is ready for application in the business community because it is supported by three technologies that are now sufficiently mature:

Massive data collection
Powerful multiprocessor computers
Data mining algorithms

Commercial databases are growing at unprecedented rates. A recent META Group survey of data warehouse projects found that 19% of respondents are beyond the 50 gigabyte level, while 59% expect to be there by second quarter of 1996.1 In some industries, such as retail, these numbers can be much larger. The accompanying need for improved computational engines can now be met in a cost-effective manner with parallel multiprocessor computer technology. Data mining algorithms embody techniques that have existed for at least 10 years, but have only recently been implemented as mature, reliable, understandable tools that consistently outperform older statistical methods.

In the evolution from business data to business information, each new step has built upon the previous one. For example, dynamic data access is critical for drill-through in data navigation applications, and the ability to store large databases is critical to data mining.

The core components of data mining technology have been under development for decades, in research areas such as statistics, artificial intelligence, and machine learning. Today, the maturity of these techniques, coupled with high-performance relational database engines and broad data integration efforts, make these technologies practical for current data warehouse environments.

Data mining can be used to help Managers make Strategic Decisions about Business Operations

The complexity of decision-making in manufacturing, business and medical applications is rapidly increasing, as the world is becoming data-driven. To cope with this increasing complexity in a changing environment, new modelling and computing paradigms are needed. The salient features of the data mining for decision-making are:

(1) Adaptability of decision-making models to the evolving decision environment,

(2) Ability to handle changing qualitative and quantitative data,

(3) Short decision response time,

(4) Large and overlapping problem domains,

(5) Interpretability of the results,

(6) Process rather than problem orientation.

Data mining methods can be used to support the development of decision models by inducing models to be used for classification, prediction and data segmentation.

Decision Support for Prediction, Classification and Data Segmentation

Data mining is the core of the knowledge discovery process which aims at the extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) information from data stored in large databases. Discovered patterns and models can frequently be used to support decisions. More specifically, numerous data mining algorithms have been designed with the goal to support specific decision-making situations. For example, predictive data mining algorithms result in models that can be used for prediction and classification, whereas descriptive data mining algorithm result in patterns and clusters, which can be used for data segmentation.

In contrast with model discovery, pattern discovery is better suited for decision support in exploratory data analysis. Instead of model construction, their goal is the discovery of individual subgroups or clusters uncovering regularities in a segment of the entire dataset. In prediction/classification tasks, the quality, for decision

making is measured in terms of the prediction/classification accuracy of the induced model, whereas in exploratory data analysis the quality of decision-making depends on the specific task addressed.

Basically data mining is concerned with the analysis of data and the use of software techniques for finding patterns and regularities in sets of data. It is the computer, which is responsible for finding the patterns by identifying the underlying rules and features in the data. The idea is that it is possible to strike gold in unexpected places as the data mining software extracts patterns not previously discernable or so obvious that no one has noticed them before.

Data Mining and Decision-Making: Model-oriented support assumes that models exist prior to decision-making activities. Developed by researchers and analysts, they are embedded in a support system. In contrast, in data-oriented support no model as given, rather it is constructed from the analysis of available data. Data mining and knowledge discovery techniques are used to extract knowledge and formulate models. This approach depends on the availability of large datasets, often stored in a data warehouse. The sequence of key activities in this type of support is given in figure above.

The realization of a decision problem or opportunity may initiate construction. This type of support, however, may be initiated by the user but it may also originate with the system itself. A routine analysis conducted, e.g., by online analytic processing may indicate the existence of a problem and invoke an extensive data analysis with statistical and other data mining techniques leading to model construction and problem formulation. Only at this stage will the decision-maker may enter the process of finding a solution to the problem.