Many people have heard about the concept of a data warehouse, but not many know what it actually is. A data warehouse is a database that aims to centralize data storage within a given organization. For instance, a company might have 5 different databases to hold various types of data. One day they decide to store all of that data in a single central database – this would then be called a data warehouse.
Why do we need data warehouses?
Warehousing enormous amounts of data was not possible until quite recently. Luckily, incredible advancements in computing technology now allow us to create massive central databases that can store and access information on the go. Data warehouses help to make data more accessible and complete. It is important to note that both central data storage and retrieval are equally important aspects of a data warehouse.
The process of making all data available in one place helps to maximize access and speed up data analysis. Various data mining techniques are then used to analyze this data and find patterns and relationships within (often) seemingly unrelated data clusters.
Another common use for data warehouses is to provide reports about trends (or simply change) in data. For this reason, most data warehouses keep track of changes to data.
Why do we need data mining?
Data mining is primarily used by large companies that collect a lot of consumer data. This data is then analyzed with a goal to help improve conversion rates, maximize profits, improve safety procedures or product safety and so on.
Without data mining, people would not be able to process large amounts of data accurately. A computer can spot patterns and associations within billions of entries in a single dataset, which is simply too much for any human to process.
There are many complex algorithms used to accomplish the task of mining information from data. For instance, some approaches are designed to analyze large amounts of data to find anomalies (things that just don’t fit in), which can then be investigated further. In the real world, this technique might be used by the IRS to analyze annual tax returns and then review those that are marked as “not fitting in”.
Another common example of data mining is an approach called association learning. This approach is used by large retailers to create personalized product recommendations based on products which you have previously purchased, used or simply looked at.