Smart consolidation to a logical data warehouse

For the first of what I am hoping will become a weekly habit of sharing what is top of mind in my role at IBM, I am essentially adding to my previous post re: Workload Optimized Systems…

Single Data Warehouse -> Logical Data Warehouse

There is a growing recognition that the notion of a single “data warehouse” (one that incorporates all available data and supports all types of analytics workloads) is outdated given the explosive growth of information and types of analytics systems.   Of course I am not the first to make this observation.

I have been discussing this concept with my team at IBM and with analysts who spend a great deal of time with clients.  While I could not tell you who coined the term Logical Data Warehouse until I read Mark Beyer’s guest post on Merv Adrian’s blog, I can tell you the concept is becoming fairly well recognized.  As an example, it is a fundamental part of the game plan at IBM, given our growing portfolio of systems that are optimized for different types of analysis.  For more details, read The Logical Data Warehouse: Smart Consolidation for Smarter Warehousing by my colleague Phil Francisco.

Different systems for different analytics

To give you a brief idea why this is important, consider systems optimized for a variety of analytics:

1) Operational Analytics (IBM Smart Analytics Systems)

What is it? Balanced performance for complex analytic queries and a high volume of concurrent operational transactions

How it can be used: To support a call center with operational insights at time of contact

2) Deep Analytics (Netezza appliances)

What is it? Optimized performance and simplicity for analytics workloads that do not include operational transactions

How it can be used: Complex data mining and predictive analytics

3) Time Series Analytics (Informix TimeSeries)

What is it?  Using time series specific data structure instead of a typical relational one to dramatically reduce storage space required and speeds both data loads, analytics  and operation reports.

How it can be used:  For unlocking hidden insights among the growing volume of data from Smart Meters and sensors in all kinds of “smarter” system

12/14/11  add:  I just read this article that does a fine job explaining this value – and including the spatial data management optimization of Informix software, too.     Efficient Vehicle Tracking System Software Solution with Informix

4) Streams Analytics (InfoSphere Streams)

What is it?   Ultra low latency analysis of information flowing through a system before it is even stored – if ever

How it can be used:  Telemetry from medical devices in an Intensive Care Unit

5) “Map-Reduce” (Hadoop) Analytics (Infosphere BigInsights)

What is it?   Sometimes used as a synonym for Big Data, these systems enable analysis over a very broad and diverse set of information such as available to us via the Internet.

How it can be used:   To analyze petabytes of structured data including weather reports, tidal phases, geospatial and sensor data, satellite images, deforestation maps and weather modelling research, in an effort to pinpoint the right installation location for new Wind Turbines… all within one hour!  (Read more about how Vestas is using BigInsights)

Do you know of other kinds of  data analytics systems, or how organizations are using various types in concert to gain new insights and greater competitive edge?