The convergence of Big Data, NoSql and Smart Consolidation

Thanks to a number of recent blogs and discussions with colleagues and analysts, I have realized that a many of  the topics I am involved in are converging.   Big Data, NoSQL, workload optimized systems and smart consolidation/logical data warehouse  for an analytics ecosystem are typically considered separate topics and dealt with in isolation.  But as I have observed the similarrities in how they are discussed, I realize it makes a lot of sense to view them as different aspects of a common idea.  (I am not sure yet what a good lable might be so I am not offering one here.)

A recent example is Philip Howard’s blog from Bloor Research IBM and Big Data.   Philip writes about the different systems for addressing different “big data” challenges in a similar way that I did regarding different analytics systems.  And discussions about the value of new (and in some cases not so new) data management approaches under the label “NoSQL” are additional examples of matching the workload requirements to the proper system to optimize for top performance, efficiency and simplicity.  Watch this interview with IBM Fellow Curt Cotner on NoSQL and Hadoop at the recent IOD 2011 conference for more detail on what IBM is up to on this topic.

In 2005 when I joined the IBM Information management team, many of my friends questioned a move to a segment of the business that was considered “done”.   The answer to every data management challenge was Relational Database Management Systems and the major and minor RDBMS players were firmly established and the market had achieved a steady state. 

One of the reasons I love this business is that there is no such thing as a permanent steady state.  I believe that since 2005 the Information Management and Analytics advances have been among the most dynamic in the industry and impactful for our clients.  The current set of  hot topics indicates we are not nearly done with this current period of advancement.

How do you see these various threads melding together?  Do you see others that are as well?