I would imagine you are thinking that headline is a pretty bold statement. And when I tell you that BLU Acceleration is an exciting capability being introduced in the new DB2 this quarter, you may think it bolder still.
If you have not read any of my past blogs, you may be asking “what does database software have to do with Big Data?“ The most important thing to remember is that meeting today’s “big data” challenges requires different types of systems that use different technologies for managing and analyzing different data in different ways. This is why the world now has a diverse set of NoSQL systems that have been added to the traditional SQL database systems. And this why IBM has added new systems (e.g., for Stream and Hadoop processing) as well as new NoSQL capabilities added to SQL systems (e.g., XML and RDF Graph database adds to DB2, and TimeSeries and Spatial database capabilities in Informix.)
In a recent discussion with an industry analyst, I was surprised to learn that he considers in-memory, columnar management of a SQL relational database to also be NoSQL. He revised my definition of NoSQL to be – Not Only traditional row-based relational data management via SQL. And so with the introduction of BLU Acceleration in the new DB2, it becomes a NoSQL data system for another reason. BLU Acceleration is dramatically easier and faster for analytics on terabytes of data. For many organizations, this enables cost effective analytics of more data and for more users.
In his blog, consultant and IBM Champion Dave Buelke called BLU Acceleration – Best yet for Big Data! He asserts that there are cases where Hadoop systems are being used or considered for analyzing data, where using BLU Acceleration will be a more simple and lower cost solution. (Note: neither he nor I am asserting this is true for all Hadoop uses cases. The point is – no one technology, including Hadoop, is the best answer for all needs.)
Speaking of User Groups, my thanks to the International Informix User Group team that hosted their conference this past week in San Diego. It was great meeting with members of this community and seeing both new and familiar faces among the attendees. A lot of positive feedback about the enhanced capabilities in the new Informix 12. This includes extending the use of Dynamic In-memory (technology shared with BLU Acceleration) for TimeSeries data – simplifying and accelerating operation analysis and reporting of growing smart meter and sensor data.
For more Big Data stories and to add your thoughts, I encourage you to join the conversation at the Big Data Hub.
Big data is all about scaling the use of data beyond the norms of the current era of information technology.
You could reasonably argue that the first big data era began more than a half-century ago. On May 25, 1961, President John F. Kennedy gave a speech to the U.S. Congress in which he declared the goal of landing a man on the moon, and returning him safely to Earth. The amount of data generated and managed throughout the program quickly outgrew data systems of the time. A brand new “Information Management System” (IMS) was created by IBM and other members of the Apollo team to tackle this new big data challenge.
Now, fast forward more than 50 years and we have ushered in a new era of big data, ignited by the global “Internet of things,” mobile, social and cloud computing, and instrumented systems of all kinds. Now every transaction, tweet or meter reading has potential value to enhance or destroy a customer relationship; to drive a new business opportunity; or to catch a bad guy. New types of data systems are needed to handle more data and more types of data, faster and more cost effectively than systems that were state of the art just a few years ago.
The key to making big data work for business is using systems that are designed for workload optimized performance and simplicity. In some cases that means completely new systems to handle challenges like analyzing data in motion, or spreading complex work among a large number of distributed systems. In other cases, new capabilities are added to proven systems such as IBM DB2 and Informix, to provide a new mix of production grade capabilities – e.g., for both SQL and NoSQL databases.
Solving today’s big data challenges often requires combining the structured, optimized approach of traditional database systems with the less structured, exploratory approach of new systems. In fact, modern versions of technology created decades ago may be the best choice for new enterprise challenges; ones that also benefit from their time-proven stability, maturity, and manageability.
So what’s the role of a relational data system in this big data era?
Some IT professionals may take relational and pre-relational database technologies for granted, but they remain the trusty workhorse in most data centers. These proven platforms continue to handle the growing volume of data and faster transactions from applications that conduct business every second of every day. They also enable deep analysis of that data to help organizations make better decisions with the speed needed to affect business operations as they execute.
Organizations leading the pack in big data ingenuity are the ones using the best combination of systems – traditional or new – for each need. For many organizations building complex systems, running global banking networks, or delivering millions of packages around the world everyday, that includes using the modern descendent of the data system that played a small role in a giant leap for mankind.
Look for more thoughts about Big Data at the speed of business from me and other followers of database technology in the coming weeks.
And if you’re interested in IBM’s next Big Data event, go to this link for details. http://ibm.co/BigDataEvent
Information on Demand 2012 was another great week this year, with a record number of attendees – over 12,000 IBM clients, partners, analysts, reporters and IBMers from around the world.
For those of you who did not join us last month, here is a summary of announcements made at the event. Also, the folks at Wikibon have assembled a nice set of videos and articles you should check out. Actually, those of you that were there would also find these summaries valuable.
A few to interviews to highlight given the subject of this blog:
- Tim Vincent: Rolling Your Own Database Distracts from Delivering Big Data Business Value
- Nancy Kopp-Hensley: PureData Helps Customers Transition form Planning to Executing on Big Data
- Nancy Pearson: We’re Changing the Economics of IT
- Jason Gartner: PureSystems is Innovative, Not Just Repackaging
- Pete McCaffrey: PureSystem Removes Admin Burdens for Customers
- and shameless plug for my interview: Big Data Requires Mix of Technologies
For me it was a particularly exciting year as it also marked the end of “launch month” for our new PureData System. But the real excitement of this event is the in person interaction with clients, partners, IBM Information Champions, and analysts. In a job dominated by conference calls and video chats, having the opportunity to participate in less formal conversations is a welcome change. It is particularly interesting to listen to exchanges among different clients about the challenges they face, and how they are using IBM technologies to meet them.
Speaking of clients and IBM technology.. the InfoSphere, Data Management, and System z product demo rooms and hands-on lab sessions were packed all week. This conference continues to be a nice mix of technical details and strategic discussions about the application of technology to improve business results. I spoke to several clients who each had a large group at the conference made up of business and IT leaders as well architects, developers and data professionals.
On a final note, Barenaked Ladies and One Republic both put on great shows. A great mid-week break from the very full days of business and technical talk.
I hope we see all of you next year… November 3-7, 2013
For those who may have noticed, I should explain my long absence from this blog. For the better part of this year my team and I have been “heads down” on preparing for and executing the introduction of the new IBM PureData System. Not having much time to spare was only a part of my excuse. The real reason was lack of energy and inspiration to write even one more piece beyond what was needed for the launch and for the IOD 2012 Conference last week..
Now that both events are behind us, it is time for me to get back on track….
PureData System is the newest member of the IBM PureSystems family of expert integrated systems I wrote about in April. It is offered in 3 models that deliver optimized performance for transactional, analytic and reporting, and operational analytic workloads. As an expert integrated system, each PureData System model is integrated software, hardware and built-in expertise that simplify the entire system life cycle – from procurement through retirement.
PureData System provides an efficient, high-performance and high-scale data platform – delivering data services needed for different types of transactional and analytic application workloads. Providing these values for data services needed for different types of applications requires software and hardware that are designed, integrated and tuned specifically for each type. Typically, organizations spend their valuable time and resources to design systems of general purpose components and then procure, integrate, configure, tune, manage and maintain each system for its specific use. PureData System dramatically reduces time, cost and risk when deploying and maintaining these systems.
- PureData for Transactions: integrates DB2 pureScale to deliver high-available, high-throughput transaction database clusters that easily scale without the need to tune the application or database. This PureData System is available in 3 size configurations and can be used to consolidate more than 100 database servers.
- PureData for Analytics: is powered by Netezza technology and is the newly enhanced replacement to the Netezza 1000 (formerly known as TwinFin). It is optimized for simplicity and performance for analytics and reporting data warehouses. This new model delivers 20x concurrency and throughput for tactical queries compared to the previous version Netezza technology, and offers the industry’s richest library of in-database analytics functions.
- PureData for Operational Analytics: integrates InfoSphere Warehouse software for operational data warehousing that can support continuous data ingest and more than 1000 concurrent operational queries, while balancing resources for predictable analytics performance. It also delivers DB2′s adaptive compression which has been used by clients to achieve up to 10x storage space savings. This PureData System model is a new generation that replaces the Smart Analytics System 7700.
And if that were not enough, we have also integrated the power and simplicity of Netezza technology with the reliability and security of System z to deliver cost efficient, high-performance analytics and operational analytics on data manages by DB2 for z/OS. System z clients now have the opportunity to greatly simplify and reduce cost of analyzing their most critical business data.
- DB2 Analytics Accelerator: The same Netezza technology that powers the PureData System for Analytics, also powers the newly enhanced DB2 Analytics Accelerator which integrates with DB2 for z/OS for high performance analytics – without modifying applications or the database. The new High-performance Storage Saver capability reduces demand on System z storage space without sacrificing performance.
- zEnterprise Analytics System: combines the new zEnterprize EC12 and DB2 Analytics Accelerator for a hybrid system that merges capabilities optimized for different workloads in a single, highly reliable, and secure system. The zEnterprise Analytics System 9700 and 9710 models have now replaced the Smart Analytics System 9700 and 9710.
That’s a good (re-)start… I will save my IOD 2012 recap for next week to make sure I get back on my weekly pace.
PS. My thoughts and prayers are with all those still suffering the effects of Sandy.
Here is another question where conventional wisdom about “the right answer” has been proven wrong: can IBM System z be the best solution for data warehousing and analytics? For many of my early days in the database software and systems business the debate raged about performance and price performance implications of using System z for analytics workloads. Recent client stories I’ve heard tell me that the advances delivered in DB2 10 for z/OS, and the Netezza powered DB2 Analytics Accelerator, have firmly answered the question.
For those that have not heard of DB2 Analytics Accelerator, it is a Netezza data warehouse appliance that integrates directly with DB2 for z/OS such that deep analytics queries are routed to it without any need to alter the application. Transactional and operational queries are handled by DB2 as usual, and all data remains under the industry’s highest level of security and availability.
Also, you should know the Smart Analytics System models 9700 and 9710 are integrated offerings that include Cognos BI, InfoSphere Warehouse and DB2 for z/OS software on a zEnterprise z/196 or z/114, respectively.
If you are finding it hard to believe this is a real change in the game, consider the following client examples from our Banking Industry team:
European Bank Group adds IBM DB2 Analytics Accelerator to System z over Exadata
This banking consortium has IT teams that are Oracle technology friendly, and had invested in an Exadata system last year. They were considering moving BI workload to the Exadata system but the IBM team demonstrated the benefits of a BI infrastructure based on IBM System z with the DB2 Analytics Accelerator. The client chose the IBM solution.
Federal tax authority chooses IBM Smart Analytics System 9700 after DB2 10 for z/OS blows away Oracle in a performance benchmark
A benchmark between Oracle Database and DB2 for z/OS was the first step in this decision process: DB2 proved to have 10 times better performance in the benchmark. In addition to superior performance, other decision factors for choosing IBM Smart Analytics System over Oracle included:
- An end-to-end solution, including comprehensive data warehousing and business intelligence software
- Reliable hardware
- In-depth services that will support deployment and operation of the new platform
IBM System z selected over Teradata at one of the world’s oldest banks
This bank needed an integrated data warehousing solution for corporate, financial, and marketing information across the bank to reduce costs, improve revenue and drive better profitability. Factors in choosing IBM System z over Teradata included:
- Significant savings in hardware, software, operating and people costs
- Faster time to value with a reduction in the time required to deploy Business Intelligence solutions
- Industry leading scalability, reliability, availability and security
- Simplified and faster access to the transactional and operational data on System z
North American Bank moves off Teradata in Favor of IBM Smart Analytics System
Teradata was the warehousing standard at this bank and its team had a misconception that IBM System z was not leading-edge technology or the most cost effective solution. Fortunately, the team also had open minds and a desire to find the data warehousing and analytics solution that delivered the best value for their business. The result: a transition from Teradata to an IBM Smart Analytics System powered by System z.
Never say never
Now don’t get me wrong. I am not saying that System z is the best analytics system choice for all clients in all situations. I am saying that you should not assume it isn’t the best choice for you and your situation. Make business decisions based on the reality of today’s facts, not based on outdated misconceptions.
I am starting this post at 31,000 ft on way home from Atlanta where I spoke at another CIO Forum and Executive IT Summit. I also spoke at one in Seattle earlier this year. These are well run events that are specifically for CIOs and senior IT executives. I really enjoy hearing the exchanges among these peers who are facing many similar challenges, regardless of what their companies do. Exchanges are often very productive with cards exchanged for follow-on actions or continuation of the discussion.
Full disclosure: IBM sponsors this event series which will be held at 16 cities across the US and Canada in 2012. The keynote discussion is about Big Data challenges and the new technologies for tackling them. We also do a follow-on session on either Evolving Data Warehouse Architectures or Information Integration and Governance. Today I was talking about the evolving architectures.. many of the points I’ve shared in my earlier posts.
Feedback has generally been positive, but even more so today, I thought. I had discussions with IT leaders from higher education, marketing services, financial services, health care, and IT consulting services. They all were talking about the need to evolve their environments to gain more insights from more types of information, and deliver it to business users faster. It made me feel good about the focus we currently have with our clients.
But it also made me think that we are not too far from having to figure out the next chapter in the story. The only way to be among those leading the way forward, is to always be scouting ahead to see what is over the horizon. The great thing about the computing business – at least over my career so far – is that we are never done inventing the future.
Note: Another disclosure.. I am actually finishing and posting this a week later. The holiday weekend with family and friends was too good, the need to connect and post this flew right out of my mind.
I was at a family gathering this weekend where I had the opportunity to talk to a friend who is also now in a leadership role at a large technology company. Our discussion about the growing availability and use of information reenforced my belief that we are experiencing an exciting evolution in the business world that is having a profound global impact. But most of our discussion was not about technology, it was about global skills availability and growth. I had a very similar conversation the day before with one of my neighbors who is also in a technology leadership position.
Thinking about this topic on the drive in this morning, I recalled that am overdue in publicly welcoming new members of our IBM Champions program. IBM Champions are IT professionals and educators who make significant contributions to their communities: evangelizing IBM solutions, sharing technical knowledge and expertise, and growing and nurturing independent communities.
We recently expanded the set of Information Management Professionals in the IBM Champion program to 158 Champions across 28 countries. My thanks to all of these folks who contribute their time and talent to strengthen and expand their professional communities.
Given the global need for these skills.. and the need to develop individuals for well paying jobs, there is a tremendous opportunity for educators and experienced mentors to help grow a new generation of Information Management professionals and data scientists.
P.S. For anyone noticing the drop off in my blogging pace lately, we can blame spring fever and my need to spend time at family events… and on my yard and golf game. (I have lots of work to do on the latter to keep up with my son!)