It all sounds great and doors for companies will be opening in the future for all to take advantage of big data. But it's not that straightforward.
Big data is not something you should begin to think about soon, big data is here now. This is the conclusion not just of vendors trying to sell you the technology to deal with it, but of analysts looking at what businesses need to do to address this immediate plethora of data.
"Businesses are now working in an incredibly different environment compared with 10-15 years ago," said Carl Streatfield, presales manager for business intelligence and In-Memory at SAP.
"In 2005, mankind created 150 exabytes of information. In 2011, 1,200 exabytes will be created. This is the new world of big data.'"
That is a lot of data. They are numbers which many could not have imagined when they started working in IT, but companies must put the shock and awe to oneside and work out the best way of dealing with this data in a competitive way.
"Opportunities to improve the bottom line exist in a flood of information; however, gaining insight from data becomes challenging as it grows extremely large," said Brian Hopkins, senior analyst at Forrester, in his Big opportunities in big data' report.
"Emerging technology applies the power of distributed, virtual computing to the problem of large data, providing new tools and techniques that shift the way businesses use information to compete."
The cloud is one method a lot of vendors are choosing to put their weight behind to deal with the data surge hitting their customers.
EMC is one key example, building its annual conference around the "journey to the private cloud" and "big data" for the past three years, along with its annual sponsored survey conducted by IDC to come up with new and more terrifying measures for data size.
The thinking behind using cloud would be to remove these huge volumes from your own network, allowing for an optimisation of your own data centre without clogging it with reams of unstructured data.
However, as we have already pointed out, this data could be turned into amazing IP for companies, discovering what your customers really want and using the information to provide it. This is where firms get very nervous about moving to the cloud for security reasons and ensuring they have control over their property.
That being said, Hopkins does believe it is one of the most viable options.
"Forrester finds the elastic, scalable nature of the cloud to be a primary driver of big data implementation," he said. "Virtual versions of commodity infrastructure such as servers and storage blocks are now commonplace."
"Further, virtual networking and intelligent workload management tools enable elastic scaling of this infrastructure on demand, creating an ideal environment for cheaply sorting and processing large data sets across distributed systems."
Don't wait till it's too late
There are many options for companies both big and small to address the problem and opportunity that this data explosion brings. But they key message we have discovered is don't wait; big data needs to be tackled now before it floods your data centre and used to its full advantage.
"Enterprises of all sizes can take advantage of the information they gather, and use this increased knowledge into their networks, services, and customer behaviours to reduce costs and enhance customer satisfaction," concluded Mahony.
"In an era where customers have all the power, the ability to define new services based on actual usage patterns and behaviours is a powerful competitive advantage."
