Increasingly complex data storage requirements mean that businesses have to learn lessons from previous examples of “monolithic” data transformation tactics, according to NetApp CEO George Kurian.
Speaking to ITPro, the chief executive said previous issues with Hadoop, a distributed big data storage framework, show that organizations risk data-fueled bottlenecks which harm innovation if storage considerations aren’t flexible enough to match evolving needs.
Hadoop was once hailed as a key tool for organizations processing large datasets ranging from gigabyte size to petabytes. Thousands of organizations used - and continue to use - the framework, including Meta.
However, Hadoop’s limitations have often been widely bemoaned by users, many of whom went “all in” with the solution, Kurian said during his opening keynote at NetApp Insight. These limitations included the lack of real-time data processing and the ability to efficiently handle new data types, including AI and ML models.
Kurian said that, with the emergence of generative AI, the need for efficient and high-performing data processing capabilities should be a critical priority for firms hoping to capitalize on their data and maximize value.
"Many organizations said ‘let’s take a top down, monolithic approach to solve my data problems and put all of my data into this one solution called Hadoop,” he told ITPro at a press briefing.
“Hadoop, meanwhile, had its own proprietary analytics framework, its own data schema, its own proprietary computing models and storage,” Kurian added.
“And suddenly the world changed and Hadoop was no longer capable of dealing with new data types. It couldn’t deal with AI or ML models and so on.”
Kurian noted that instances of “top down”, sprawling data transformation projects more often than not resulted in longer-term headaches for organizations, many of whom were forced to fundamentally alter their entire data architecture.
“So now you’re trying to replatform,” he said. “And trying to replatform a Hadoop landscape is brutal. You have to replatform the analytics framework, your data schema, your computing architecture, your schedulers and your storage.”
Kurian’s comments to ITPro formed part of a broader message on data transformation and leadership throughout the duration of the NetApp Insight conference.
In his opening keynote, Kurian emphasized that data leaders have to evolve to accommodate for rapidly changing business conditions and the emergence of technologies such as generative AI.
Manage your data in a feature-rich unified environment
DOWNLOAD NOW
Kurian pointed to research from Boston Consulting Group that found “data laggards” - leaders who are slow to modernize data architectures for changing requirements - could risk falling behind their more agile, forward thinking competitors.
Far from calling for a widespread rethinking of data architectures, both Kurian and NetApp have been keen to emphasize that businesses adapting to modern data management requirements should act “flexibly” in their transformation practices.
“What we are saying is you want to be able to preserve flexibility in certain parts of your architecture,” he explained. “In your analytics framework, for example, you want to be able to preserve flexibility because the technology change is so fast right now.”
“You don’t want to have all of your layers of your architecture to be slammed together in one monolith. You want to have a layered model where some are stable and some are flexible.”
