More needs to be done to address the data science skills gap on an education and training front, according to Teradata CTO Stephen Brobst.
Speaking to IT Pro at the big data analytics firm's Partners Conference and Expo taking place in Nashville, Tennessee this week, he said the widely reported big data skills gap is at risk of widening without urgent action.
"The demand for the skill sets far outstrips the supply," he told IT Pro. "There's no question.
"We don't do enough from an education point of view to get those skill sets into our new grads and so on, because we teach them math but we don't teach them statistics."
A recent survey of 300 business and technical decision makers in the UK, carried out by Opinium Research on behalf of EMC, revealed that almost two-thirds of respondents lack the skills needed to understand how to use customer data ethically and responsibly.
Addressing this issue, he added: "The people that are already out there... probably don't have [focus on the ethics related to data], and so it's the responsibility of companies to deliver opportunities for their people to attend those seminars.
"If you're working with data, you probably should be required to get some level of education around ethical use and treatment of the data particularly in industries like healthcare or financial services," Brobst continued.
Another concern in the industry highlighted in the same survey was how a stagnated or outdated infrastructure can prevent organisations from taking full advantage of the benefits big data would offer them.
"We talk about these data lakes, and sometimes these data lakes become data dumping grounds and nobody knows what to do with it," Brobst said. "They're very good at getting the data in, [but] very few people are actually getting answers or information out. You get no credit, in my book, for storing lots of data.
"Which goes back to data skill sets do you have the data scientists who can exploit the data, convert the data into the information, the action and so on."
According to Brobst, the people currently heading up companies are those who need to be convinced of the value in data-driven business more than most, having climbed the ranks in a culture that existed before it's time.
The next generation will be automatically data-focused, but the skill sets required will be hugely altered.
Rather than relying primarily on gut instinct or experience when making decisions, those decision makers will have to ask the right questions of the data and subsequent analytics presented to them. And the problem may lie in the tools available to them as much as it does the skills shortage.
"Very often data scientists have to be computer scientists. A data scientist shouldn't have to have a PHD from Stanford University to get value out of data, they should be focused on the data and creating value, not programming as a computer scientist.
"So we have to have better tools to allow data scientists to get value from data without being a computer scientist," he added.
