Return on investment (ROI) from AI has plummeted over the last three years, according to research from Appen, and the issue rests on how enterprises are maximizing the use of their data.
Where 56.7% of deployed AI projects showed meaningful ROI in 2021, that figure slumped to 53.5% the following year and 51.9% the year after, before dropping another few percentage points to 47.3% in 2024.
Over the same period, the mean percentage of projects making it to deployment has also dropped. Where 55.5% of AI projects were deployed in 2021, just 47.4% were deployed in 2024.
The report, dubbed ‘The State of AI 2024,’ fielded responses from US-based IT decision makers at firms with over 100 members of staff.
The surveyed IT leaders put data management as the leading obstacle for AI projects, with 48% citing bottlenecks related to sourcing, cleaning, and labeling data. This figure is up 10 percentage points from 2023.
While a less prominent cause of bottlenecks, lack of data was cited as an obstacle by 29%, up from 22% in 2023. The average accuracy of data has also been falling, the study warnedWhere accuracy was 63.5% in 2021, that figure had fallen just under 10 percentage points to 54.6% by 2024.
“This may be a result of the increasing complexity of AI systems and the corresponding data annotation requirements, which increasingly require specialized domain-specific knowledge and quality control across large datasets,” the report said.
What can IT leaders do?
The report described data quality as essential to the success of AI projects, with many of those surveyed acknowledging the importance of driving data quality through active human involvement.
80% of respondents acknowledged the importance of ‘human-in-the-loop’ machine learning (ML) and accepted that human insight is key to the refinement of AI systems.
To overcome challenges in data accuracy, Appen said organizations must pay closer attention to long-term value creation and shift their focus away from short-term gain. This will allow them to build a more robust foundation on high-quality data over time.
“High-quality, bespoke human-generated data can significantly improve the performance and reliability of AI models. By incorporating expert-labeled training data and rigorous evaluation processes, enterprises can better align their models with real-world needs, enhancing accuracy and relevance,” the report said.
What other problems are hampering ROI?
When undertaking AI projects, businesses need to have an approach that takes into account the full AI lifecycle, Matt Armstrong-Barnes, chief technologist for AI at HPE, told ITPro.
“Considering AI – especially GenAI – is data, power, time and resource intensive to deploy and maintain, businesses need to take the necessary steps and lay the groundwork for their deployments so they don’t run before they can walk,” Armstrong-Barnes said.
Similarly, Bill Conner, CEO at Jitterbit, told ITPro that a ‘rip-and-replace’ approach to AI implementation is not advised as it could create longer term issues for internal adoption and create unnecessary risk.
“Infused AI capabilities, on the other hand, will deliver more choice and flexibility to match the technical and business objectives of their organization,” Conner told ITPro.
