Can AI coding tools be trusted? Developers aren’t so sure - over a third are concerned about AI-generated code quality despite widespread adoption and productivity gains

Over one-third (39%) of software developers have little to no trust in AI-generated code despite most admitting to using AI coding tools daily, new research shows.

Google’s ‘State of DevOps’ report found over 75% of surveyed developers use AI for at least one daily professional responsibility. Chief among use cases is code generation, with 74.9% using the technology to write code.

While over a third of respondents reported the productivity benefits of AI, many lack confidence in the code produced.

This low level of trust expressed by respondents indicates a need to manage AI integration more thoughtfully, the report said, and teams must ensure they evaluate AI’s role in workflows carefully to mitigate downsides.

It’s worth noting that the majority of those surveyed (87.9%) reported that they had at least some level of trust in AI-generated code.

There are also issues in software delivery performance. The increase in AI adoption was mirrored by an estimated 1.5% decrease in software delivery throughput, the report explained, along with an estimated 7.2% reduction in software delivery stability.

“Our data suggest that improving the development process does not automatically improve software delivery — at least not without proper adherence to the basics of successful software delivery, like small batch sizes and robust testing mechanisms,” it said.

The report suggested three recommendations on the back of its findings, such as the establishment of clear guidelines for AI use, addressing procedural concerns, and the assurance of open communication around AI impact.

Google added that businesses should encourage the exploration of AI tools and create time dedicated to AI experimentation.

Do AI coding tools cause more harm than good? 

Google’s research marks the latest in a string of studies exploring the benefits and potential pitfalls of AI coding tools, with developers flagging repeated concerns over code security and consistency.

A recent study from Black Duck Software aligns closely with Google’s study, highlighting growing distrust among developers with regard to AI-generated code quality.

The study, published earlier this month, found that around 90% of developers are using the technology in their daily workflows. However, less than one-quarter of respondents described themselves as ‘very confident’ in their organization’s policies and processes for testing AI-generated code.

Notably, issues with code quality were found to be slowing down development lifecycles due to the fact security testing practices had to be expanded to accommodate for potential inconsistencies.

Similar research from Gitclear found that AI-generated code can be often of a lower quality and is more difficult to maintain, while news earlier this year revealed code generated by GitHub Copilot could be dangerously flawed.

On the other hand, research from GitHub found software development teams created more secure software and better quality code through the use of AI coding tools.

Despite lingering concerns, research has highlighted the benefits of AI coding tools for developers globally. A study from GitHub in August found the use of AI coding tools has significantly improved productivity and helped reduce strain on developers.

A key benefit highlighted in the study was the time savings afforded by the use of AI tools, according to GitHub. Nearly half (47%) of respondents stated they’ve been able to free up more time to ramp up collaboration with peers and focus on system design, for example.

Other developers reported that the time savings afforded by AI tools has allowed them to learn new coding languages.