Agile development is fading in popularity at large enterprises - and developer burnout is a key factor

Agile development methodology is facing significant obstacles as the tech industry goes through a wave of changes including developer burnout, shifting working environments, and the rise of AI.

Agile is a software development methodology first formulated back in 2001 which aims to deliver software faster by breaking monolith projects up into smaller elements. These can then be completed in ‘sprints’ of weeks or months which are monitored at daily stand-up meetings to check on progress.

The idea is that developers and business execs should work together daily on projects and that changes should be welcomed even at a late stage in a project.

Rapid iterations – and, if necessary, failures – are the best way to get projects finished, according to this approach.

“Working software is the primary measure of progress,” is one of the 12 principles of the Manifesto for Agile Software development. All of this stands in stark contrast to the ‘waterfall’ approach which was dominant previously.

This values lots of up-front planning and projects delivered in a systematic fashion - which critics argue leaves less room for adapting to change - and a greater chance of total failure.

But now a ‘State of Agile’ survey from software development company Digital.ai shows that while agile has many fans, particularly in smaller firms, it is struggling to make an impact in larger organizations.

“From AI to developer burnout, hybrid work environments and unrelenting demand, change is happening in every organization in every industry. At this moment in time, it feels like agile is having difficulty adapting,” the report said.

The study found that while smaller organizations continue to consider agile as a powerful productivity and organizational framework that exhibits “obvious benefits”, medium-sized and larger companies are less satisfied with what agile can do for them, and are more likely to pick a software development strategy that uses a number of different frameworks.

Use of agile development is still widespread

According to the survey of 788 tech workers, 69% of respondents said their IT, software development, and delivery teams use agile, as do nearly half of engineering, product, and R&D teams (48%), along with 28% of business operations and 20% of marketing teams.

Roughly 44% of respondents said enterprise agile works very or somewhat well for their organizations. Smaller companies were more likely to say they were happy (52%) compared to 43% of larger businesses.

Over two thirds (71%) of respondents said they were using agile in their software development lifecycle, while 42% said their organizations use a hybrid model that includes agile, DevOps, and others.

“The larger the organization, the more likely it is to use a hybrid model,” the report said, noting that bigger teams are also more likely to still use waterfall.

The Digital.ai report said the most successful agile implementations are in small companies. Three quarters of small companies (compared to 62% of large companies) said at least 50% of their applications were delivered on time and ‘with quality’.

But while in last year’s survey 72% of respondents said they were satisfied with agile practices in their company, this year’s study found that has dropped to 59%.

The biggest reason for this is that nearly half of survey respondents (47%) blamed a generalized resistance to organizational change or culture clash – up seven points from 2022.

They also blamed a lack of leadership participation and a failure of management support, and while a third said that business teams did not understand what agile did or could do.

Respondents who were unhappy with agile adoption rates blamed “too many mixed systems” in their companies, while second on the list for blame was siloed teams causing a culture clash and inconsistent use of agile.

Agile has been gradually spreading beyond software development, particularly as a result of demand for broader digital transformation projects which are viewed by CIOs as a way of injecting a little of the excitement and energy needed to crack open legacy systems and find new value.

But for organizations wanting to move agile from team level to company-wide, there are still a number of problems ahead.

One-in-three blamed leadership figures “not understanding and putting roadblocks up, either knowingly or unknowingly”, closely followed by “teams not empowered to be self-organized and/or self-sufficient”.

People in medium and large companies are more likely than those in small ones to say their challenge is that everything is siloed.

“What’s clear from the data is that, when agile works, it works – there are concrete benefits for the organizations who have gotten it right,” said Derek Holt, Digital.ai’s CEO.

“Agile still provides us with the best opportunity to manage these transitions and drive software delivery toward maximum business value.”