GitHub: 30% of Copilot coding suggestions are accepted

Adding AI into the development lifecycle has resulted in a sea change in software development as the percentage of accepted code suggestions creeps upwards.

The research, published by GitHub comes a year after the company’s Copilot coding assistant became generally available. The AI pair programmer is intended to provide coding suggestions in real-time, speeding up the development process.

GitHub Copilot has been activated by more than one million developers and adopted by over 20,000 organizations. It has generated more than three billion accepted lines of the code.

By 2030, the research estimates that global GDP could be boosted by more than $1.5 trillion thanks to productivity improvements driven by generative AI coding assistants. These improvements would represent a productivity gain of 15 million “effective developers.

The report is based on analysis of the coding habits of 934.533 Copilot users with code acceptance rates picked up from telemetry.

How many coding suggestions does Copilot accept?

According to the report, an average of 30% of coding suggestions from Copilot are immediately accepted, although this increases with use. During the first 3 months of use, 28.9% of suggestions were accepted, rising to 32.1% during the next 3 months. The acceptance rate approached 34% in the 6th month.

While still meaning that approximately two thirds of suggestions are rejected, this represents an improvement over the 26% observed at launch, which followed a 12-month technical preview.

The trend is clear, and indicates developers becoming more comfortable with the suggestions as time goes by. Additionally, coders are better understanding how to use the tool in order to generate meaningful results.

Who is using Copilot?

The report measured the experience of developers by their number of repository actions on GitHub (prior to their use of Copilot.) Developers with less repository activity had a higher acceptance rate (averaging 31.9%) compared to the most experienced, which averaged 26.2%.

Less experienced developers are therefore deriving the most apparent productivity benefit from AI coding assistants.

GitHub anticipates that in the future, as tools evolve (for example, with the implementation of natural language models), approximately 80% of code will be written with AI. The research describes this as a democratization of software development as people with non-technical backgrounds are able to build applications.

During a discussion at a recent Kubecon, Chris Aniszczyk - CTO of the Cloud Native Computing Foundation - remarked that Copilot will currently “get me probably 80% of the way there” in terms of the basics, but added “it’s not perfect”. He also noted ongoing issues with code attribution.

What does Copilot mean for junior developers?

As the rise of AI pair programming tools continues, the skills of junior developers will need to change. The research compares the technology change to the shift from punch cards through compilers and high-level programming languages.

It also suggests that new business models could arise, making comparisons to the arrival of the steam engine and the Internet, and the higher-order work generated.

Demand for developers continues to be strong, despite waning from the boom seen in recent years and the report insists that this “demand for software and software developers” will only increase with the ongoing wave of digitization.

An alternative interpretation of the trends shown in the research is that productivity increases will come at the expense of junior developers, effectively choking off the pipeline of more senior staff.

GitHub disagrees, and states: “this collision of AI and the software developer will not lead to a decrease in developer jobs - it will lead to AI augmenting developer potential and accelerating human progress.”