Gemini Code Assist could be Google's secret weapon to challenge GitHub Copilot

The launch of Gemini Code Assist, Google's new pair programmer assistant, represents a bold move from the tech giant to unlock developer value and push ahead of competitors when it comes to AI productivity offerings. 

Powered by its new and powerful large language model (LLM) Gemini 1.5 Pro, Gemini Code Assist was one of the standout announcements at Google Cloud Next 2024. 

There’s a real sense Google Cloud has found a killer application here, with the model’s one million token context window allowing it to generate outputs based on a business’ entire codebase for context.

Google Cloud has used the event to announce several services that compete with Microsoft’s Copilot approach to AI assistants. Notable among these are its Vertex AI Agents, natural language assistants that businesses can ground and train on their own data to deploy for specific tasks. 

But Gemini Code Assist stands out as a truly unique offering in the market.

While the accuracy of Gemini 1.5 Pro will come down to benchmarking, its mammoth context window far exceeds competitors such as GitHub Copilot. OpenAI’s GPT-4 Turbo, which underpins Microsoft’s Copilot Pro offering, has a context window that maxes out at a comparatively small 128,000 tokens. 

This gives Google Cloud a competitive edge, with the coding assistant capable of generating and translating code into a simple one-shot query-output.

Gemini Cloud Assist supports over 20 programming languages, including Java, C++, SQL, Python, and PHP. In addition to code generation, it's capable of large-scale code translation, with the context window also allowing businesses to convert swathes of source code or even entire codebases into another programming language.

In conversation with Darren Mowry, MD of AI Startups at Google Cloud, ITPro discussed the inherent business value of cutting down coding time and streamlining the process of translating code within AI pair programmers.

“Most founders are technologists,” Mowry began, pointing to the clear market Google Cloud has among startups – some 60% of generative AI startups being Google Cloud customers compared to 50% last year.

“I remember having a great discussion with a founder years ago, who made a comment to me of ‘I can always raise more money. I can always hire more people. I can’t invent more time’.

“So anything that you can do to collapse time, to allow me to develop a product faster to differentiate more quickly, and to expand my product set… If you can help me do those things I'll spend time with you. If you're not going to help me do those things. I don't have any time.”

Gemini Code Assist has already delivered tangible benefits

Google has already used Gemini Code Assist internally to boost the productivity of its own developers and reduce toil, while customers who were given early access to the tool have also reported significant gains.

For example, the AI-powered tech services firm Turing is already using Code Assist for internal development, giving its workers coding suggestions based on its own code. and has measured a 30% productivity improvement as a result.

“Developers were able to set their environments 55% faster than before, there was over 48% increase in unit test coverage for the code, and 60% of developers reported that they were now able to focus on more satisfying work.

A keynote demonstration saw Paige Bailey, product manager for generative AI at Google Cloud, migrate a customer-facing web feature based on a brief by the design team. 

Bailey stated that it would normally take days for a developer to begin to familiarize themselves with their company’s codebase – which may consist of more than 100,000 lines of code – and even more time to complete the task of migrating the web feature and check the configurations of its associated microservices.

“This is because Gemini's code transformations with full code base awareness allow us to easily reason through our entire codebase,” Paige told the audience. 

“In comparison, other models out there can't handle anything beyond 12-15,000 lines of code.”

“We’re also finding that for a lot of developers, it’s really difficult if you're trying to afford pair programming which is a best practice,” said Caroline Yap, MD, Global AI Business at Google Cloud.

“But now, when you can have the AI also be the pair programmer with you and can also suggest, ‘oh, this is how you would write a unit test’ or it takes away the toil of being a developer so you can focus on building the experiences  you're trying to build as a developer versus all the other stuff.”

Brad Calder, VP and GM at Google Cloud stressed that Gemini Code Assist can work with codebases hosted anywhere.

“On premises, GitLab, GitHub, GitBucket, and even across multiple repositories. We do not require your code to be only in GitHub.”

Calder also noted that “the other leading provider only supports repositories in one country, which may be inadequate for companies who have strict data residency requirements”. 

This is another way in which Gemini Code Assist acts as a broad competitor to other pair programmers on the market. 

Potential in the legacy codebase space

With the ability to process and even translate thousands of lines of code in one go, Gemini Code Assist carries great potential for firms looking to migrate and update outdated, legacy codebases. 

When ITPro raised this topic at a Google Cloud Next roundtable, Yap and Nenshad Bardoliwalla, director of product management for Vertex AI at Google Cloud, agreed that legacy code translation is a topic of great interest throughout the sector.

“We have a very interesting situation happening in the industry,” Bardoliwalla said.

“The people who know how to run these systems are passing away. They're literally not going to be here anymore and there's nobody who can help you understand the business rules, the semantics of the data, etc.

“Try finding a COBOL engineer. It’s next to impossible.”

But while both acknowledged the potential for tools like Gemini Code Assist for translating codebases, Bardoliwalla cautioned against the idea that translation of legacy code can be achieved through LLMs alone.

“Language models help in a number of the use cases… some of the folks who work on our mainframe monetization programs in Google Cloud have found LLMs useful, but there are a number of other technologies surrounding that that allow us to do migrations in a very elegant way.”

Bardoliwalla emphasized the need for data processing routines and semantic analysis of code alongside raw translation, while confirming that customers are very engaged in the idea of mass code translation.

It’s clear that Gemini Code Assist has an important role to play in addressing the pressing legacy issue, alongside the boosts to developer productivity it can enable.