GPU memory vulnerability could allow hackers to access LLM responses - and Apple, Qualcomm, and AMD products were all at risk

A GPU memory vulnerability dubbed ‘LeftoverLocals’ could expose LLM responses to hackers through leftover local memory, researchers have warned. 

Apple, Qualcomm, AMD, and Imagination are among the big name GPU vendors named as vulnerable according to research posted on the Trail of Bits blog. The GPUs have all been vulnerable to varying extents from as far back as September 2023, when researchers first began their investigation.

Researchers were able to build a proof of concept (PoC) of the potential attack, citing an attached video in which they listen in to another user’s interactive LLM session by recovering a GPUs local memory.

From their tests, the researchers concluded that ‘LeftoverLocals’ could leak around 5.5mb per GPU invocation on an AMD Radeon RX 7900 XT, enough data to reconstruct an LLM with worryingly high precision.

With GPUs used extensively to support the high performance requirements of AI inferencing, this news is likely to raise serious concerns among enterprise AI users.

Eleanor Watson, IEEE member and AI ethics engineer at Singularity university, told ITPro that although this particular vulnerability would require physical access to a GPU, the research highlights serious data protection risks.

“Whilst this particular type of exploit requires direct access to the GPU and its memory, it’s indicative of the challenges of keeping our interactions with AI systems private,” she said.

“I expect that further vulnerabilities will be uncovered which are broadly applicable to a wide range of LLM systems, leading to a widespread doxxing of interactions and generations, and associated embarrassment,” she added.

Vendors have acted swiftly on GPU memory vulnerability  

To defend against the vulnerability, GPUs need a built in system to clear local memory between kernel calls. Some GPU vendors, including Nvidia and Intel, are already doing this, whereas others need to keep pace. 

In response to the news of ‘LeftoverLocals’, AMD stated plans to create a new mode that “prevents processes from running in parallel on the GPU and clears local memory between processes on supported products”, with an expected rollout of said mode by March 2024.

“This mode would be designed to be set by an administrator and not enabled by default,” the firm said in an advisory. “Supporting documentation for the new mode, along with details of how to update AMD products, will be provided in a future update to this security notice.”

Apple responded to Trail of Bits but did not issue specific details of their patch, while Qualcomm issued a partial fix and Imagination a full patch in December.

A failure to patch this vulnerability could see attackers targeting a variety of GPU applications and LLM sessions, including those within privacy-sensitive domains.

Open source LLMs, however, are still the main target. Despite their ability to be rigorously audited, their reliance on closed-source GPUs means they are particularly susceptible to this attack.

“A lot of security hardening will need to be done on AI systems to make them more resistant to these kinds of vulnerabilities”, Watson said.

“This is a necessary component of the ongoing professionalization of AI, along with mastering challenges such as confabulation/hallucination in models”, she added.