AI developers should be pursuing stronger identity controls linked to automatic ‘kill switches’ as well as more open source innovation to ensure AI systems aren’t breached or used for malicious purposes, an expert has said.
With stronger identity controls linked to a kill switch, businesses could run authentication on each API call to an AI model and cut connections that are deemed illegitimate.
Identity controls could also be used to dictate what prompts can be passed to an AI model to prevent inputs that attempt to abuse a model’s vulnerabilities, and even to shut down a system altogether if unauthorized parties were to escalate their privileges over it.
Kevin Bocek, VP of ecosystem and community at Venafi, told ITPro that strong identity controls when it comes to AI systems could be used to give organizations peace of mind and keep data safe.
He outlined the fact that while AI can unlock enormous benefits for business is also comes with significant risks, before calling for systems that could temporarily disable an AI model’s network connections, or even the entire model, if it was beginning to act dangerously.
“And of course, how do we stop connections? We stop it based on identity,” said Bocek.
“We know code being run has an identity, we can control which code runs on everything from a desktop to a Kubernetes cluster based on its identity. We know that when systems are connecting from one API to another, what do they do? They authenticate. Great, we can authenticate models too.”
Many generative AI systems operate using pre-trained models stored on the cloud that return data via application programming interfaces (APIs), and a kill switch could be used to sever the connections between malicious users and the model’s server.
Observability for developers
Notice critical changes and discern the true causes of errors.
DOWNLOAD FOR FREE
Bocek called for greater authentication when it comes to running AI models, citing existing identity protections that allow IT managers to control which code runs within environments ranging from a desktop to a Kubernetes cluster.
He cited a number of identity authentication protocols that could be used as the basis for these AI security measures, including transport layer security (TLS), secure shell (SSH), and the Secure Production Identity Framework for Everyone (SPIFFE) which sets out standards for securely identifying open source software.
He also rejected fears over the risk posed by open source AI models, and urged for greater innovation in the space in order to keep up with cyber criminals.
At present, threat actors can abuse these connections, Bocek said, and set out three degrees of threat posed by hackers when it comes to AI: Theft, compromise, and escape.
Theft and compromise are two areas that a kill switch linked to API identity management could prevent, as they comprise attacks in which threat actors use techniques like prompt injection to reveal source code or model weights.
Escape refers to a situation in which a model begins to act in a consistently hostile way, either as the result of a hacker or due to an error, and begins to self-replicate via its available connections.
The benefits of an open source approach
Meta’s large language model (LLM) LLaMA was leaked online in March 2023, raising fears that threat actors could use the model without oversight to enhance malicious operations. Mandiant researchers have warned that AI will drive a new era of social engineering, for example.
But moves to limit the open source community’s access to AI models out of fear that threat actors could get their hands on sophisticated systems were misguided, Bocek said.
“Security through obscurity has always been proven to fail. It doesn't allow researchers to do their jobs, it doesn't allow us in the private sector as well through implementation to do our jobs and learn about risks and make improvements, it just doesn't.
“The same thing I believe holds true now as we head to the future, with machine learning and generative AI, large language models. Open source is where innovation is happening. We wouldn't have what we know today is like ChatGPT or LLaMA, if it wasn't because of the open source community.”
Pursuing a closed approach to AI, without allowing for open source innovation, could empower threat actors and leave legitimate developers behind.
“Put everything under wraps, and you only allow the adversary the opportunity to innovate and to also create malfeasance.”
The public sector could play a greater role in securing AI systems from a regulatory perspective and already weighs in on security policies such as data breach reports. For example, the SEC now requires public firms to disclose data breaches within four days.
The EU's Second Payment Services Directive (PSD2), which requires financial services to adopt additional layers of security such as multi-factor authentication (MFA) for online transactions, is an example of the kind of identity regulation that will be necessary.
“That is something that has a recognition in the regulation, and that's probably a guidance too of some of the ways that we'll see this play out with the machine identity used for machine learning, or LLMs in use as we look to the future.”
On events like the upcoming UK AI Safety Summit, which aims to bring global partners together to discuss AI safety and establish methods through which the risks of frontier models could be curbed, he struck a cautious tone.
“Certainly, we will not get agreement on regulation,” he said.
“We're not going to see companies agreeing to stop innovation or research, that's for sure. But I think the best that can come from this is awareness, which is a great step ahead.”
Though Bocek told ITPro the private sector will ultimately be accountable for AI safety, the work being done on this issue across the sector remains fragmented.
Gartner’s Avivah Litan recently told ITPro that the Frontier Model Forum, a partnership between Microsoft, Google, OpenAI, and Anthropic, intended to lay out new standards for frontier model safety, had “practically zero” chance of producing a common solution for AI safety due to competition.
