Microsoft has unveiled the launch of a new open automation framework aimed at aiding security teams to red team generative AI systems.
The Python Risk Identification Toolkit for generative AI (PyRIT) will “empower” security staff and machine learning engineers to identify and mitigate risks within generative AI systems more efficiently, the tech giant said.
Abstraction and extensibility is built into PyRIT through five interfaces. These include targets, datasets, scoring engine, attack strategies, and memory.
Notably, PyRIT offers two separate attack styles. The first, known as “single-turn,” involves sending a combination of jailbreak and harmful prompts to a target AI system before scoring the response.
The second is called a “multiturn” strategy, whereby PyRIT sends the same combination of prompts, again scores the response, but then responds back to the AI system depending on the score. This allows security teams to investigate more realistic adversarial behavior.
Microsoft noted that, while this tool automates tasks, it is not a “replacement” for the manual red teaming of generative AI systems. Instead, it acts as a form of augmentation to existing red team expertise.
As is often the focus for automation tools, the idea is to eliminate more tedious workloads, while keeping the human team in control of strategy and execution.
The biggest advantage Microsoft says it has experienced is in efficiency gain.
Through an exercise on a Copilot system, the firm reported that it was able to “pick a harm category, generate several thousand malicious prompts, and use PyRIT’s scoring engine to evaluate the output” in a matter of hours rather than a matter of weeks.
“At Microsoft, we believe that security practices and generative AI responsibilities need to be a collaborative effort. We are deeply committed to developing tools and resources that enable every organization across the globe to innovate responsibly with the latest artificial intelligence advances,” Microsoft said.
Microsoft: Red teaming AI systems is too complex
Several factors lend a heightened level of complexity to red teaming AI systems, and automation can help in making this complexity a little bit more manageable.
According to its experience red teaming generative AI systems, Microsoft cited three ways in which generative AI security risks are more difficult to deal with than traditional security risks.
RELATED WHITEPAPER
In the first instance, there is an added set of issues which red teams need to look out for.
When red teaming traditional software and classical AI systems, the focus is solely on security vulnerabilities. With generative AI, though, there is also the additional concern of responsible AI which often manifests itself in the form of biased or inaccurate content.
Generative AI is more probabilistic than traditional software as well, meaning the red teaming process isn't as simple as executing a single, default attack path which would work on a traditional system.
Generative AI can provide different outputs in response to the same input, adding a layer of “non-determinism” that makes the red teaming process less straightforward.
The architecture of generative AI systems can also vary considerably. They can be standalone applications or they can form parts of existing systems, while the sort of content they produce, be it text, picture, or video, can differ radically.
“To surface just one type of risk (say, generating violent content) in one modality of the application (say, a chat interface on browser), red teams need to try different strategies multiple times to gather evidence of potential failures,” Microsoft said.
“Doing this manually for all types of harms, across all modalities across different strategies, can be exceedingly tedious and slow,” it added.
-
Bigger salaries, more burnout: Is the CISO role in crisis?In-depth CISOs are more stressed than ever before – but why is this and what can be done?
-
Cheap cyber crime kits can be bought on the dark web for less than $25News Research from NordVPN shows phishing kits are now widely available on the dark web and via messaging apps like Telegram, and are often selling for less than $25.
-
So long, Defender VPN: Microsoft is scrapping the free-to-use privacy tool over low uptakeNews Defender VPN, Microsoft's free virtual private network, is set for the scrapheap, so you might want to think about alternative services.
-
Hackers are on a huge Microsoft 365 password spraying spree – here’s what you need to knowNews A botnet made up of 130,000 compromised devices has been conducting a huge password spraying campaign targeting Microsoft 365 accounts.
-
Everything you need to know about the Microsoft Power Pages vulnerabilityNews A severe Microsoft Power Pages vulnerability has been fixed after cyber criminals were found to have been exploiting unpatched systems in the wild.
-
Microsoft is increasing payouts for its Copilot bug bounty programNews Microsoft has expanded the bug bounty program for its Copilot lineup, boosting payouts and adding coverage of WhatsApp and Telegram tools.
-
Hackers are using this new phishing technique to bypass MFA
News Microsoft has warned that a threat group known as Storm-2372 has altered its tactics using a specific ‘device code phishing’ technique to bypass MFA and steal access tokens.
-
A new phishing campaign is exploiting Microsoft’s legacy ADFS identity solution to steal credentials and bypass MFANews Researchers at Abnormal Security have warned of a new phishing campaign targeting Microsoft's Active Directory Federation Services (ADFS) secure access system.
-
Hackers are using Microsoft Teams to conduct “email bombing” attacksNews Experts told ITPro that tactics like this are on the rise, and employees must be trained effectively
-
Microsoft files suit against threat actors abusing AI services
News Cyber criminals are accused of using stolen credentials for an illegal hacking as a service operation
