Spying, internet surveillance and hacking are secretive worlds, but sometimes the need for secrecy can get in the way of their ultimate goals. Being asked to show how you have obtained evidence can mean giving up an asset that is worth more than the actual evidence itself.
The predicament was highlighted last year in the US, when FBI officials used a previously unknown exploit to infiltrate, take over and investigate a dark web child abuse ring.
The PlayPen web group breach where government officials reportedly used a Tor browser flaw to identify the IP addresses of members led to the identification of 135 suspects in the US and 8,700 members in 120 countries.
However, as the case of one defendant came to trial, court rulings made it clear that, to seal a conviction, the officials would need to disclose how the evidence against the alleged paedophiles was obtained. Rather than reveal their exploits, federal prosecutors dropped the case, but were able to keep the possibility of further legal action alive presumably in case the exploit was later made public and no longer had value as a stealth tool.
The ruling "deprived the government of the evidence needed to establish defendant Jay Michaud's guilt beyond a reasonable doubt at trial," the prosecutors said when dropping the case. "The government must now choose between disclosure of classified information and dismissal of its indictment. Disclosure is not currently an option.
"Dismissal without prejudice leaves open the possibility that the government could bring new charges should there come a time when the government be in a position to provide the requested discovery."
Fragile evidence
Although the idea of potentially letting thousands of paedophiles off the hook is unpalatable, experts say that refusing to disclose methods is sometimes in the interest of the greater good. "There are cases where secrecy matters for a while, as methods can be fragile," said Ross Anderson, a security engineer and computing professor at the University of Cambridge.
"When we analyse malware families and publish our research, we may withhold information about some design error in the software that makes the malware easy to detect," he said. "When I worked on click fraud, we could often detect a botnet because its behaviour failed one of a large number of randomness tests. We'd keep quiet about that too."
What's true for researchers is equally true for security officials. One security insider said long before the data-sniffing revelations of the Snowden Files that even if agencies could break encryption, they would never make it known, preferring to let adversaries send messages in the mistaken belief that their messages were safe.
However, experts agree the secrecy can have a negative effect if taken too far. "The agencies have built secrecy into a cult, in ways that are counterproductive," said Anderson. "Recently when people from the security service came to a Royal Society event, they made themselves conspicuous by wearing blank name badges; and the recent furore over the CIA leaks showed that that organisation has serious problems dealing with its own attack tools."
Keeping cyberweaponry secret is also problematic from a "classified status" point of view, as making it a state secret would prevent it being used without legal implications. "You can't classify a piece of attack code as 'Secret' if you plan to embed it in a Russian diplomat's laptop, as he doesn't have security clearance," explained Anderson. "But how do you work with unclassified material in an environment where everything else is classified?"
Showing your hand
One of the few reasons a surveillance team might show off something in its arsenal is to thwart or intimidate another group of state hackers, in the same way actual weapons are stockpiled as a deterrent. "A preventive cyber-attack on or quick retaliation against the computer networks of other countries suspected of providing support to hackers may appear the only response capable of deterring future incidents," reads a report by the European Union Institute for Security Studies on the subject. "Such actions, however, may undermine the international system in the long run and further muddy the already difficult international debate surrounding cyber norms."
Legislation can also be used to prevent technical capabilities from being revealed. The Freedom of the Press Foundation in the US has been trying to persuade judges to make the FBI reveal how it snoops on journalists using "National Security Letters", but has been stonewalled.
These FBI legal tools, which need no judicial oversight, involve accessing communications data, and prohibit a target's ISP from discussing the data request. Following freedom of information requests from the Freedom of the Press Foundation, a US judge ruled that publicly disclosing the FBI's "methods on how it spies on journalists could hamper national security". Such a position makes it impossible to protect journalists, the group claims.
Compromising everyone's safety
The situation in which governments hoard secret details of vulnerabilities that they can use to target individuals and organisations angers both privacy advocates and technology firms, which are often chastised if they don't make vulnerabilities and fixes public at the earliest opportunity. "We're all made less safe by the CIA's decision to keep rather than ensure the patching of vulnerabilities," said Cindy Kohn, executive director of the Electronic Frontier Foundation "Even spy agencies like the CIA have a responsibility to protect the security and privacy of Americans."
"The agency appears to have failed to accurately assess the risk of not disclosing vulnerabilities to responsible vendors and failed to follow even the limited Vulnerabilities Equities Process," Kohn added.
The insistence on secrecy also concerns many privacy advocates who feel consumers are treated like pawns, both by software companies that don't always disclose security breaches and the security services that actively create them. "Almost every aspect of cybersecurity involves information asymmetry," said Privacy International's ireann Leverett. "Privacy is no different. As a consumer, you don't know if companies take strong steps to protect your privacy or not, and companies that refuse to grant government access without warrants don't know if security services can compromise them anyway," he said.
"In short, security and privacy are a consumer issue, and the governments cannot be both be adversary and defender of privacy."
