Society faces a massive "crisis of trust" with regards to data-driven innovation which could lead to heavy legal restrictions unless it's resolved urgently.
Public confidence is critical to advancements in a host of areas like IoT and smart cities according to the Information Commissioner's Office (ICO) executive director for technology policy and innovation Simon McDougall. Trust, he added, is especially critical to the government's national data strategy.
This is because, without trust as to how personal data is being used, shared, and processed, even the most coherent of national strategies will collapse when the public mood turns foul.
"I have to say, that as one single metric right now, we ain't doing so good. And nobody here's doing that good," he told audience members at a forum on the UK's national data strategy, held in London on Tuesday.
He added that if the ICO had established a physical dashboard showing levels of public trust there'd be "red lights blinking everywhere, klaxons blaring, various ticker tape sprawling out saying 'danger, danger'".
He was speaking directly to an audience that comprised representatives from the tech industry, and technical staff with other organisations. Attendees included BT, Fujitsu and KPMG, as well as public sector organisations HMRC and the Office for National Statistics.
"We are having a crisis of trust right now in how innovation is happening, how our data is being used," he continued. "And that is a challenge for us a regulator, this is a challenge for everybody in this room, and it's a challenge for anybody envisioning new and innovative uses of data in the future."
He added that trust slumped following a series of historic bungled government IT projects and schemes such as New Labour's confused ID cards system. But the private sector has "really driven the crisis" in the last few years, with the likes of the Cambridge Analytica and Facebook scandal.
McDougall was joined at the event by a host of speakers including chair of the digital framework task group Mark Enzer, and Roger Taylor, chair of the government's Centre for Data Ethics and Innovation.
Taylor highlighted in his comments that a failure to address ethical concerns now will be extremely damaging, but that success could lead to major benefits; adding a high-risk/high-reward dimension to any national data strategy.
He argued that one clear benefit of harnessing the power of data is the creation of a more open and accountable society.
This may seem like a strange thing to say, he added, at a time when these technologies are being associated with abuse, unethical practices, and a lack of accountability for powerful organisations like social media companies. This is in addition to how states such as China are harnessing citizens' data in authoritarian ways, like the recently-introduced social credit.
"To suggest that there is a real opportunity in terms of creating open and transparent and democratic society might seem a little counter-intuitive," said Taylor. "But it's worth just stressing that all these problems I've just described, they are all down to a common cause. And that is the power of these intelligent networks to understand how we are behaving collectively.
"That power exists. And what we have not yet really just found is a way we can harness that power in a way that is consistent with democratic values, and that is consistent with the sort of open, pluralistic society we want. But the opportunity to get that right is enormous."
As for the consequences of failing to close this trust deficit, the ICO's McDougall pointed to a 'GM foods moment' in which any potential for exploring data-driven innovation is neutered due to overwhelming public cynicism.
"We went straight from there being some initiatives and trials to there being a media outcry and the whole thing being banned. And we did not realise either the risks or opportunities of behind that because we stopped too soon."
He added that tech firms and organisations like his own must work to restore public trust in data use. If not, then initiatives such as facial recognition and sharing large datasets within the NHS, for example, risk suffering the same fate.
