Researchers have called DeepMind's use of NHS data "inexcusable" and called for more transparency with NHS and healthcare data deals.
Last year, Google's parent firm Alphabet announced that DeepMind, the British deep learning firm it acquired, was working with the NHS on a kidney disease monitoring app called Streams.
Shortly after the announcement, it was revealed by the New Scientist that DeepMind had access to a wide range of healthcare data on 1.6 million patients at three Royal Free NHS Trust hospitals in London.
DeepMind and the NHS revamped the data sharing agreement at the end of last year, but the first deal is still being investigated by the Information Commissioner's Office.
"Initially received with great enthusiasm, the collaboration has suffered from a lack of clarity and openness, with issues of privacy and power emerging as potent challenges as the case has unfolded," noted a Health and Technology paper, written by former New Scientist journalist Hal Hodson and University of Cambridge researcher Dr Julia Powles.
The paper said the failure of the hospital and DeepMind to engage with patients and citizens about their health data was "inexcusable", particularly following similar issues with the failed care.data patient record sharing scheme and the findings from various reports into data practices, including the 2013 Caldicott review.
"The clear takeaway from these reports and recommendations and indeed the entire regulatory apparatus around healthcare is that patients should be able to understand when and why their health data is used, with realistic options for effective choice," the paper noted. "Patients should not be hearing about these things only when they become front-page scandals."
The deal between DeepMind and the NHS hospital should serve as a "cautionary tale and a call to attention," the paper concluded.
"Digital pioneers who claim to be committed to the public interest must do better than to pursue secretive deals and specious claims in something as important as the health of populations," the paper said. "For public institutions and oversight mechanisms to fail in their wake would be an irrevocable mistake."
DeepMind and the hospital released a joint statement: "This paper completely misrepresents the reality of how the NHS uses technology to process data. It makes a series of significant factual and analytical errors, assuming that this kind of data agreement is unprecedented. In fact, every trust in the country uses IT systems to help clinicians access current and historic information about patients, under the same legal and regulatory regime."
In response, the authors noted that the accusations of "factual inaccuracy and analytical error were unsubstantiated" and called on the deep learning firm and hospital to respond "on the record in an open forum".
