Google has completed a controversial take-over of the health division of its UK AI acquisition, DeepMind.
The personnel move had been delayed as National Health Service (NHS) trusts considered whether to shift their existing DeepMind contracts — some for a clinical task management app, others involving predictive health AI research — to Google.
In a blog post yesterday Dr Dominic King, formerly of DeepMind (and the NHS), now UK site lead at Google Health, confirmed the transfer, writing: “It’s clear that a transition like this takes time. Health data is sensitive, and we gave proper time and care to make sure that we had the full consent and cooperation of our partners. This included giving them the time to ask questions and fully understand our plans and to choose whether to continue our partnerships. As has always been the case, our partners are in full control of all patient data and we will only use patient data to help improve care, under their oversight and instructions.”
The Royal Free NHS Trust, Taunton & Somerset NHS Foundation Trust, Imperial College Healthcare NHS Trust, Moorfields Eye Hospital NHS Foundation Trust and University College London Hospitals NHS Foundation Trust all put out statements yesterday confirming they have moved their contractual arrangements to Google.
In the case of the Royal Free, patients’ Streams data is moving to the Google Cloud Platform infrastructure to support expanding use of the app which surfaces alerts for a kidney condition to another of its hospitals (Barnet Hospital).
One NHS trust, Yeovil District Hospital NHS Foundation Trust, has not signed a new contract — and says it had never deployed Streams, suggesting it had not found a satisfactory way to integrate the app with its existing ways of working — instead taking the decision to terminate the arrangement. Though it’s leaving the door open to future health service provision from Google.
A spokeswoman for Yeovil hospital sent us this statement:
We began our relationship with DeepMind in 2017 and since then have been determining what part the Streams application could play in clinical decision making here at Yeovil Hospital.
The app was never operationalised, and no patient data was processed.
What’s key for us as a hospital, when it comes to considering the implementation of any new piece of technology, is whether it improves the effectiveness and safety of patient care and how it tessellates with existing ways of working. Working with the DeepMind team, we found that Streams is not necessary for our organisation at the current time.
Whilst our contractual relationship has ended, we will remain an anchor partner of Google Health so will continue to be part of conversations about emerging technology which may be of benefit to our patients and our clinician in the future.
The hand-off of DeepMind Health to Google, which was announced just over a year ago, means the tech giant is now directly providing software services to a number of NHS trusts that had signed contracts with DeepMind for Streams; as well as taking over several AI research partnerships that involve the use of NHS patients’ data to try to develop predictive diagnostic models using AI technology.
DeepMind — which kicked off its health efforts by signing an agreement with the Royal Free NHS Trust in 2015, going on to publicly announce the health division in spring 2016 — said last year its future focus would be as a “research organisation”.
As recently as this July DeepMind was also touting a predictive healthcare research “breakthrough” — announcing it had trained a deep learning model for continuously predicting the future likelihood of a patient developing a life-threatening condition called acute kidney injury. (Though the AI is trained on heavily gender-skewed data from the US department of Veteran Affairs.)
Yet it’s now become clear that it’s handed off several of its key NHS research partnerships to Google Health as part of the Streams transfer.
In its statement about the move yesterday, UCLH writes that “it was proposed” that its DeepMind research partnership — which is related to radiotherapy treatment for patients with head and neck cancer — be transferred to Google Health, saying this will enable it to “make use of Google’s scale and experience to deliver potential breakthroughs to patients more rapidly”.
“We will retain control over the anonymised data and remain responsible for deciding how it is used,” it adds. “The anonymised data is encrypted and only accessible to a limited number of researchers who are working on this project with UCLH’s permission. Access to the data will only be granted for officially approved research purposes and will be automatically audited and logged.”
It’s worth pointing out that the notion of “anonymised” high dimension health data should be treated with a healthy degree of scepticism — given the risk of re-identification.
Moorfields also identifies Google’s “resources” as the incentive for agreeing for its eye-scan related research partnership to be handed off, writing: “This updated partnership will allow us to draw on Google’s resources and expertise to extend the benefits of innovations that AI offers to more of our clinicians and patients.”
Quite where this leaves DeepMind’s ambitions to “lead the way in fundamental research applying AI to important science and medical research questions, in collaboration with academic partners, to accelerate scientific progress for the benefit of everyone”, as it put it last year — when it characterized the hand-off to Google Health as all about ‘scaling Streams’ — remains to be seen.
We’ve reached out to DeepMind for comment on that. Update: The company told us it is now purely focused on fundamental science research areas, which includes medical questions, rather than applied healthcare — citing its research into protein folding as an example. Whereas it said Moorfields and UCLH are interested in the translation of research into applicable technologies beyond fundamental research, making Google Health a better fit.
Co-founder Mustafa Suleyman, who’s been taking a leave of absence from the company, tweeted yesterday to congratulate the Google Health team.
When we started DeepMind Health 3 years ago it was because we believed good research and smart software could make a difference to patients & nurses & doctors. Proud to be part of this journey. Huge progress delivered already, and so much more to come for this incredible team. https://t.co/zynBrrlgUc
— Mustafa Suleyman (@mustafasuleyman) September 18, 2019
DeepMind’s NHS research contracts also transferring to Google Health suggests the tech giants wants zero separation between core AI health research and the means of application, using its own cloud infrastructure, of any promising models it’s able to train off of patient data and commercialize by selling to the same healthcare services providers as apps and services.
You could say Google is seeking to bundle access to the high resolution patient data that’s essential for developing health AIs with the provision of commercial digital healthcare services it hopes to sell hospitals down the line, all funnelled through the same Google cloud infrastructure.
As we reported at the time, the hand-off of DeepMind Health to Google is controversial.
Firstly because the trust that partnered with DeepMind in 2015 to develop Streams was later found by the UK’s data protection watchdog to have breached UK law. The ICO said there was no legal basis for the Royal Free to have shared the medical records of ~1.6M patients with DeepMind during the app’s development.
Despite concerns being raised over the legal basis for sharing patients’ data throughout 2016 and 2017 DeepMind continued inking NHS contracts for Streams — claiming at the time that patient data would never be handed to Google. Yet fast forward a couple of years and it’s now literally sitting on the tech giant’s servers.
It’s that U-turn that led the DeepMind to Google Health hand-off to be branded a trust demolition by legal experts when the news was announced last year.
This summer the UK’s patient data watchdog, the National Data Guardian, released correspondence between her office and the ICO which informed the latter’s 2017 finding that Streams had breached data protection law — in which she articulates a clear regulatory position that the “reasonable expectations” of patients must govern non-direct care uses for people’s health data, rather than healthcare providers relying on doctors to decide whether they think the intended purpose for people’s medical information is justified.
The Google Health blog post talks a lot about “patient care” and “patient data” but has nothing to say about patients’ expectations of how their personal information should be used, with King writing that “our partners are in full control of all patient data and we will only use patient data to help improve care, under their oversight and instructions”.
It was exactly such an ethical blindspot around the patient’s perspective that led Royal Free doctors to override considerations about people’s medical privacy in the rush to throw their lot in with Google-DeepMind and scramble for AI-fuelled predictive healthcare.
Patient consent was not sought for passing medical records then; nor have patients’ views been consulted in the transfer of Streams contracts (and people’s data) to Google now.
And while — after it was faced with public outcry over the NHS data it was processing — DeepMind did go on to publish its contracts with NHS trusts (with some redactions), Google Health is not offering any such transparency on the replacement contracts that have been inked now. So it’s not clear whether there have been any other changes to the terms. Patients have to take all that on trust.
We reached out to the Royal Free Trust with questions about the new contract with Google but a spokeswoman just pointed us to the statement on its website — where it writes: “All migration and implementation will be completed to the highest standards of security and will be compliant with relevant data protection legislation and NHS information governance requirements.”
“As with all of our arrangements with third parties, the Royal Free London remains the data controller in relation to all personal data. This means we retain control over that personal data at all times and are responsible for deciding how that data is used for the benefit of patient care,” it adds.
In another reduction in transparency accompanying this hand-off from DeepMind to Google Health, an independent panel of reviewers that DeepMind appointed to oversee its work with the NHS in another bid to boost trust has been disbanded.
“As we announced in November, that review structure — which worked for a UK entity primarily focused on finding and developing healthcare solutions with and for the NHS — is not the right structure for a global effort set to work across continents as well as different health services,” King confirmed yesterday.
In its annual report last year the panel had warned of the risk of DeepMind exerting “excessive monopoly power” as a result of the data access and streaming infrastructure bundled with provision of the Streams app. For DeepMind then read Google now.
Independent experts raising concerns about monopoly power unsurprisingly doesn’t align with Google’s global ambitions in future healthcare provision.
The last word from the independent reviewers is a Medium post penned by former chair, professor Donal O’Donoghue — who writes that he’s “disappointed that the IR experiment did not have the time to run its course and I am sad to say goodbye to a project I’ve found fascinating”.
“This was a fascinating exploration into how a new governance model could be applied to such an important area such as health,” he adds. “It’s hard to know how this would have developed over the years but… what is clear to me is that trust and transparency are of paramount importance in healthcare and I’m keen to see how Google Health, and other providers, deliver this in the future.”
But with trust demolished and transparency reduced Google Health appears to have learnt exactly nothing from DeepMind’s missteps.
Google gobbling DeepMind’s health app might be the trust shock we need
Comment