Artificial intelligence in the real world

The winner of Highland Marketing’s #HealthTechToShoutAbout award in 2020 was Royal Bolton NHS Foundation Trust for its deployment of AI in the fight against Covid-19. Our advisory board invited divisional medical director Rizwan Malik to talk about the project and the lessons that other trusts can learn.

As the novel coronavirus spread across Europe in spring 2020, chest imaging became an important tool for diagnosing patients with Covid-19. Clinicians found they could diagnose the disease from a chest x-ray far more quickly than they could confirm it from a blood test, which typically takes 24-hours to turn round.

As part of its Covid-response, Royal Bolton NHS Foundation Trust looked for a way to make it easier for its doctors to use chest x-rays as a primary tool for the diagnosis, without having to wait for a radiologist to become available.

It deployed a decision support tool from Qure.ai to help them interpret scans and calculate the radiographic severity score that indicates whether a patient is deteriorating and so likely to need oxygen support or intensive care.

An entry to the Health Tech Newspaper’s Health Tech Awards 2020 explained: “Doctors get a very quick interpretation, right at the time of scanning, about whether changes are likely to be Covid-related and, more importantly, whether things are getting worse or, of course, better.

“This has meant more efficient triaging and care of patients, optimised the use of critical care resources, and [made sure] the patient’s journey through the hospital is quick and streamlined.”

A solution for the long-term

Royal Bolton was the first NHS organisation to work with Quire.ai. However, the algorithm used at the trust had been trained on 25 million images to spot the abnormalities associated with TB. The algorithm, which had CE certification for TB, had then been prepped on another 11,000 images to identify the similar changes associated with Covid-19.

In the longer-term, it is the algorithm’s ability to track progression that is likely to be valuable. Royal Bolton’s award entry argued that this could enable medical staff to monitor and make decisions about patients without having to make regular calls on the over-stretched radiology department.

It could also support trainee radiologists and provide a quality check on the work of clinicians across the piece. Dr Gareth Hughes, critical care lead, wrote: “I think a significant benefit will be seen in the post-Covid era [because] it will improve diagnostic accuracy when our junior doctors first assess patients. It should also improve the correct coding and identification of community-acquired pneumonia.”

It was the potential for use beyond Royal Bolton and beyond the immediate crisis that impressed Highland Marketing’s judges, who made it the winner of the awards’ #HealthTechToShoutAbout category. Chair Jeremy Nettle said: “It shows how AI can be used to improve systems and processes, and that should be of great interest to other trusts that want to do the same.”

Challenges in the healthcare AI market  

For its last board meeting of 2020, the board invited Rizwan Malik, the trust’s division medical director, to expand on what he had learned from the Quire.ai deployment, and what other organisations that want to do the same should bear in mind.

Malik started by saying the trust had a bit of a head-start when Covid-19 hit, because it had set up a digital innovation board to discuss AI in response to approaches from companies.

“The board included our chief information officer, and the medical director, who was keen to see what we could do in the field, and a couple of radiologists who were interested in this space,” he said. “That was useful, because I had needed to get buy-in when this came up, I’d only just be getting sign-off now.”

One of the things the board had discovered was that there were hundreds of suppliers in the market, but only a small number had a CE mark – and even fewer had certification to use their algorithm as a diagnostic device.

Also, companies were coming to the trust and asking it to find a use for their products, instead of identifying a problem they could solve. In doing so, many were adopting an approach that focused on cost-saving and doing away with radiologists: “which is like trying to sell mint-sauce to lamb chops.”

Yet, Malik said, they were often “defensive” about their data sources and who would be responsible if a deployment had adverse consequences. To make progress, the digital innovation board had come up with a set of questions to ask:

Where is the data coming from, and is it relevant to the population that the algorithm will be applied to? Who owns the data and who benefits commercially? Are suitable data sharing agreements in place?

How secure is the data, and “if it is squirted into a cloud-based product, what actually happens to it”? Does the product have approval and, if it does, is that approval relevant to the use-case at hand? And: “Why do it, if nobody has done it before?”

Malik said it’s possible to turn this last question around, and say: “Why can’t we do it, just because nobody has done it before?” But there needs to be a good use-case – “a problem to solve, and a way to solve it” – plus good arguments to address clinical concerns, and a decent business case – “because if it just looks expensive, it’s going to be ‘chief executive says no’.”

Ready to address the Covid-19 crisis  

In essence, Royal Bolton was able to move ahead quickly with the Quire.ai algorithm because it could answer these questions. “I can pinpoint the day that this went from an abstract thought to something that we should do,” Malik said.

“It was a day in February, when I was watching a TV report about Italy, where doctors were falling like flies and those remaining were having to make these terrible decisions about who should get ventilator beds. I wondered if we could use AI to make better decisions. I read about this company, Qure.ai, and it seemed to have something presentable to address that.”

Qure.ai could answer the data question. It had a mature product, trained on TB and re-tooled on Covid-19 scans. “So, we knew the data was applicable to what we wanted to do.” When it came to security, Qure.ai had a cloud-based product, but Royal Bolton asked it to install a server within its own firewall. “So even Qure.ai is unable to see it without a timed permission from our IT department.”

On regulation Malik explained that: “The trust and Qure.AI held a dialogue with the MHRA and worked with NHSX and the Royal College of Radiologists on a ‘cookbook’ and ‘guide’ on how to deploy AI and how to integrate it with radiology systems.”

And on cost Malik said: “Our medical director got around that by going to the finance director and saying: ‘there is a pandemic on, so you can have no hospital, or you can have this mitigation to help keep the lights on’. And, to be fair to the finance department, they came right on board.”  

With clinicians, communication is key

The advisory board were interested in the cultural aspects of the deployment and how Malik and his team had been able to address clinical concerns.

Malik said a lot came down to communication; being clear about who the product was intended to support, and what it could and couldn’t do. At Royal Bolton, the Qure.ai product has not changed the role of radiologists, who remain responsible for reporting.

Instead, it has been deployed to support doctors, but they remain responsible for diagnosing and treating patients. If the product throws up a false positive, for example, they can over-ride it. Otherwise, it provides a useful aid or backup for their skills.

“We were very clear that this was not about replacing people, but about support and quality,” Malik said. “And there was a point at which people went from being sceptical, to being accepting, to seeing the benefits.

“Initially, there was some ‘oh, we’ll humour him and do this to get him to go away’, and then people realised they were not being pushed out, but they were getting things done much faster, and now they want more.

“We have just had virtual-RSNA [the conference of the Radiological Society of North America] and two people who really challenged me on this deployment came back from the event and said they’d heard about a couple of things and when could we do them? That gave me a warm glow.”