I actually think this is just further manifestation of the massive scam that is health insurance.

Artificial inflation

I hate AI, but this is just sour grapes from an insurance company. Medical providers and insurance companies are in a constant battle over payment with insurance companies trying ever more creative ways to deny payment and services (which I think anyone who has been a patient in the US can vouch for). This article claims that the providers are using AI guidance to justify better compensation from the insurance companies. If the compensation to providers for the same medical procedure didn't keep dropping down every year, this arms race wouldn't need to exist. What we really need is a purge of non-clinical hospital administrators (whose numbers have exploded over the past 2 decades) and of insurance personel with multi-million dollar salaries.

A lot of medical AI systems have potential but aren’t there yet. Like with LLM’s they are often oversold and underdeliver. But we will need them in the end if doctors are going to have to do a multitude of the work with the same amount of staff. But need to make sure we steer well clear of malignant AI companies looking to make a quick buck.

So this is a phase of trial and error. We have to do independent studies on software and look more at academically driven and open source options.