The tests were conducted in a controlled benchmark setting, meaning its usefulness in a true medical environment is up for debate. Still, Microsoft was eager to declare victory.

...

But the study could only imperfectly simulate the back and forth a doctor has with a patient by having DxO talk to a “gatekeeper agent” that “mediates all information requests drawing from the underlying case record.” Put another way, it’s an AI diagnosing a patient that’s played by an AI.

I wonder how many news outlets covered this breathless headline with the caveat that this was AI talking to AI being used to "prove" AI is better at diagnosing humans than humans.....