It depends on many things. The hard line for me would be is this running locally, on a server with the same IT management as my actual data, or on a third party servers. If the doctor either don't know this, or can't give adequate proof that it isn't running on some third party servers, then all the "prioritize your privacy" aren't worth shit.

But that's only the point where I give a hard no. The way it is used would also matter a lot. Is it used as a clutch for reference searching, or a full self driving decision making process that will write me a prescription in the end? This part is the same whether it's for medical advice or for anything else: if the user is skilled enough to be able to evaluate/validate the output of the process faster than it would have taken them to do it manually, then there might be some value. Some usages fits into this. Some don't. Summarizing large documents you did not read does not work as a safe thing, because, you'd have to read the document to check the summary. Getting the summary of a drug/sickness/whatever that you know about but need a reminder of, could be ok.

tl;dr: it have to run in a privacy-enabled context (no third parties), it have to be used as a clutch (no skipping work), and the user have to keep is brain en mental activity alive enough to steer the system instead of being dragged by it. As things stands right now, I doubt there's a lot of doctors that would fit all three points, but in the future, maybe.

Nope

Hard to say this about any doctor.

I have the one whom I trust, fundamentally, for real. I'd give him a free pass even for that.

All the others, I would abandon.

If my doctor needs AI to treat me then they ain't no doctor, they're middlemen.

You're asking a forum that's in a strongly anti-AI bubble, so the answer you're going to get is both obvious and useless. You might as well be asking a bunch of Vegans whether you should have steak for dinner. You admit yourself to having a strong knee-jerk reaction.

However, it looks to me like their use of AI is a perfectly reasonable one. It's just making transcripts and summaries of sessions. I do that all the time with personal logs and meetings, I've got a couple of local models that run entirely on my computer. So I don't see the big deal here.

If you feel you really need this medical help then maybe don't rely on the advice of a bunch of people you know are going to instantly react negatively to AI regardless of any other details.

Absolutely not. If they’re going to defer me to a machine instead of their decades of work and expertise, I will find someone else.

AI summaries often make up details, omit what is important, and get stuff wrong. Every error may follow you forever complicating diagnosis and treatment and ultimately can harm or kill you.

One of our doctors started using AI transcription and summary. I find it lacking substance after AI is done summarizing. You can see her thought process when she type her notes, it's thorough but concise. The AI summary is definitely short, but it's not about shortening, it's about handing your note to another doctor and that doctor is able to follow through with the plan.

Your provider then reviews the content of that note to ensure its accuracy and completeness.

You know they're not gonna do that, in practice.

Show him the EULA for copilot (where it's for entertaining purposes only), and tell him you'll be going elsewhere and leaving an appropriate review.

Jokes on you, they don't even give you time to read it and expects you to sign everything...

Just tell them you don't want it and they will almost certainly accommodate. Most therapists are still trying to get a sense of the tool and how their patients will feel about it, they won't be looking to fire patients over it.

I've been looking at the SimplePractice tool as I am IT guy for a couple of therapy clinics, plus my own therapist wanted a guinea pig. I am not thrilled by the terms of service, which allow the company to do whatever they want internally with your data. But the tool is legit useful for therapists (it's to help them justify your ongoing care to your insurance company, who wants specifics on the how but not the what you're discussing. The therapist winds up reviewing the summarization before it's sent to the insurance company. It won't affect your care. The LLM is supposed to be HIPAA compliant, but I am really curious about how that works.

I am looking more at on-device models for the future. Notetaking is a big burden for therapists, who often wind up keeping multiple versions of your notes: one for you (you have the right to request them), one for insurance which is coded to their treatment plans, one for the court or judge if there's a divorce or other proceeding involved. One for the parent if it's a teenager. So a tool like this has the real power to cut back on a therapist's weekend admin time. There's just no reason it can't be a local model. A Mac Mini sitting in the corner, with a mic input switch for clients who don't want it on.

I can't answer to mental health, but for a GP, I'd be surprised if many of them would use it. There's got to be some pride involved, no? I don't think they'd trust the damn things anymore that we do. So, maybe they'd make gestures to appease their stupid administrators, but they have the power to ignore the things and do what they want, at least for now.

I have all sorts of anxiety surrounding AI. Most of the anxiety comes from the misuse, copyright issues and departure from critical and creative thinking. However, one of the fields I actually think it could be very useful and of great benefit is medicine.

That being said, I'd be a no as well. The way this is worded and he track record we've seen with privacy doesn't fill me with much confidence. Feels like another instance off loading of thinking rather as a tool for better diagnosis.

It sounds like America from the process. The confluence of commercialization of healthcare and tools that can make it look like time and attention has been used leads to some bad places. I'd be very sceptical about any advice medical or otherwise I recieved.

The unfortunate truth is that without these tools the cost of care will be higher for health companies not using the tools. Which means bespoke human led care will be a luxury in America in the near future. I don't think it's a reality you are going to be able to avoid.

I would push back at every opportunity, double check all of the information you are getting, ask pointed "why this" questions, make doctors clearly communicate that they are the ones giving the recommendation. At the end of the day a good doctor with AI tools is likely to do a better job.

For note taking only, id be fine IF it was all run locally with no ability to be trained on.

Id want assurances from the Dr that they also carefully review the notes immediately after or that I get to see the notes before leaving due to the risk of hallucinations that could cause future care problems.

They could have it visible on a screen while youre in the room with you to help you be sure its accurate.

Edit: id care less about it being local if it wasn't medical/legal in nature.

nope

My GP is at a big medical center and the medical center coded their own internal AI transciptionist that listens to our conversations. It is sandboxed and only has access to internal systems. Overall, it saves my GP time and the notes are accurate. I asked a lot of questions when it first came out, and I think you should ask questions about it too.