377
Lawyer Cites ChatGPT-Invented Fake Witnesses in Murder Appeal (www.404media.co)
submitted 1 day ago by themachinestops@lemmy.dbzer0.com to c/technology@lemmy.world
49 comments fedilink hide all child comments
top 49 comments
sorted by: hot top new old
[-] uriel238@lemmy.blahaj.zone 13 points 20 hours ago

According to the order, the judges found Aarons to be in direct contempt of court, referred him to the Disciplinary Board for further consideration, and barred from appearing before the court pending the outcome of the board’s investigation. They also threw him off the case in question and assigned a public defender to his former client.

If they don't feel the consequences themselves, lawyers will just continue to use LLMs to do their research.

This is another example why we need LLMs to be required to cite sources.

[-] Armok_the_bunny@lemmy.world 3 points 18 hours ago

Or just ban LLMs from use in anything except recreational contexts at most.

[-] Fmstrat@lemmy.world 1 points 12 hours ago

Aarons said he hadn’t told his client directly that he was caught using ChatGPT on the case because he’s currently serving life in prison.

He got called out on this by the judge, too.

[-] finley@lemmy.zip 166 points 1 day ago* (last edited 1 day ago)

There needs to be a rule that anyone who does this should be immediately disbarred

[-] Goodlucksil@lemmy.dbzer0.com 7 points 1 day ago

Isn't presenting fake evidence enough for disbarring on that country?

[-] AltoCuddle@lemmy.world 100 points 1 day ago

I mean. He should be imprisoned, no? Does perjury not apply to the lawyers too?

[-] Davel23@fedia.io 77 points 1 day ago

As far as I'm aware perjury can only be committed by someone under oath. And lawyers are not placed under oath. But I agree, the penalty for this shit should be a lot harsher than it's been so far.

[-] Chozo@fedia.io 27 points 1 day ago

I've been to court, and it never even occurred to me that the lawyers are not placed under oath. But they're the ones that do most of the talking during a case. I wonder that the reasoning for not being under oath is.

[-] Armok_the_bunny@lemmy.world 2 points 18 hours ago

Putting a lawyer under oath would often mean forcing them to either violate that oath or violate attourney-client privledge, as part of the oath is "the whole truth".

[-] snooggums@piefed.world 33 points 1 day ago* (last edited 1 day ago)

They are advocating/representing, not testifying. A defense attorney does not make statements of fact, they call witnesses and experts for that. They introduce reasonable doubt.

They call witnesses and experts to present facts and other testomony which is why they are under oath.

[-] AltoCuddle@lemmy.world 10 points 1 day ago* (last edited 1 day ago)

Fair enough, but I don’t like it. I guess I’ll settle for contempt of court.

[-] snooggums@piefed.world 8 points 1 day ago

They are still able to be held accoubtable for breaking ethincs, and in my opinion they should be disbarred for presenting AI hallucinations and any other evidence they haven't spent any time verifying.

[-] Carl@anarchist.nexus 13 points 1 day ago* (last edited 1 day ago)

It’s because you only need to take an oath if you’re testifying. And arguing isn’t testifying. Lawyers are already bound by ethics standards for things like lying. For instance, if a lawyer knows you killed someone, they can’t openly say you didn’t.

They can argue that the evidence is lacking. They can argue that the prosecution hasn’t done enough to prove your guilt. They can find gaps in the prosecution’s case and start poking holes to provide reasonable doubt. But they can’t put a witness on the stand who they know is going to lie in your defense. And that includes themselves. Also, no lawyer would want to take the stand, because then the opposing lawyer would have a chance to cross-examine them directly while they’re under oath, and likely have them testify against their own client. It would be a very big can of worms.

[-] mkwt@lemmy.world 11 points 1 day ago

What the lawyers say is not under oath unless they go to the witness stand and swear in. The main reason is that lawyers mainly are in the court to make opinionated arguments, not to give facts.

Lawyers do have a "duty of candor" to the "tribunal" at all times, however. This is because they have taken an oath (different from the witness oath) to become "officers of the court". This means they cannot knowingly misstate facts or law to the court.

But the maximum exposure on a duty of candor violation is court sanctions, like fines or adverse court outcomes, possibly contempt, and bar discipline, and so on. It's not the crime of perjury.

[-] KAYDUBELL@lemmy.world 5 points 1 day ago

Lawyers are under oath, at least in my state. When they get sworn in by the Supreme Court they take an oath to essentially tell the truth in court and alert the court if they know of any lies (like if their client commits perjury). They just don’t have to be sworn in at every hearing.

[-] Karyoplasma@discuss.tchncs.de 3 points 1 day ago

Perjury requires intent to deceive anyway and being fucking stupid is not deceitful.

[-] Mastengwe@sh.itjust.works 2 points 1 day ago

Absolutely agree

[-] fartographer@lemmy.world 7 points 1 day ago

They should be subjected to whatever punishment is recommended by asking ChatGPT the most effective way to ensure someone would never again try to erode the foundations of society.

[-] finley@lemmy.zip 2 points 1 day ago

Interesting, but unreliable. Kinda the point here.

[-] mkwt@lemmy.world 15 points 1 day ago

There is a rule that covers fake citations, AI or not. But it's not immediate.

What's happens is the court makes a disciplinary referral to the state bar association, which handles barring and disbarring. At minimum there's some kind of additional bar discipline hearing before adverse action.

Each federal court, district or appellate, can also independently revoke admission to the bar of that individual court.

[-] plz1@sh.itjust.works 66 points 1 day ago

Last I checked, submitting false testimony was a crime. If these lawyer stop getting slaps on the wrist, the slop legal work will stop.

[-] blackshirt@lemmy.world 3 points 23 hours ago

stop the slop.

nice slogan.

[-] schipelblorp@sh.itjust.works 52 points 1 day ago

It seems whatever actions courts are taking, it's not enough if it keeps happening.

Aarons was sanctioned $5,000

How much do lawyers charge to defend a murder charge? I'm guessing a lot more than that.

[-] notthebees@reddthat.com 3 points 1 day ago

This is before the Bar association does something to him.

[-] schipelblorp@sh.itjust.works 2 points 20 hours ago

I did some digging, and it looks like the courts are the ones issuing all the fines. Couldn't find much in the way of disbarrments.

[-] notthebees@reddthat.com 2 points 19 hours ago

Probably because State Bar actions are currently happening

[-] schipelblorp@sh.itjust.works 1 points 19 hours ago* (last edited 19 hours ago)

Oh, yeah, I guess that all takes time. Looking forward to some entertaining reading in the future. Judging from this case, defenses based on ignorance are already getting thoroughly savaged, so hopefully we can expect some consequences.

[-] Armand1@lemmy.world 38 points 1 day ago

“It's little comfort to know that my stupidity is what brings us together this afternoon,” New Mexico attorney Stephen D. Aarons said in a hearing last month before five judges. He “assumed” ChatGPT would generate a “bulletproof summary of proceedings.” Instead, ChatGPT generated false testimony from “wholly fabricated witnesses,” according to an order of direct contempt from the New Mexico Supreme Court.

Aarons’ reasoning for trusting ChatGPT, he said in the August hearing, was that he’d heard of doctors using AI for medical research.

[-] floofloof@lemmy.ca 21 points 1 day ago

Some people are smart enough to be a lawyer yet still really fucking stupid.

[-] ctrl_alt_esc@lemmy.ml 8 points 1 day ago

You don't have to be smart to become a lawyer, lots of memorizing and some street smarts will do just fine

[-] wonderingwanderer@sopuli.xyz 1 points 1 day ago

Aarons’ reasoning for trusting ChatGPT, he said in the August hearing, was that he’d heard of doctors using AI for medical research.

I don't think the AI being used in medical research is ChatGPT (or any LLM for that matter), but if they are then that's concerning... He's still a dumbass though.

[-] MountingSuspicion@reddthat.com 9 points 1 day ago

I had to check the date of the article to make sure it was recent. How a lawyer hasn't heard of the several times other lawyers have had issues with ChatGPT I have no idea. There's no way this guy didn't perjure himself be feigning ignorance. I feel like the bar needs to send out a written notice requiring some kind of acknowledgment to all lawyers so they can no longer claim ignorance if this happens to them.

[-] Manjushri@piefed.social 14 points 1 day ago

In a quote in the article, one of the judges even points this out to him rather harshly.

“So, Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?” one of the judges asked Aarons. “Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”

....

“So, either you buried your head in the sand. And that's a choice to do that. An intentional choice to be uninformed,” the judge continued to Aarons. “Or, you took a gamble. And neither of those are consistent with the code of conduct. So I'm really struggling with your response, saying, ‘Well, I didn't understand. I didn't know about hallucinations.’ My 13-year-old nephew knows about hallucinations. My 75-year-old stepmother knows about hallucinations. So, either you made a decision not to be aware of what's happening in the world in using this tool appropriately, which is a willful choice. Or you decided to roll the dice.”

The article also provides links to five other cases where lawyers used an LLM and filed AI hallucinations as fact. This bozo is clearly in the find out phase.

[-] ryannathans@aussie.zone 6 points 1 day ago

You can claim ignorance for breaking the law?

[-] sunnie@slrpnk.net 12 points 1 day ago

If you’re one of the people tasked with enforcing the law, then yes you are not expected to know it. If you are a mere citizen, then you are expected to know the law inside and out.

[-] MountingSuspicion@reddthat.com 3 points 1 day ago

In theory, no. In practice, yes. I think it's generally fine for people to be able to use discretion in a lot of cases. That discretion is not always evenly applied, but there are plenty of laws that the average person doesn't know and has no reason to even think about. I'd rather live in a world where the justice system is able to give people leniency and the benefit of the doubt than otherwise. If one of your brake lights is out it's possible you have no idea. Technically it's illegal to be driving. Most people would agree that it's a good thing that you can be given a warning or something instead of have to face the same penalty as a person knowingly driving with the same issue.

[-] teslekova@lemmy.ml 1 points 1 day ago* (last edited 1 day ago)

You very explicitly cannot. The judge may take it into account when sentencing though.

[-] Armand1@lemmy.world 2 points 1 day ago

Yeah I heard of a case like this at least a year ago. I was wondering if we were reposting an old article, but no. That makes it worse, and explains the judge's reaction.

[-] Atelopus-zeteki@fedia.io 3 points 1 day ago

There are ways that LLMs are being used in science, AND a human mind needs to vet those findings for validity. smh.

[-] Triumph@fedia.io 2 points 1 day ago

Using AI for medical research probably doesn't mean using a publicly available chat bot.

[-] Tollana1234567@lemmy.today 0 points 1 day ago* (last edited 1 day ago)

AI is already being used for writing faux papers in journals, stem and medical ones. its just now they are getting caught. even using it for "medical scans" is highly suspect and likely wont see the light of day. a version of its being used in dental scans already by some.

[-] PrettyFlyForAFatGuy@feddit.uk 1 points 20 hours ago

I use AI for some of the documents i create

I also read and iterate on what it writes

Don't trust the hallucination machine

[-] Kjell@lemmy.world 4 points 20 hours ago

I have seen far too many times that people send meeting minutes generated by AI which is wrong on one or several points. After that it is difficult to trust anything from that person.

[-] 01189998819991197253@infosec.pub 9 points 1 day ago

Can't fix stupid.

But I don't have to interact with it.

[-] Trigger2_2000@sh.itjust.works 2 points 1 day ago

In the past, I've advocated for people to not use AI. I think I've changed my mind just now.

Everyone should be required to use AI - that way, we can see how stupid they are when they don't check it.

[-] frustrated_phagocytosis@fedia.io 1 points 1 day ago

Think he saw this ai summary and got all excited like, look at all this evidence in support of reasonable doubt for my client's guilt! Guess the previous defense just ignored all this obviously helpful witness testimony for some reason...

this post was submitted on 05 Oct 2026
377 points (99.2% liked)

Technology

88627 readers
2942 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws