55
submitted 1 day ago* (last edited 1 day ago) by inari@piefed.zip to c/programming@programming.dev
you are viewing a single comment's thread
view the rest of the comments
[-] dan@upvote.au 44 points 1 day ago

This is interesting because even though they designed the language to be impossible for LLMs to use, Claude still scored nearly 70% on their benchmark.

[-] OldMrFish@feddit.dk 24 points 21 hours ago* (last edited 20 hours ago)

Alternative headline: Claude scores 70% on benchmark in esoteric language it hasn't been trained on. That's basically an ad for Claude Code..

[-] dan@upvote.au 4 points 17 hours ago

Not only was Claude not trained on it, but the language is actively trying to be hostile towards it.

[-] Mirodir@discuss.tchncs.de 21 points 22 hours ago

And that's without any of this being in its training set. I wonder if they realized how big of a self own that is.

[-] aldhissla@beehaw.org 7 points 22 hours ago

As well-intentioned as this might be, I doubt they understand how trained models work.

this post was submitted on 19 Oct 2026
55 points (76.7% liked)

Programming

28886 readers
941 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS