‘But the AI is super good at updating code!’ — well, no, it isn’t (pivot-to-ai.com)

submitted 1 day ago by dgerard@awful.systems to c/techtakes@awful.systems

3 comments fedilink hide all child comments

‘management started quietly asking us to write more of it ourselves again’

https://www.youtube.com/watch?v=8JA8i0Cks6Y&list=UU9rJrMVgcXTfa8xuMnbhAEA - video
https://pivottoai.libsyn.com/20261111-but-the-ai-is-super-good-at-updating-code-well-no-it-isnt - podcast

time: 6 min 03 sec

you are viewing a single comment's thread
view the rest of the comments

[-] zbyte64@awful.systems 2 points 6 hours ago

There is a benchmark out there that hints at this phenomenon: https://www.scbench.ai/ The idea is to track how well the model modifies code over multiple change requests. No surprise for readers here that the LLM will bitrot your code quickly, and the odds of it doing so is worse than a coin toss.

this post was submitted on 12 Nov 2026

26 points (96.4% liked)

TechTakes

2781 readers

147 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 3 years ago

MODERATORS

dgerard@awful.systems