you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 02 Nov 2026
17 points (94.7% liked)
TechTakes
2778 readers
373 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 3 years ago
MODERATORS
I looked again, and it seems they just flat out deleted every other ECI measurement except for Astra
Here’s what it looked like then
and heres what it looks like now
Which is suspect because all of those ECI measurements were behind pace
I looked through their methodology page (https://github.com/spicylemonade/AI-2027-tracker/blob/master/docs/methodology.md) and the only reason I can find is that AECI fits METR better, and that “The three ECI variants are too close to separate on 12 tests, so the chart keeps the simplest one.” Yet they say that ECI and AECI cannot be compared, so why keep Astra and get rid of all the other? They kept in multiple different models for Opus and Mythos, so why remove GPT 5.4 Pro, Sol and Opus’s ECI? Also shows that the website had AI slop assistance via Claude
ECI is supposed to be some sort of aggregate benchmark measurement, and AECI the same except the A is for Anthropic?
Yes.
ECI is the benchmark measurement from Epoch AI, and AECI is benchmark measurement from Anthropic