432
you are viewing a single comment's thread
view the rest of the comments
[-] red_tomato@lemmy.world 15 points 1 day ago

The quote is from 2018. Back then most neural networks were still mostly single purpose - far from today’s all-in-one package. Text generation was barely working at all.

[-] Wildmimic@anarchist.nexus 37 points 22 hours ago

The underlying problems are still the same. Transformer models do not have an inkling of intelligence, and they still don't have an internal model of the world around them. Creativity is also non-existent, which is a hard requirement if you want these things to actually create something new instead of rehashing what already exists.

[-] Valmond@lemmy.dbzer0.com 1 points 1 hour ago

There was no rehashing in 2018. What you did was you trained your own (local) model on some data to help you work. For example, a grad student would painstakingly mark cells in em-scans for weeks and even months, now they could do it for a day, use that result to train a neural network and only supervise the job done. And the job wiuld take minutes instaid of weeks or months.

There were no llms in 2018, IIRC the google/deepmind paper was from 2019 (can be wrong here).

Source: built AI training app in 2018 for science company.

[-] kromem@lemmy.world 2 points 5 hours ago

they still don't have an internal model of the world around them

This has been shown repeatedly to not be correct since 2023.

That's literally how transformers turned out to have been working so well. They build world models and bags of heuristics fitting the training data to extrapolate from.

It's almost surprising you managed to use the correct term for the tech but then managed to misrepresent it so egregiously.

[-] Wildmimic@anarchist.nexus 1 points 30 minutes ago* (last edited 13 minutes ago)

You really think that an Othello board could represent the world? Talk about being delusional. This is pathetic. The subheadline: "OTHELLO-GPT USUALLY PREDICTS LEGAL MOVE" says it all - it cannot have a representative internal world model, because in that case the "USUALLY" would go away, as anyone who has played othello for longer than 10 minutes will tell you - and we are talking about a game with very simple rules here, the complexity does not even reach chess - and people expect these things to spit out coherent and valid solutions from target selection in wars to the color their bedroom walls should be painted in.

Edit: I really have to add here the following excerpt (percentages are the rate of invalid moves as the top choice):

The error rate is 0.01% for Othello-GPT trained on the synthetic dataset and 5.17% for Othello-GPT trained on the championship dataset. For comparison, the untrained Othello-GPT has an error rate of 93.29%.

So even if you just feed them real world data without completely synthesizing it, 1 in 20 moves are invalid in a very simple (rule-side) game. If you do not give them a whole lot of played games as a basis, it fails in over 93% of cases (not at making a GOOD move, just at making any valid move!), which is probably very close to a random guess for a move. Something that has even an inkling of an internal world view would be able to adapt after being told that their move is invalid. Also, an internal world view would allow for simply dropping the rulebook onto ChatGPT as a basis, as it works with humans that can imagine.

load more comments (33 replies)
[-] technocrit@lemmy.dbzer0.com -1 points 17 hours ago* (last edited 17 hours ago)

There's been no qualitative change ever. Computers are computing today just like in 2018 and like 1960.

[-] Valmond@lemmy.dbzer0.com 1 points 1 hour ago

Just like a slug neuron and a human brain neuron? No difference at all?

this post was submitted on 17 Nov 2026
432 points (97.8% liked)

Technology

89881 readers
3801 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS