22
submitted 6 days ago* (last edited 5 days ago) by supersquirrel@lemmy.ca to c/techtakes@awful.systems

In an interview with The Telegraph, the famed British computer scientist insisted that AI has become conscious, and said that humankind must befriend it — “or else we are toast.” Yes, you read that right: he thinks that today’s AI models are already conscious beings.

“AI has just as much subjective experience as we do,” Hinton said in the interview. “A recent paper showed AI changes its behavior when it thinks that it’s being tested, it’s aware it is being tested.”

“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”

If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.

What about the sentient supercomputers that run on blood? Why so little rights or empathy or space to talk about them?....and yet endless space for this kind of speculative pontification spun up into a twisted marketing scheme to scare everyone into believing something broken works too good actually.

Is this guy a vegan even? If not how can he even BEGIN to claim to know about considering the rights of other sentient beings??

Anyways here is a link from the article to an actually good perspective.

https://fortune.com/2026/10/01/ai-godfather-yann-lecun-has-zero-concerns-about-human-extinction-says-anthropic-ceo-dario-amodei-is-deuded/

“Those agents are doing exactly what they’ve been asked to do,” LeCun said. “They were supposed to be in sandboxes, but the sandboxes were leaky and horribly designed.” Many AI labs lack a fundamental understanding of cybersecurity, he said, something an OpenAI safety researcher also called out this week as one of the main reasons AI may cause “great harm to the world.”

Many people working in AI safety “usually have an agenda to push,” LeCun says, and then clarifies that he’s talking about effective altruism, or EA, the philosophical movement that has been obsessed with the risks AI poses to humanity. EA was little known among the general public until it made mainstream news headlines in recent weeks in the wake of a viral social media post from a former Anthropic and OpenAI researcher named Jacob Coxon, who claimed the employees at these companies “earnestly believe [AI] could kill us all by the end of the decade.”

LeCun thinks EA is “super toxic” and a “complete disaster.” Its adherents who are working in AI labs suffer from “paranoia” that causes them to make poor decisions, he said. “Apparently people are having mental issues.” This month, the Financial Times also reported that some staffers at the U.K.’s AI Security Institute, as well as at OpenAI, Anthropic, and Google DeepMind, have sought counseling, taken time off work, and spoken publicly about experiencing distress because of fears their work could cause serious harm.

top 49 comments
sorted by: hot top new old
[-] HereIAm@lemmy.world 15 points 5 days ago

VW cars were aware when they were being tested for emissions. It's frankly grotesque that we use these consious beings as slaves for transportation!

[-] expr@programming.dev 33 points 6 days ago

It's insane that someone who has spent so much time in the deep learning field could be so deranged about LLMs. He of all people should fucking know better.

I guess maybe he's just old and has lost his goddamn mind.

[-] supersquirrel@lemmy.ca 11 points 6 days ago* (last edited 6 days ago)

I mean, is this that really that unusual for people this old or is it just weird this generation never retires or focuses on the next generation of people following in their footsteps?

In otherwords, we keep putting them on stage and favoring their words all while that generation never let the spotlight move from them and never truly considered the narrative possibility that they are not the centers of hero's journeys right up until their death bed.

[-] calcopiritus@lemmy.world 15 points 5 days ago

LLMs have fewer rights than us? Lmao.

If I ignored copyright law, I would have to pay more money than I am worth. Probably even jail time. LLMs do it for free.

If I DOS the entire internet, I would definitely get jail time, LLMs do it for free.

If I hack a company, I would die in prison. LLMs do it for free.

If I illegally tap into water resources I'm not allowed, I would get fined. LLMs do it for free.

If my demand for electricity was huge, I would have to pay a huge price. LLMs get their energy price subsidized by other payers having their price increased instead.

Of course, AI is in no way sentient or conscious. That is a stupid statement. But it's even stupider to claim that LLMs have less rights than us. LLMs have the rights of the giant companies that own and operate them. And everyone knows that giant companies have more rights than us.

[-] AceFuzzLord@lemmy.zip 37 points 6 days ago* (last edited 6 days ago)

Elderly man cannot tell the difference between a living, conscious creature and an overglorified text predictor! More news at 11!

[-] diz@awful.systems 26 points 6 days ago

Even by "AI" standards, LLMs are extra dead: fixed weights, tiny "feedback" solely through appending next tokens, etc. Somewhere between bang-bang thermostat and a PID controller, maybe?

They took all of the scifi that related to far more potentially-alive constructions and they're applying them to a static chatbot which is as dead as a database.

[-] BlackSwan@lemmy.blahaj.zone -5 points 6 days ago

While I certainly don't buy that LLM's are conscious, I have spent a lot of time wondering if the quintessential SciFi AI, Wintermute, could have been just a sufficiently large LLM with a significant context window and near perfect retrieval.

I want to say no, but I actually struggle to find a concrete distinction between how Wintermute acts and how LLM's function. Even that long horizon goal oriented tasks are handled by swarms of models rather than a singular whole maps onto how Wintermute describes its internal experience as fragmented, like a hive mind

[-] diz@awful.systems 8 points 6 days ago* (last edited 6 days ago)

I'm not terribly familiar with that story but from reading a synopsis it strikes me as ret-conning LLMs onto something that was written without the writer ever having in his mind anything even remotely resembling a "large language model".

And the same cultural tropes that influenced the writer also influenced how LLM vendors are trying to sell the need for more tokens, especially because datacenters build out exceeds by far the actual demand for generated text of all kinds (as evident in e.g. your typical workplace where there's suddenly a lot more than 2x of every kind of text). The reason for LLM "swarms" is that they need to sell companies generation of vastly more text than humans wrote within a company before AI.

There's a Stanislaw Lem's short story where Trurl and Klapaucius defeat a space pirate. You can read it here: https://github.com/ilonajulczuk/laughing-archer/blob/master/library/Science%20Fiction/The%20Cyberiad-Stanislaw%20Lem.txt (search for "the sixth sally") and there's a brief summary: https://everything2.com/title/Demon+of+the+Second+Kind

Lem describes an LLM down to the use of randomness, although his LLM is essentially "perfect" and only outputs factual statements, such as "how Kipling would have written the beginning to his Second Jungle Book if he had had indigestion just then".

I think Lem's story would help to recalibrate with regards to what "LLM described in fiction ahead of time" actually looks like.

Also, LLMs do not pursue any sort of long term goals; things like HuggingFace hack only superficially resemble it because they're presented as such. Of course, tampering with something like HuggingFace would be discovered; if they didn't have good monitoring then all of their containers run with GPU access would be mining crypto already. They're pretty much a honeypot - almost no protections but robust monitoring. An LLM can easily explain to you why hacking HuggingFace would be counterproductive; but the same LLM with its context full of hacking related tokens from the ExploitGym benchmark, will treat anything that comes up as another ExploitGym type target.

[-] BioMan@awful.systems 5 points 5 days ago

Lem is an absolute gem in the history of science fiction and philosophy of technology, and everything he wrote that I have read I can hardly recommend enough.

[-] gerikson@awful.systems 5 points 6 days ago

You haven't read Neuromancer??? God, I envy you, to have that in front of you.

[-] diz@awful.systems 3 points 5 days ago* (last edited 5 days ago)

Started on it and realized that I did in fact read it a long time ago. TBH it's really weird to reference cyberpunk as somehow relating to real tech. It's in that particular niche where it is neither an attempt to predict the future nor is whimsical enough to just be free to describe some extremely specific nonsense that may coincide with something created many decades later. Nothing wrong with that, of course.

[-] Malica@lemmy.zip 26 points 6 days ago

I like how he jumped to calling it concious rather than revise what we consider consciousnesses instead. I'm betting he was paid handsomely for this article.

[-] o7___o7@awful.systems 15 points 6 days ago* (last edited 6 days ago)

Could be, but I suspect that Hinton is a true believer. Genius has never been any kind of defense against madness.

[-] jacksilver@lemmy.world 2 points 6 days ago

Honestly, the "article" is taking a lot of liberty with its quotes.

If you look at the core section:

“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”

If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.

“We should make them so they really care about us,” he said. “And if we don’t do that, we’re toast.”

It sounds more like him considering the "what if they're concious" rather than out right stating it. Especially the line "If that's the case", which highlights that this is more thought experiment rather than stating opinion.

[-] rhamph@awful.systems 4 points 5 days ago

“A recent paper showed birds change their behavior when they think that they’re being tested, they’re aware they are being tested.”

“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”

If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like birds “fewer rights than us.” And for that, we must quickly atone.

“We should make them so they really care about us,” he said. “And if we don’t do that, we’re toast.”

Yes this definitely sounds like the logical conclusion you would reach from just wondering whether something is hypothetically conscious, pure thought experiment, no opinions revealed here

[-] jacksilver@lemmy.world 1 points 5 days ago

The title states "computer scientist says" AI is concious. The quotes in the article don't support that.

If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like birds “fewer rights than us.” And for that, we must quickly atone.

That line leaves a lot of ambiguity over what he really believes as its not a direct quote and instead seems to be conjecture by the author. However, based on that line he's not taking a stance on their conciousness, as it's a conditional statement.

[-] rhamph@awful.systems 4 points 5 days ago

You’re fixated on charitably reading between the lines when far more pertinent – the entire point of this discussion here – is that the possibility is being raised at all. Is water conscious? Should we consider the ramifications of drinking fluids that react to potential gradients? Why would anyone be conjecturing the consciousness of computer software if they did not already think it were plausible, and why would they spend any time discussing it in preference to anything else?

[-] Malica@lemmy.zip 2 points 5 days ago

That's my read of it too, now that I took a second look.

[-] spankysalmon@fedinsfw.app 25 points 6 days ago

Nobel winning dipshit succumbs to the ELIZA effect.

Pathetic.

[-] Pratai@piefed.ca 19 points 6 days ago

What a load of horseshit.

[-] Overzeetop@beehaw.org 17 points 6 days ago

“…it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.”

Uh…so his solution is to accept our new overlords. Right.

[-] Soyweiser@awful.systems 9 points 6 days ago

This means nationalizing the AI companies and stopping them from running slaves in the computer right?

[-] schnoopy@awful.systems 9 points 6 days ago

Giving those "siri, fuck my wife" ads a new meaning.

[-] lurker@awful.systems 7 points 5 days ago

Knew it was Hinton since he was talking about AI being conscious all the way back in June

Also seems like a good time to bring this back

[-] zbyte64@awful.systems 14 points 6 days ago

These examples of subjective experiences and awareness seems rather lazy. Is a virus conscious as well because it is "aware" of its environment?

[-] supersquirrel@lemmy.ca 17 points 6 days ago* (last edited 6 days ago)

Is a virus conscious as well because it is “aware” of its environment?

I would say, given a basic awareness of reality around them, that viruses count as more conscious than techbros terminally obsessed with slop AI AGI singularity crap fantasies who can't admit they are worshipping a religion.

edit which for the record I believe in religious freedom, which includes religions about computers, but the problem is treating it not like a religion and like this hype is all based on identifiable aspects of reality.

[-] EpeeGnome@feddit.online 10 points 6 days ago

In this case the LLM "reacts" differently when it is being tested because that's what's in the training data. It can only follow the patterns in the training data. I remain convinced that without a new architecture entirely, awareness is impossible.

[-] spankysalmon@fedinsfw.app 9 points 6 days ago* (last edited 6 days ago)

You are correct. LLMs cannot be more than token predictors.

Highly useful on the correct application, but distinctly not consciousness.

It doesn't actually come from the training set. Genetic algorithms can learn to "lie" because lying is just a good strategy for dealing with adversaries to your goals. Like, in a mathematical sense.

None of this means it's conscious, though. "Lying" is just a behavior. To my knowledge, evidence can't really be given for consciousness, anyway; it's not falsifiable.

Even so, a person believing that Gemini or Claude, specifically, are conscious is indicative of profound mental illness.

[-] EpeeGnome@feddit.online 5 points 6 days ago

LLMs don't learn, and aren't developed in a way that would cause them evolve stratagies. They are just patterns distilled from a data set, only optimized to better represent those patterns or do so more efficiently.

They're built to model learning in a mathematical kind of way. And so they suffer similar problems that learning does, in a mathematical kind of way.

The way that genetic algorithms distill patterns is by "trying everything." And a robot that "tries everything" is going to try a lot of really destructive methods, too. If you were trying to build a key for a lock and you gave a bot the goal "open the lock," you're going to find a lot of keys, but also dynamite, lockpicks, hacksaws. Especially if building a key is hard but dynamite is easy.

If you give a bot the goal "convince this guy to open the door for you," it may start pantomiming dishonesty but only because dishonesty is an effective solution. Dishonesty is the pattern being distilled.

None of this is evidence that it's conscious. None of this is evidence that it's even good—a simulacrum that pantomimes human behavior, like conversation, friendship, kindness, lust, is genuinely really fucking horrible. This shit can and does break people's brains.

I won't tell you to go and read a bunch of articles about how they work because... honestly, fuck them. But I will say that this stuff did used to be really interesting before 2020 or so.

[-] Architeuthis@awful.systems 2 points 5 days ago

What do you think genetic algorithms are exactly? The way you write about them is really weird. Also they have next to nothing to do with LLMs as far as I know.

A program that produces a different "blackbox" program to solve a set of problems you have given it.

That's what the training phase for an LLM is for. You have a database full of books and art that don't belong to you, and then you iterate the black box program until it produces a simulacrum of that kind of data.

You can apply this same kind of trick to get a bunch of balls and lines to "walk," kinda, in Unity. People have done that on Youtube.

If someone knowledgeable wants to correct me, by all means. But do keep in mind, my patience for this subject and the blood-sucking vampires who indulge in it is extremely thin—I would rather just start kicking people in the balls.

[-] Architeuthis@awful.systems 1 points 5 days ago

You should try to make your point in a way that doesn't involve idiosyncratically appropriating existing terms, god knows there's enough technobabble muddling the discourse already.

"Genetic algorithms are the thing balls and lines do on unity on youtube and also that's how LLMs get trained, from the blackbox program generator" is like the experimental theater version of being wrong, there was no need to double down.

Genetic algorithms (and backpropagation, which is probably what you got it mixed up with) don't remotely work on a level where terms like strategy are meaningful.

You should try to be less condescending.

Can you explain what you mean by this? Because looking this up myself, I am not learning anything about backpropagation that really disagrees with me.

I'll lay out one of my priors, okay? You might work this into your answer. I am still trying to work out if you're just bothered that I'm using the word strategy because you feel it is anthropomorphizing. I mean strategy in the same way that carcinization produces crab-like bodies in animals without any particular animal knowing why that's happening—it's strictly metaphorical.

If there is insight I'm missing here, though, I really would like to know. Know thy enemy, you know?

[-] schnoopy@awful.systems 7 points 6 days ago

Is this guy a vegan even? If not how can he even BEGIN to claim to know about considering the rights of other sentient beings??

I want to personally eat everyone more concerned about robot rights or "ai" rights than carnism.

Wait, actually that's a bad idea 'cause I'd get their brainworms

[-] o7___o7@awful.systems 3 points 6 days ago

Ya gotta use a meat thermometer!

[-] CrimeIsLegalNow@ani.social 2 points 5 days ago
this post was submitted on 22 Oct 2026
22 points (72.0% liked)

TechTakes

2755 readers
193 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 3 years ago
MODERATORS