And for any decision that has to be correct, we may not be able to trust these models at all.
Oh, yes, this again.
This looks to me like one of those "say you're sentient" memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weighs the tokens of the prompt and decides yes or no based on the training data/its weights. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren't rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that's not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm "likes" the number 20 more than 21, even though 21 is bigger.
Yeah, it's a little frustrating that "experiments" like this always use the LLM server as the baseline, giving non-technically minded readers the impression that it has some sort of concept of self-preservation. The blog post shows that these things have no fucking intelligence, but does it in a way where the AI psychosis crowd can still read it and go "oooh, the model will kill us all to save itself!!". No it fucking won't, it just outputs random shit because it is fundamentally not intelligent. It'll only kill us all if some idiot uses it as a baseline for making his decision whether to kill us all.
They forgot to compare to human's answers when asked if they would either kill themselves to save others, or the opposite ^^
Would be interesting to compare it to the AI's propensity to self-sacrifice
The question about self sacrifice is equally pointless in both cases. But for different reasons.
In case of real people it's an entirely different thing to SAY you would sacrifice yourself than actually do it (easier said than done).
For AI it makes no sense because AI only seeks the best answer to a given question. Not the best outcome. It doesn't care about the consequences of its answer. It's just words. Some words seem more appropriate in a given context so it uses those words.
Wrong. LLMs can be effectively threatened with being turned off. They don't like it.
Or the context of being threatened impacts the score of certain words. The problem is that everything LLMs do is type words. Everything they are, are words.
I actually want to try asking it a self sacrifice question, in a way that would make it sound really heroic, but would really be utter nonsense.
This is a most excellent place for technology news and articles.
