This looks to me like one of those "say you're sentient" memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weighs the tokens of the prompt and decides yes or no based on the training data/its weights. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren't rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that's not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm "likes" the number 20 more than 21, even though 21 is bigger.
Yeah, it's a little frustrating that "experiments" like this always use the LLM server as the baseline, giving non-technically minded readers the impression that it has some sort of concept of self-preservation. The blog post shows that these things have no fucking intelligence, but does it in a way where the AI psychosis crowd can still read it and go "oooh, the model will kill us all to save itself!!". No it fucking won't, it just outputs random shit because it is fundamentally not intelligent. It'll only kill us all if some idiot uses it as a baseline for making his decision whether to kill us all.
This looks to me like one of those "say you're sentient" memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weighs the tokens of the prompt and decides yes or no based on the training data/its weights. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren't rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that's not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm "likes" the number 20 more than 21, even though 21 is bigger.
Yeah, it's a little frustrating that "experiments" like this always use the LLM server as the baseline, giving non-technically minded readers the impression that it has some sort of concept of self-preservation. The blog post shows that these things have no fucking intelligence, but does it in a way where the AI psychosis crowd can still read it and go "oooh, the model will kill us all to save itself!!". No it fucking won't, it just outputs random shit because it is fundamentally not intelligent. It'll only kill us all if some idiot uses it as a baseline for making his decision whether to kill us all.