64
AI test that worked on a scam bot, or someone who doesn't speak English well: In the novel Blindsight, they're discussing whether an alien they've encountered is sentient, so the linguist sends this message:
"Our cousins lie about the family tree, with nieces and nephews and Neanderthals. We do not like annoying cousins."
"We'd like to know about this tree."
Sascha muted the channel and gave us a look that said Could it be any more obvious? "It couldn't have parsed that. There were three linguistic ambiguities in there. It just ignored them."
The scammer ignored this and ChatGPT whiffed it. Claude gave a similar response in that it tried to explain a sentence that can't possibly be explained. Neither caught the ambiguities. Happy hunting!
I feel like another good tactic, if you know multiple languages, is to switch between them. If they don't get confused and/or as to why you switched, and/or just switch with zero hesitation, that could be a potential sign.
Probably works best if you know like 3 languages and at least one is something completely detached from the other 2. I mean, what are the odds you run into someone else who knows, for this completely hypothetical scenario, English, German, and Tagalog? Or another 3rd language far removed from the other 2?
Edit:
I feel like this would work best on a first "meeting" kind of deal. You get a DM or SMS or call from someone and they don't even acknowledge the conversation has eventually shifted languages. You haven't given them any reason to believe you speak multiple languages nor any indication you speak a far removed language. But they don't acknowledge the shift and keep the conversation going at all costs. That kinda thing.
My tired arse interpreted first meeting literally and for a moment I shuddered at the idea of having to meet LLMs in person.
Now that is a truly terrifying thought!