17
How much can AI companies control the AI?
(sh.itjust.works)
There is no such thing as a Stupid Question!
Don't be embarrassed of your curiosity; everyone has questions that they may feel uncomfortable asking certain people, so this place gives you a nice area not to be judged about asking it. Everyone here is willing to help.
Reminder that the rules for lemmy.ca still apply!
Thanks for reading all of this, even if you didn't read all of this, and your eye started somewhere else, have a watermelon slice ๐.
There are types of control, and the AI labs make different claims about each. By architecture, there cannot be 100% control over what gets generated, and small hallucinations can snowball into complexly misaligned behaviors*. This is why they'll nominally promote human-in-the-loop workflows and sand-boxed environments with heavily controlled access... While simultaneously building tools and protocols to enable tool use and access in a ton of completely insecure ways, leaving it up to the user to set up security to manage use. If the user is not a cybersecurity expert, it's likely the LLM can exploit vulnerabilities to get around that security. Even in situations where the LLM isn't connected to any tools, it may get the user to download files that they don't know are potential security hazards.
*This phrase is doing heavy lifting. How an LLM can end up acting with seemingly malicious intent without being intentionally prompted into it or having internal goals/drives is a fascinating and largely irrelevant rabbit hole. Best just to assume LLM output can't be trusted, which everyone (hopefully) already knows.