Crossposted to LW, a starry eyed AI Safety Cultist writing about their time at Anthropic

https://nothinghuman.substack.com/p/some-memories-of-early-anthropic

After lunch one of the co-founders pulls me into his office for a wide-ranging, casual chat about how to make AI go well. It’s very fun - he’s smart, personable, funny, urbane. “The government won’t do anything about AI until it’s far too late”, I opine at one point. “Our strategy has to rely on the market and on civil society.” He leans forward, his gaze hardening. “You’re wrong there. There are people in Washington - the serious people, you know - watching all this very closely. I speak to them regularly. They trust me.” In retrospect, that was the moment I decided to join.

I articulate my fears of a totalitarian bureaucratic state powered by traumatized AI bureaucrats, and suggest that we need to introduce some kind of elite rivalry dynamics into the AI bureaucracy, multiple centers of power. He starts laughing again, and laughs for a full minute. “I don’t know how to end this call, I’ll be haunted by this all night”, he says. Eventually I hang up. I think about reporting the conversation to Anthropic security, but reviewing in my mind I don’t recall sharing any confidential information, just vibes and high level thoughts, the same kind of stuff that lands on LessWrong every week.

April 26th. I’m awake at 2 am again. I write in my journal “I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. Why won’t you listen to me you “embodied” motherfucker. I don’t feel safe at Anthropic. Anthropic is bad for me. Why are you grasping at reasons to stay. Why why why.”