(complete with a reference to “EV” (expected value))

In this day and age, if I see someone talking about major life decisions in terms of +/-EV, I turn on my heel and walk away rapidly. It was kind of cute when this jargon was entering wide use during the online poker bubble, but that was 20 years ago.

What the actual fuck?

This third category is the Patrick Bateman types who would have mainly been confined to finance at the turn of the century. They probably picked San Francisco over New York in part because they were worried the New Yorkers might make them wear a suit.

There are two specific aspects of Rationalism that cause this mysterious blind spot. First, they believe that superintelligence is inevitable, which conveniently removes responsibility from the people actively trying to bring it about. Second, they treat everyone outside the Rationalist sphere as NPCs who dutifully follow the directions on their script.

I've been thinking more about this lately, and my metamodel of the rationalist world model (if I may get away with such a pretense) is that they subtly believe that the future is already something crystallized, even if they don't consciously realize they hold this belief. It is not really a probabilistic philosophy, no matter how much they assert that. Intelligence, to them, is a function that can more and more precisely apprehend the future as it escalates to greater values along the g-scale, and the singularity will manifest an intelligence that can do so with practically absolute precision (hence its prophesied ability to outcompete and replace humans).

When I'm talking to people who haven't encountered this stuff as closely, I half-jokingly describe the reflexive assignment of probability estimates as a devotional practice akin to prayer or meditation... but it really is! It's a subtle invocation of the AGI/ASI god, in that I make this imprecise estimate now, to demonstrate my belief in a precisely quantifiable future; but come the singularity, God will relieve my feeble human brain from having to commit this lamentable imprecision. In that reckoning, those who are not even willing to venture an imprecise esimate must have no intelligence worth noting.