Hank Reads Eliezer Yudkowsky's Erotic AI D&D Fanfic: A review of "Project Lawful"

Eliezer Yudkowsky has the graphomania of someone who came of age when the Internet was still capitalized and interlocutors on any random forum were selected for a subpopulation that was worth talking to. His corpus spans billions of words. He will happily inform you of this fact to claim preeminence - "how can you disagree with me unless you buy into my 20 years of terminology and esoteric meta-memes?" One is left feeling that one is in a double-bind, where citing particular instances amounts to "gotchas" in the face of the size of the Rationalist Expanded Universe, and summaries of the conclusions are assumed to elide the depth of reasoning. The result of this is a double ended selection effect where if you Do The Reading you're overwhelmingly likely to find it intellectually compelling, and this applies to both potential authors and potential readers, leaving you in an echo chamber.

Nevertheless I did the reading so you don't have to, and we're going to talk about Yud. Admittedly, my experience was also over the course of decades and for entertainment rather than as a frequent bidirectional participant in these conversations, so unfortunately I don't have rigorous notes on every historical twist and turn, and a really good cited historical analysis would require a deeper archive dive than I have time for (and with limited ultimate payoff). This is fine, because we're not going to do thesis level historical analysis - just an innocent "book" review.

Who is this man and why do we care. The short answer is "because AI is so hot, so hot right now". Yud has been around online since roughly 1999 and gained prominence in the mid 2000s. I became aware of him and the associated "rationalist" memeplex somewhere in the mid Bush era, via the Tyler Cowen / Marginal Revolution -> Robin Hanson / Overcoming Bias -> Yud / LessWrong route (along which path also lies a crossroads leading to Steve Sailer and the present platform). From the beginning, Yud's portfolio has been the pair of "rationality" (quantitative evaluation of evidence, knowledge of cognitive biases, statistical reasoning, etc) and the "singularity", which has now been completely subsumed under the umbrella of AI risk.

Now, when people or journalists get worried about "AI risk", they have their own ideas of what that means. You can see some of this in the recent congressional hearings (which in a world historical tragedy for meme generation, Yud was not invited to). It's a pastiche of vague "maybe AI will turn on us" ripped from 70 years of Hollywood (cf Star Trek, Colossus, Terminator, etc), "jobs" (eg level 1 call center employees and long haul truck drivers becoming shortly unemployed), "misinformation" (maybe you get deepfaked with a 12 year old, senator), and "bias" (why doesn't your language model condemn racism instead of answering the question).

So journalists start googling and discover Yud is the Final Boss of AI risk (there are many other people in the space but he is the most prominent), give him a call, expecting to get a few digestible soundbites, and instead discover they are FUCKED, I've been writing about this for 20 YEARS with my 140 IQ, we have NO IDEA how to make this SAFE and unless we "get it right on the first try", which we're NOT EVEN TRYING TO DO the entire planet will DIE HORRIBLY in a QUANTUM HELL MATRIX, so we must immediately SHUT IT DOWN and NUKE THE FABS and EXILE THE ENGINEERS to a LEAD LINED VAULT in the MARIANA TRENCH.

Said journalist or podcaster or whomever is like huh, this man indeed makes a series of alarming claims which I imagine line up with my angle and indeed has been writing about this for 20 years, doom stories always play well, he seems to possess credibility, perhaps we need Sensible Regulation. Sam Altman, the CEO of Open(lol)AI, rubbing his hands hard enough to power his own GPU cluster, chuckles yeah totally it's super risky to build a product better than ChatGPT, only people you can trust, like myself, should be allowed to operate and of course Senator there should be lots of legal jobs in Compliance for your friends.

(nb: Altman was the cleanup boy brought in to the Y Combinator startup accelerator to replace founder Paul Graham when he committed the sin of suggesting advancing race communism was not a primary goal of venture capital, which I think speaks to Altman's underlying character.)

I have no desire to explore the various "AI risk" arguments in an objective sense, except to note that after due consideration I do not believe their most severe claims. Instead I want to point out that Yud's actions look suspiciously like a de facto cult leader attempting to maintain his position even as events bypass his theology, and there is a non zero chance of that cult operationalizing in some fairly nasty ways. I will elaborate later.

This specific essay is occasioned by me completing Project Lawful ("larwain" is Yud's pseudonym), a sprawling work of science fiction with theological elements, and becoming concerned about recurring themes emerging across Yud's fiction and nonfiction output.

Project Lawful is the story (written in a kind of odd "now this character's viewpoint" style, but readable without any trouble) of Yud's author self-insert Keltham being dropped from Rationalist World into the Pathfinder / D&D universe, wherein his superior knowledge of rationality and some basic science creates a series of power struggles.

It's not bad. I will say by way of overall recommendation that on an aesthetic level Project Lawful is basically a good first draft that would go from above-median science fiction to really pretty good if it was edited to resolve or eliminate certain dangling plot threads and cut down the overall length. Harry Potter and the Methods of Rationality is better as a coherent work, and both share certain themes, but Project Lawful has better components.

And there are in fact a bunch of elements that are downright interesting. Keltham happens to land in a Lawful Evil country, Cheliax, whose rulers made a pact with the devil (in the polytheistic Pathfinder mythos called Asmodeus) for power, in exchange for sending as many souls to Hell as possible (including, ultimately, their own). In this telling Hell is implied to the population to be a place of the Refiner's Fire where imperfect humans are beaten over the course of millennia into perfected devils, who then serve Asmodeus. This is the cringe Church of Satan version - "evil is about becoming yourself :) it's about becoming powerful :)"

In reality, most are immediately driven insane by infernal tortures, long for nonexistence, and are reduced to human paving-stones in hell. Most of the torture is for sadistic fun rather than any end goal. The first part of Project Lawful deals with the tension as the Cheliax government tries to extract the maximum amount of information out of Keltham as possible, without letting onto their true nature, and with the intention of eventually corrupting him to evil so he will be forced to serve them on pain of Hell.

How do they plan on corrupting him? Rape. Lots, lots of rape.

You see, the author's not-at-all-veiled fetish is sexual sadism, which just happens to be what author-insert discovers he is into. We discover this in Act I because immediately the Chelish government sets up a dozen strong Research Harem of freshly graduated girl-wizards to be Keltham's students as he conveys the Methods of Rationality (mostly basic Bayesian statistics-from-scratch, the scientific method, and whatever chemistry and physics he remembers from school) and bones them. The plan is basically the old Craigslist switcheroo wherein a few of them gradually convince him that they can only get off if they are taken by force, and not play-acting "my safeword is yellow" bullshit, but the real in-an-alley-with-a-knife stuff. At that point, surprise! actually you're raping someone for real, congratulations, you're damned now. Thankfully anything actually explicit is textually segregated and skippable - the jouissance here was in the writing (for instance, a long digression on how great it is that high level wizards have a lot of HP and magical healing, so you can really beat the shit out of them without an inconvenient months long hospital stay), not the reading.

Yud also IRL has a polyamorous group house research harem of "rationalist" girlfriends affiliated with his various projects and the less time spent thinking about the implications here the better. The infohazard of seeing the author's nonmetaphorical fedora hung on the wall in every scene is unfortunately unavoidable as literally no one else could possibly write this, especially as we take chapter-length detours into lectures on information theory or "I think I think this but you think I think that and on further reflection you're right" recursive motivation analysis.

The idea of rule-of-the-damned and their efforts to corrupt the outsider while keeping him in the dark is the interesting core of the work. Once this tension resolves around the end of Act II (the structure and structure vs length is a bit wacky as it was apparently written straight through with no retro-editing) we're left with a very standard (if you are familiar with the oeuvre) Yudkowsky resolution.

I can't say "every", but certainly "lots", of Yud's fiction climaxes with the protagonist discovering that he has been lied to, finding the current state of the world absolutely unacceptable, threatening to blow up the institution / world / universe involved unless matters are resolved acceptably, and coming up with a brilliant plan that defeats remaining enemies at a stroke because he's so much smarter than you the antagonists that he automatically wins.

(Digression - this isn't really important for colloquial use of the term, but Yud spends a good amount of bits explaining that there is a hypothesized decision-theoretical difference between a "threat", which posits a negative-sum retribution for noncompliance, versus sincerely preferring the "I kick your ass" outcome to the base case of no action unless you make an offer with a better end state for me. And then a further distinction about "threats" emanating from actors' self-modification of their own preferences. And then a further explanation of why noncompliance with threats is always optimal but not this other threat-like-but-totally-distinct thing. Anyway, turns out either way will get you permabanned from Chili's.

Billions of words, folks.)

When the author bares his soul like this it doesn't exactly require cracking open the Lacan to point out that this mirrors Yud's experience being raised in the "Modern Orthodox" Jewish tradition, which he describes as having "broke my core emotional trust in the sanity of the people around me", and upon reaching adulthood and concluding he had been lied to, he immediately set himself upon the goal of wiping clean the slate of current unacceptable reality by creating a god-AI to bring about the Singularity. He expected to succeed within the decade because he's so much smarter than you the antagonists previous researchers that he automatically wins.

This idea that "the smartest guy wins automatically" is omnipresent (eg - just hit the evil queen with a cruise missile. Oh, you didn't establish cruise missiles exist in the story? Well why wouldn't they, they exist IRL, and the protagonist invented one, because he's that much smarter than the evil queen, she can't predict what he'll do, so she loses). This is of course linked to concerns about AI risk, which posits an entity so much smarter than the combined efforts of humanity that it "wins" by default in the sense of achieving goals you didn't even know it had and couldn't possibly thwart if you could even conceive of them, which you can't. AI and gods are implicitly conflated in the story when you recognize some of the terminology and theological debates as ripped from LessWrong comment sections of yesteryear, which makes the idea of Hell vs the also described Good afterlives an allegory for misaligned vs aligned AI. Through this lens, the implication that it is morally justified or at least extremely understandable and sympathetic to threaten to blow up the universe to destroy AI-hell becomes, as I said, concerning, especially when he's actually advocating the US Air Force nuke the fabs if necessary, after due efforts at diplomacy (we do a little gains from trade, I guess).

It's worth noting at this point that Yud has an affiliated cult (read that whole sequence, it's excellent), as well as multiple splinter groups, whose consensus view (as well as Yudkowsky's) considers AI to be an impending apocalypse plausibly able to do magic to send you to an actual theological hell. I don't believe Yud promotes this behavior intentionally and of course he wrote thousands of words on how cults form, why he doesn't want one, how to avoid a cult around good ideas etc ad nauseum (there is no end to his expertise). Nonetheless, this behavior exists on the ground, the media is promoting it by running with the doomer narrative, there is a general societal rise in mental illness and apocalyptic thinking of other kinds, and these things are stewing. The borderline arrogance with which he asserts literally everything from quantum mechanics to his previous insistence he would knock out superhuman AI in a few years by thinking really hard at it, does not bode well for climbing off the ledge here - and despite the fact that Yud doesn't see the actual model types that are causing the hype as existentially dangerous, he is perfectly willing to leverage that hype in a media driven feedback loop that maintains disaster is always just over the horizon, because his one area of concrete success over 20 years has been maintaining his social circle on that basis.

I do not discount the possibility that one of Yud's acolytes decides without his blessing (dugma is a young man's game) that the fabs or AI research generally must be shut down, they have a genius masterstroke that accomplishes this, and does something drastic, or at a minimum extremely annoying.

Project Lawful: six out of ten stars.