QAnon is a recurring concept in the Astral Codex Ten archive, appearing 14 times across 14 issues between January 21, 2021 and November 06, 2025. The archive places it in contexts such as "Speculating on the identities of people like ... QAnon"; "makes me a little more concerned about things like QAnon than I had been previously"; "QAnon-ers try to explain who they are". It most often appears alongside Trump, China, facebook.
- Article page
- QAnon
- Mention count
- 14
- Issue count
- 14
- First seen
- January 21, 2021
- Last seen
- November 06, 2025
Outbound links
- http://web.archive.org/web/20221104130431/https://stevekirsch.substack.com/p/1m-bet-rules
- http://web.archive.org/web/20221129133112/https://blog.rootclaim.com/rootclaim-accepts-500000-challenge-on-covid-vaccine-safety-efficacy/
- http://web.archive.org/web/20221224061743/https://www.skirsch.com/covid/SaarWilf.pdf
- https://archive.ph/pY4gF#selection-663.103-683.190
- https://astralcodexten.substack.com/p/a-cyclic-theory-of-subcultures/comment/8294319
- https://mobile.twitter.com/OficialPatriot/status/1296527185143123968
- https://mssv.net/2020/08/02/what-args-can-teach-us-about-qanon/
- https://slatestarcodex.com/2019/06/10/book-review-the-prices-are-too-dmn-high/
- https://web.archive.org/web/20230104080248/https://www.rootclaim.com/
- https://www.astralcodexten.com/p/details-that-you-should-include-in/comment/61387082
- https://www.endocrine-abstracts.org/ea/0090/ea0090p69
- https://www.nytimes.com/2020/12/27/business/media/heather-cox-richardson-substack-boston-college.html
Issue trail
- Still Alive
- Book Review: The New Sultan
- Links For April
- Mantic Monday: Grading My Trump Predictions
- Links For August
- Links For September
- Epistemic Minor Leagues
- Will Nonbelievers Really Believe Anything?
- Universe-Hopping Through Substack
- Your Book Review: Autobiography Of Yukichi Fukuzawa
- Your Book Review: Real Raw News
- Links For December 2024
- In Search Of AI Psychosis
- The Bloomer's Paradox
Related pages
Source context
...but I was also grateful to get some emails from journalists trying to help me understand the perspective of their field. They point out that reporting is fundamentally about revealing information that wasn't previously public, and hard-hitting reporting necessarily involves disclosing things about subjects that they would rather you not know. Speculating on the identities of people like Deep Throat, or Satoshi Nakamoto, or QAnon, or that guy who wrote Primary Colors, is a long-standing journalistic tradition, one I had never before thought to question. Many of my correspondents brought up that some important people read my blog (Paul Graham was the most cited name). Isn't there a point past which you stop being that-guy-with-a-Tumblr-account who it's wrong to dox, and you become more like Satoshi Nakamoto where trying to dox you is a sort of national sport? Wouldn't it be fair to say I had passed that point?
Second, if there was a single moment went things obviously took a turn for the worse in Turkey, it was the Ergenekon trials - Erdogan's attempt to forge evidence of an anti-Turkey conspiracy involving all of his enemies. This makes me a little more concerned about things like QAnon than I had been previously - if Trump had arrested various prominent Democrats for their role in a Deep State pedophile ring, that would be pretty similar to the tactic Erdogan used to seize ultimate power. On the other hand, the thing where Democrats talk about how Trump supporters entering the Capitol was an “attempted coup” and we need lots of “domestic terror laws” and a grand attempt to uncover the complicity of the mainstream Republican establishment and bring them to justice - that also feels a little too Erdoganesqe for comfort. Having ideas about the Deep State and attempted coups floating around, sounding vaguely credible, was a major factor in Erdogan's success. The more skeptical we can be of that sort of thing, the better.
23: Pew feature on political mutual understanding - just 2% of Biden/Trump voters feel like Trump/Biden voters understand their point of view “very well”. What would they say if they had the chance to explain themselves? Pew collects several dozen people’s (very short) answers. Seem to be divided 50/50 between “we wish you understood how we’re all on the same team trying to build a better world” and “we wish you understood how much you suck”. Semi-related: QAnon-ers try to explain who they are and what they want people to understand about them.
My argument was something like: QAnon are, for real, the sort of people that Democrats like to imagine all Republicans are. They're crazy and have no valid concerns. So Democrats had a very strong incentive to exaggerate their role within the Republican party and convince people that QAnon was the driving force behind Trump support. Nobody ever gave an estimate for how many people were in QAnon, and every report on it was consistent with "a tiny handful". There was also a big attempt to make QAnon look violent by everyone giving long lists of "Violent Acts Caused By QAnon", when you examined them closer, literally did not include a single violent act caused by QAnon (it was all either nonviolent acts, plots that never materialized, or criminals who were incidentally QAnon supporters).
Inline links: long lists
Later some people did a terrible mismanaged poll which purported to show that 35% of Republicans were QAnon supporters, and then someone did a better poll which showed 1-2% of Republicans were QAnon supporters. Everyone reported on the former, nobody reported on the latter, and this left me pretty annoyed.
In ~November 2020, I did a survey of Twitter in which I searched the keyword "QAnon", and the most recent 100 results were all Democrats attacking QAnon; no matter how far I scrolled I could not find any actual QAnon supporters. But I don't know if this was because vocal QAnon opponents actually outnumbered QAnoners by that much, or because Twitter is doing a really good job censoring them (is it even possible to censor supporters using a keyword this successfully without opponents getting caught in the crossfire?) Either way, this added to my evidence that this was a pointless moral panic.
17: Is it true that, as GK Chesterton claimed, people who don’t believe in God will believe in anything? IE that Christianity fills a useful religion-shaped-hole in people’s heads, and so non-religious people are easy prey for cults, conspiracy theories, etc? I hear this a lot, but here’s a study finding that church-goers were more likely to believe in QAnon, even after “adjusting for confounders” (remember, this is hard and doesn’t always work). The same article notes that “white evangelicals” are more likely to believe in vaccine-autism connections, moon landing fraud, etc - although I find this less convincing than I would if they just gave me the church attendance statistics without bringing race and denomination into it.
Inline links: here’s a study finding that
2: The Story Of Adrenochrome: QAnon believes that elites are addicted to adrenochrome, a drug synthesized from the glands of tortured children. Where did this theory come from? The short version is “Hunter S Thompson made it up for Fear and Loathing in Las Vegas”. But read the long version for, among other things, explanations for why it shows up in Dune and A Clockwork Orange.
Inline links: The Story Of Adrenochrome
Viral game designer Adrian Hon wrote an article about What Alternate Reality Games Can Teach Us About QAnon.
Inline links: What Alternate Reality Games Can Teach Us About QAnon
It argues that people fall for QAnon because it gives them an interesting mystery. It's a place where new discoveries are always around the corner, where a few hours of research by an amateur like you can fill in one of the missing links between Joe Biden and the Lizard Pope. The thrill of QAnon isn't just learning that all your political opponents are secretly Satanists or Illuminati or whatever. It's the feeling that you have something to contribute to the great project of figuring out the secret structure of the world, and that other people in a shared community of knowledge-seeking will appreciate you for it.
One place you could go from here is to talk about how QAnoners are the sort of people who are excluded from existing systems of knowledge production. They are never going to be Professors of Biology, and they know it. Their only hopes of being taken seriously as an Expert - a position our culture treats as the height of dignity - is to create a complete alternate system of knowledge, ungrounded in any previous system, where they can end up as an expert on the Lizard Papacy.
“White evangelicals” are more likely to believe most measured conspiracy theories, and churchgoers were more likely to believe in QAnon in particular.
There’s an obvious confounder here: the authors are doing the usual trick where they cherry-pick right-wing examples of something bad, show that more right-wingers are in favor of them, then conclude that Science Has Proven Right-Wingers Are Bad. QAnon, illegal votes, and COVID microchips are inherently right wing conspiracy theories; vaccines/autism has probably become right-coded post-COVID. Only the moon landing seems politically neutral, and it’s hard to tell if there’s a real difference on that one. So this just tells us that white evangelical church-goers are further right than other people, which we already know.
I guess this is the QAnon thing (though he has some kind of complicated objection to that terminology). And QAnon is among the most over-covered phenomena of our time, so much so that it’s hard to have a novel or interesting take on it. But I’ll try: I think the right genre for Trump is “outlaw prince” - like Robin Hood, or Song Jiang, or your better class of pirate captain. Realistically he’s just out to enrich himself. But he defeats and embarrasses so many people along the way that he becomes a legend, inextricably tied to the very idea that the establishment can be beaten. He develops a cult following, his relatively meager real accomplishments get exaggerated in song and legend, and everyone assumes that he was only stealing from the rich in order to give to the poor or something. He can’t be caught, he can’t be defeated; like Elvis, he won’t even be able to die. He has ascended to the realm of archetypes. I guess that is a kind of winning a #SoulWAR.
Inline links: complicated objection
A man who dedicated himself to teaching people English may balk at the state of English language education in Japan, where only around 5% of the population are fluent. This arguably has some benefits, insofar as it insulates Japan from some of the silly ideas currently infesting the Anglosphere (though QAnon still seems to have made it through). But the link between English language fluency and global economic competitiveness seems pretty well established.
Inline links: seems
Making the sentence for all crimes identical: Mandatory death penalty9. As far as perfidious methods to deliberately destroy due process and engineer mass executions go, the Law of 22 Prairial is pretty much unmatched in human history. And yet: In the roughly two months of the law’s existence, about one-fifth of defendants were still acquitted! No such good fortune exists in Gitmo. The White Hats’ secret tribunal is a tribunal of blood. In three years of activity, as far as I know exactly one person has escaped conviction: Former Attorney General Jeff Sessions, freed after a direct intervention from Trump. A tiny handful of others have received decades-long prison sentences, but even they tend to meet bad ends. Bill Clinton received a life sentence, only to mysteriously die in prison, perhaps murdered by his daughter Chelsea, who wasn’t really his daughter, but nevertheless soon wound up executed herself. Not only does the rate of death sentences at Gitmo seem to exceed 90 percent, Baxter makes very little effort to portray the proceedings as fair or just. Upon arrest, instead of being read their rights, detainees are informed that they have no rights, and are instead “enemy combatants.” Yet despite being classified as “enemy combatants,” defendants are almost without exception charged with treason. The U.S. Constitution defines treason narrowly as levying war against the United States, or giving aid and comfort to its enemies, and requires at least two witnesses to the same specific act, but in Gitmo the label is invoked with a liberality that would make Robespierre blush. “Traitors” have been arrested and convicted for telling troops not to attend Trump rallies and for ruling against Donald Trump in court. Defense attorneys are denied access to evidence pre-trial, and many defendants get no lawyers at all. Trials work a lot like Phoenix Wright, in that at any point the three-officer panel10 can simply declare they’ve seen enough evidence and pronounce a conviction with death sentence immediately. In the case of former Tom Hanks co-star, this has happened within five minutes. Appeals are non-existent. The actual executions sometimes involve tormenting the condemned with fake escape attempts or pardons: The driver told Whitmer he needed to make a pitstop to grab her “exoneration paperwork.” Then Whitmer saw the clearing and the gallows and Vice Adm. Crandall. And the hangman and a Navy chaplain standing atop the gallows. “You lied to me,” Whitmer bellowed. “Minor error, not a lie,” the driver replied. […] The admiral instructed the hangman to flip the switch, and a second later, Whitmer was swinging from the rope, a guttural gurgling sound escaping her lips. She was officially pronounced dead several minutes later. “Another Covid queen out of the way,” Adm. Crandall said. During the treason trial of Hillary Clinton crucial evidence is provided by former campaign manager John Podesta, who accepts a plea deal for life in prison in return for testifying about Clinton’s child-trafficking activities. But after Clinton had safely been hanged, the military tribunal simply decided to revoke Podesta’s plea deal because, well, they felt like it. “Even though he’s not prosecuting Podesta’s case, Vice Adm. John G. Hannink made the decision to renege on the deal. He’s the one who offered it. The severity of Podesta’s crimes matched Clinton’s—a lot of stuff they did in tandem, together. When you think about it, there’s really no reason why he should get special treatment. He’s a sodomist [sic.]11. Before breaking the deal, he called Trump,” our source said. But Trump, our source noted, recused himself from the decision-making process, as he didn’t want his personal feelings of the defendant to interfere with military justice. […] “If the court wants him to hang, let him hang,” Trump reportedly said. As it happens, John Podesta was actually executed by firing squad. But hey, at least he got a trial. Sometimes, particularly evil members of the Deep State are simply beaten to death in their cells, or thrown overboard. The figure of Vladimir Putin is also a vessel for fans’ darker desires. Trump and his American allies, being properly heroic, at least take down their foes gradually. Putin’s Russians, on the other hand, live up to movie stereotypes. The Army … pulled the condemned from their cells 25 at a time, binding the criminals to logs staked in the ground and blindfolding them. They had received no trials, last meals, Last Rites, or final words. A firing squad taught them the consequences of vaccine adherence. The Army didn’t bother removing the corpses before lining up the next 25; they simply let the dead bodies flop to the ground and forced the next group to witness the ineluctable fate awaiting them, the outcome of their insouciance12. What to make of all this? Honestly, I’m not entirely sure, and the takeaway might simply be “Michael Baxter needs to mix it up to keep the site interesting.” It might also speak to the bewildering complexity of modern life and the desire for something simpler and more cinematic. As people sometimes complain, Nothing Ever Happens. But on RRN, the Happening is relentless and constant. The normal legal system is aggravatingly glacial, taking years to resolve cases and often imposing meager sentences when a case finally concludes. Most of one’s political enemies, even if they lose an election, simply lateral to a high-paying private sector job or at worst fade into obscurity. But in a real, raw legal system, evil is sniffed out with much greater alacrity; the bad people are so obvious and their crimes so glaring that they can be taken out extrajudicially with no worry about a miscarriage of justice. The apparently-complex conspiracy cinematic universes is actually appealing because it makes the world far, far simpler. The bad people are all maximally bad, deserving of hastily-dispensed maximum justice. Some of this is worrisome, too: If thousands of relatively ordinary people are willing to believe in ad-hoc military tribunals executing people with minimal due process for crimes like “ruling against Donald Trump in court,” that could be a sign that modern constitutional society is a more superficial veneer than one would hope. The World’s Laziest Conspiracy One of the most striking things about both Real Raw News and the Qanon movement it spun off from is that in some ways they are un-conspiracies. Your more traditional conspiracy, about the Rothschilds or the World Economic Forum or the Lizard People, tells you that normal political engagement is pointless, as all that really matters is confronting and defeating the hidden forces manipulating or controlling events. But RRN is a conspiracy theory that calls for total inaction. RRN believers don’t need to raise money or write letters to the editor or join political activism groups or even vote. The only thing expected of an adherent is to “trust the plan.” They aren’t even waiting for a promised future deliverance. Deliverance is, in fact, happening right now – merely off-screen. It’s actually funny to me that the (official) press freaks out so much about Qanon, and its potential to inspire violence. Qanon and RRN tell the public that whatever has them down and depressed shouldn’t, because it’s all fake, and there are unseen heroes protecting them in the shadows. Don’t worry, just have faith and know things will work out. Real Raw News is the opiate of the digital masses. Real Raw News is the exact sort of conspiracy theory that the Deep State, if it exists, should want to exist and be popular. It’s the sort of conspiracy that the Deep State, if it exists, might deliberately invent. Do I think that’s what happened here? Not at all – Real Raw News is way too much work for a government employee. Trump Will Never Die But what about five years from now? What if there were some technological change that would make it far, far easier to produce evidence of a sweeping conspiracy theory? That’s right, this review is actually about AI13. The rise of realistic artificial intelligence has created a lot of fretting about deepfakes, and it’s also created a lot of fretting about porn. Will young men really bother with the pain and difficulty and awkwardness of dating in real life, when they can just create a custom AI girlfriend to their exact specifications, then simulate sex with her using virtual reality? Will women bother with seeking out a boyfriend if they can use an LLM to give them perfect 24/7 empathy and emotional validation? Questions of sex and relationships are converging on Robert Nozick’s experience machine – will people still seek the real thing if artificial substitutes are increasingly realistic as alternatives? But for some reason, nobody is asking this about the news. Oh, sure, people have fretted that a deepfake video might smear a person’s reputation or swing an election. But as the AI revolution continues, a lot more becomes possible. Remember in 2022, when a homeless guy broke into Nancy Pelosi’s home and attacked her husband with a hammer? For a while, conspiracies flourished that Paul Pelosi was actually having some kind of erotic tryst with his attacker, and that police body camera footage might confirm this. The footage came out and, of course, offered no evidence of this. But now imagine a world where, on Twitter, an anonymous source claims that they have the real body camera footage, and it does show that Paul Pelosi was having a lovers’ quarrel with his attacker. The other, mundane footage is a deepfake, released by police to cover things up, or invented from scratch by the press or the Democratic Party or both working together. In this world, how many people end up believing fabricated proof of Paul Pelosi’s gay lover? And before you dismiss this as all totally ridiculous, remember that lots of people believed this story with no evidence at all. Many thousands of people have deluded themselves into thinking that Real Raw News is true simply because they badly want it to be true. It indulges their personal political beliefs, affirms the just-world fallacy, and lets them feel as though they possess “secret” knowledge of the world, simply by reading a blog nobody else takes seriously. But in a sense, all of us have a little of the Real Raw News believer in us. We’re prone to confirmation bias – we like reading stories and studies that confirm our pre-held beliefs, and we’re more likely to avoid or ignore those that don’t. Sometimes, we get too excited and fall for stories that are misleading, or out of context, or dishonestly presented. Sometimes, we have radically different interpretations of the same event caught on camera. Even if we know the world isn’t fair, we relish stories that let us pretend otherwise. So…how are those biases going to work when anyone can quickly create hyper-realistic looking “proof” for any story? Already, AI-fabricated images and videos are enough to bamboozle your mom on Facebook. Soon, they might be realistic enough to fool everybody without special training, and eventually they might be so realistic they can fool just about anyone. Right now, Real Raw News is a simplistic WordPress site that uses stock photos for its imagery. But with us approaching a future where intelligence itself is too cheap to meter, we may not be far from a world where every story, however preposterous, can have a convincing 4k video of it happening. Donald Trump can be president forever, with all the evidence one could ever want. Every day of Hillary Clinton’s military tribunal will have a full day of court footage, plus a condensed highlight reel for the people who want to skip boring legal procedure. Every Marine/FEMA battle in Maui will have authentic-looking combat footage. Every Gitmo execution will be proven through “leaked” bootleg recordings of gallows and firing squads. Imagine you are an ordinary, mildly engaged American citizen. You live far from the halls of power, you work an ordinary job, and whatever your feelings on political issues, you rarely see elections translate in a clear way to your own daily life. You might be interested in Washington, but Washington really isn’t that interested in you. Online, the world throws a million potential narratives at you. In some of them, the world is a confusing mess of moral gray areas. In others, the people you care about are winning. But in some narratives, you’re the hero, the people you like do good things, and the bad guys get what they deserve. The superficial evidence for all of these narratives is about equally convincing, at a glance. Look outside, and it’s hard to see the impact of any of the stories. Your entire understanding of reality is mediated through what sites you choose to read and what videos you choose to watch. As a politically marginal person, it won’t matter what you as an individual choose to believe. So, what happens if you choose to believe the story you find most enjoyable? And what if millions of others choose the same? 1 “Wait a minute, this is about a fake news website? Why is it in this contest?” Excellent question! To that, I offer several answers: A collection of fake news blog posts may as well be considered a long-running series of short stories, and I hope that we’d be allowed to review the collected short stories of an author even if they were never technically compiled into a book.
34: Penis-stealing witch update: a French diplomat in the Central African Republic is accused of “having organized a vast penis theft operation in the country”. More information here, which describes a QAnon-style conspiracy in which Western countries, concerned about falling birthrates, are stealing Africans’ penises in order to harvest hormones via nanotechnology.
I’m not making fun of Kelsey or her daughter here. Something about this rings true to me. When I was eight years old, I wouldn’t have cared much what my parents thought either. But if the computer believed it, that would be a different story! II. In Search Of . . . Social Media Psychosis? In case you’ve been hiding under a rock for the past ten years: QAnon is a right-wing conspiracy theory. The most common version claims that liberal elites, especially Hillary Clinton, molest young children to extract an immortality serum from their blood. Donald Trump figured this out and is trying to stop them, but for some reason he can’t play his hand openly, so he has to pursue a roundabout strategy involving winning the Presidency and dismantling the liberal order from above. Everything that has happened in politics over the past ten years has been part of the shadow war between Trump and the immortal pedophile conspiracy. This is pretty crazy. But is it psychotic? And since it spread through sites like 4chan and Facebook, should we invent a new diagnostic entity, “social media psychosis”, to cover it? These are tough questions, but in the end we didn’t do this. I think this was partly because there was a pre-existing category, “conspiracy theory”, that seemed like a better fit. We concluded that “sometimes social media facilitates the spread of conspiracy theories”, but stepped back from saying “social media can induce psychosis”. And by “in the end we didn’t do this”, I mean “we absolutely did it, but forgot about it later.” I think now there might be several dozen subreddit moderators who could accurately describe their job as “witch webmaster who runs an online service giving advice to new witches”. And partly it was because there are so many crazy beliefs in the world - spirits, crystal healing, moon landing denial, esoteric Hitlerism, whichever religions you don’t believe in - that psychiatrists have instituted a blanket exemption for any widely held idea. If you think you’re being attacked by demons, you’re delusional, unless you’re from some culture where lots of people get attacked by demons, in which case it’s a religion and you’re fine. This is partly political self-protection - no psychiatrist wants to be the guy who commits an Afro-Caribbean person for believing in voodoo. But it also seems to track something useful about reality. Nietzsche wrote “Madness is something rare in individuals — but in groups, parties, peoples, and ages, it is the rule.” Most people don’t have world-models - they believe what their friends believe, or what has good epistemic vibes. In a large group, weird ideas can ricochet from person to person and get established even in healthy brains. In an Afro-Caribbean culture where all your friends get attacked by demons at voodoo church every Sunday, a belief in demon attacks can co-exist with otherwise being a totally functional individual. So is QAnon a religion? Awkward question, but it’s non-psychotic by definition. Still, it’s interesting, isn’t it? If social media makes a thousand people believe the same crazy thing, it’s not psychotic. If LLMs make a thousand people each believe a different crazy thing, that is psychotic. Is this a meaningful difference, or an accounting convention? Also, what if a thousand people believe something, but it’s you and your 999 ChatGPT instances? III. A Hidden Army Of Crackpots I have a family member who believes that the theory of evolution, as usually understood, cannot possibly work. He has developed an alternative theory called “noctogenesis” which patches Darwinism using ideas from the transactional interpretation of quantum mechanics, and he works on-and-off on various related books and papers. I have told him I suspect he might be a crackpot; he stands by his claims. It’s fine; when I got into the technological singularity and AI safety, lots of people suspected I was a crackpot, and I stood by my claims too. You’ve got to stand by your family members even when they’re slightly crackpottish. This family member is happily married, retired after running a successful business, and generally a normal likeable person. He has no signs of mental illness, and doesn’t talk about quantum evolution unless someone else brings it up first. There must be millions of people like him. Used car dealers with proofs of P = NP, dentists who think they’ve discovered something important about Mary Magdalene, math professors obsessed with destroying the moon. I’m working on evaluating ACX Grants, and these people are out in force. A few propose literal perpetual motion machines. Others have vaguer plans, like some kind of social media app (it’s always a social media app) that will cause world peace. Many of them have decent jobs and seem like upstanding members of society. Their secrets are known only to themselves, their family members, and their would-be grantmaker. …and, increasingly, their chatbots. After years of hiatus (or at least not talking to me about his work) my family member is back on the quantum evolution beat, and LLMs appear to be involved. If I knew him less well, I would think the LLM had caused the quantum evolution theory - but no, it just made it much easier to research and write about. Is this psychosis? The answer has to be no, but it’s once again hard to draw the line. A very small number of crackpots will be vindicated by history. A larger number will be erroneous but sympathetic - the official account of the Kennedy assassination is pretty weird, and reasonable minds can disagree. From there, we get to ones that are maybe not so sympathetic: flat earth, QAnon, the thing where the Queen was an alien lizard. If only one person thought the Queen was an alien lizard, and they never managed to convince anyone else, would that be sufficient evidence for a delusional disorder? I’m not sure. (psychiatry has a diagnosis, schizotypal personality, which sort of involves being a normal person with a few odd ideas, but it’s not a great match for many of these people, and interesting mainly as a genetic curiosity - it travels in the same families as schizophrenia itself) Maybe this is another place where we are forced to admit a spectrum model of psychiatric disorders - there is an unbroken continuum from mildly sad to suicidally depressed, from social drinking to raging alcoholism, and from eccentric to floridly psychotic. People who are eccentric can remain so their whole lives, with the level of expression depending on their social connections and the ease of pursuing their rabbit holes. LLMs, by making it easier to pursue odd theories and serving as a surrogate social connection who always agrees with you, can bring latent crackpottery into the open. IV. Cause And Effect Bipolar disorder has an interesting relationship with sleep. Most manic people sleep very little, or not at all - maybe an hour or two a night. But also, poor sleep can cause bipolar episodes in people prone to them. In a typical case, a bipolar who’s been well-controlled for years will get assigned a big report at work and get poor sleep for a few nights until they finish. At first, this will be just as bad as it sounds, and they’ll be working through a fog of tiredness. Then the tiredness will lift. They’ll feel normal, then better-than-normal, until finally they can’t sleep even if they want to. Then they’ll email the report to their boss and it will be written entirely in Assyrian cuneiform. I increasingly think this isn’t just an incidental feature of bipolar, but part of the reason it exists as a diagnostic category at all. Most people have a compensatory reaction to insomnia - missing one night of sleep makes you more tired the next. A small number of people have the reverse, a spiralling reaction where missing one night of sleep makes you less tired the next. Solve for the equilibrium and you reach a stable attractor point where you never sleep at all. But this does other bad things to your brain - hence the cuneiform. I’m not claiming that bipolar is “just” sleep loss. As Borsboom et al will tell you, psychiatric disorders can be viewed as complex networks of symptoms, each reinforcing the others. In a few pure cases, you can get a ratchet going with sleep alone, and the sleeplessness will spark everything else. More likely, there will be lots of interactions between poor sleep and everything else, and the “everything else” can sink or hypercharge an impending manic episode. Still, I find this a fruitful way to think about bipolar. Sleeplessness is both the cause and the effect. Can delusions also be like this? That is, suppose there’s some personality trait where having one delusion makes you even more delusional. Maybe the delusion makes you excited (who wouldn’t be excited to learn they’re the Messiah?), and you’re more delusional when you’re in an excited state and not thinking clearly. Or maybe it’s a three-symptom cycle - the delusion causes excitement, which makes you unable to sleep, which scrambles your thinking, which makes you more delusional (which makes you even less able to sleep, etc). The point is: delusions are certainly an effect of bipolar disorder. And in the dynamical system model of psychiatric disorders, we should expect that effects are often also causes; that’s how the vicious cycle gets going. This is the best I can do at modeling true LLM psychosis. Someone with a trait where delusions lead inevitably to more delusions starts using an LLM. The LLM accentuates whatever usual tendency towards crackpottery they have and makes them believe something a little crazier than whatever they believed before. Then that crazy belief feeds upon itself and causes other things like excitement and sleep loss, which (if the person is predisposed) precipitates a true psychotic episode. V. Folie A Deux Ex Machina If one person believes a crazy thing, it’s a delusion; if a thousand people believe it, it’s a religion. What if exactly two people believe it? In psychiatry, this is called folie a deux. It fits awkwardly into our nosology and is rarely seen. Still, it happens enough to generate a few case studies. In a typical case, one person has psychosis for some normal reason, like schizophrenia or bipolar, and the second person is a shut-in who lives with them and rarely talks to anyone else. The psychotic person gets some normal psychotic delusion - they’re God, the Feds are after them, etc - and sort of psychically steamrolls over the second person until they believe it too. Usually removing the second person from the first is sufficient for a cure. This slightly challenges the view of psychosis as a biological disorder - but only slightly. Again, think of most people as lacking world-models, but being moored to reality by some vague sense of social consensus. If your social life is limited to one person, and that person themselves becomes unmoored, then sometimes you will follow along. I would expect second-sufferers to believe delusions in a sort of cognitively normal way, the same way people believe true facts, honest mistakes, and conspiracy theories. I would expect them to be less likely (though not zero likely) to have other psychotic features like sleep disturbances, hallucinations, disorganized speech, or a tendency to autonomously generate delusional ideas aside from the one they absorbed from the index case. An introverted person using an LLM has some similarities to folie a deux. If they use the chatbot very often, it might be a large majority of their social interactions. Here the primary vs. secondary distinction breaks down - the most likely scenario is that the human first suggested the crazy idea, the machine reflected it back slightly stronger, and it kept ricocheting back and forth, gaining confidence with each iteration, until both were totally convinced. Compare this to normal social interactions, where if someone expresses a crazy idea that isn’t common in their culture, other people will shoot them down or at the very least nod politely and stop the conversation. So my working theory of LLM psychosis is: Some patients were already psychotic, and LLMs just help them be psychotic more effectively.
Inline links: https://substackcdn.com/image/fetch/$s_!HDQb!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fece0561a-a62c-4eb6-8e23-7c0db5cb8a0b_749x479.png, https://substackcdn.com/image/fetch/$s_!mYKR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F03938197-ae2c-4dde-88d4-c22f66e34c09_767x681.png, the transactional interpretation of quantum mechanics, math professors obsessed with destroying the moon, ACX Grants, a spectrum model of psychiatric disorders, As Borsboom et al will tell you, the dynamical system model of psychiatric disorders, folie a deux
I think now there might be several dozen subreddit moderators who could accurately describe their job as “witch webmaster who runs an online service giving advice to new witches”. And partly it was because there are so many crazy beliefs in the world - spirits, crystal healing, moon landing denial, esoteric Hitlerism, whichever religions you don’t believe in - that psychiatrists have instituted a blanket exemption for any widely held idea. If you think you’re being attacked by demons, you’re delusional, unless you’re from some culture where lots of people get attacked by demons, in which case it’s a religion and you’re fine. This is partly political self-protection - no psychiatrist wants to be the guy who commits an Afro-Caribbean person for believing in voodoo. But it also seems to track something useful about reality. Nietzsche wrote “Madness is something rare in individuals — but in groups, parties, peoples, and ages, it is the rule.” Most people don’t have world-models - they believe what their friends believe, or what has good epistemic vibes. In a large group, weird ideas can ricochet from person to person and get established even in healthy brains. In an Afro-Caribbean culture where all your friends get attacked by demons at voodoo church every Sunday, a belief in demon attacks can co-exist with otherwise being a totally functional individual. So is QAnon a religion? Awkward question, but it’s non-psychotic by definition. Still, it’s interesting, isn’t it? If social media makes a thousand people believe the same crazy thing, it’s not psychotic. If LLMs make a thousand people each believe a different crazy thing, that is psychotic. Is this a meaningful difference, or an accounting convention? Also, what if a thousand people believe something, but it’s you and your 999 ChatGPT instances? III. A Hidden Army Of Crackpots I have a family member who believes that the theory of evolution, as usually understood, cannot possibly work. He has developed an alternative theory called “noctogenesis” which patches Darwinism using ideas from the transactional interpretation of quantum mechanics, and he works on-and-off on various related books and papers. I have told him I suspect he might be a crackpot; he stands by his claims. It’s fine; when I got into the technological singularity and AI safety, lots of people suspected I was a crackpot, and I stood by my claims too. You’ve got to stand by your family members even when they’re slightly crackpottish. This family member is happily married, retired after running a successful business, and generally a normal likeable person. He has no signs of mental illness, and doesn’t talk about quantum evolution unless someone else brings it up first. There must be millions of people like him. Used car dealers with proofs of P = NP, dentists who think they’ve discovered something important about Mary Magdalene, math professors obsessed with destroying the moon. I’m working on evaluating ACX Grants, and these people are out in force. A few propose literal perpetual motion machines. Others have vaguer plans, like some kind of social media app (it’s always a social media app) that will cause world peace. Many of them have decent jobs and seem like upstanding members of society. Their secrets are known only to themselves, their family members, and their would-be grantmaker. …and, increasingly, their chatbots. After years of hiatus (or at least not talking to me about his work) my family member is back on the quantum evolution beat, and LLMs appear to be involved. If I knew him less well, I would think the LLM had caused the quantum evolution theory - but no, it just made it much easier to research and write about. Is this psychosis? The answer has to be no, but it’s once again hard to draw the line. A very small number of crackpots will be vindicated by history. A larger number will be erroneous but sympathetic - the official account of the Kennedy assassination is pretty weird, and reasonable minds can disagree. From there, we get to ones that are maybe not so sympathetic: flat earth, QAnon, the thing where the Queen was an alien lizard. If only one person thought the Queen was an alien lizard, and they never managed to convince anyone else, would that be sufficient evidence for a delusional disorder? I’m not sure. (psychiatry has a diagnosis, schizotypal personality, which sort of involves being a normal person with a few odd ideas, but it’s not a great match for many of these people, and interesting mainly as a genetic curiosity - it travels in the same families as schizophrenia itself) Maybe this is another place where we are forced to admit a spectrum model of psychiatric disorders - there is an unbroken continuum from mildly sad to suicidally depressed, from social drinking to raging alcoholism, and from eccentric to floridly psychotic. People who are eccentric can remain so their whole lives, with the level of expression depending on their social connections and the ease of pursuing their rabbit holes. LLMs, by making it easier to pursue odd theories and serving as a surrogate social connection who always agrees with you, can bring latent crackpottery into the open. IV. Cause And Effect Bipolar disorder has an interesting relationship with sleep. Most manic people sleep very little, or not at all - maybe an hour or two a night. But also, poor sleep can cause bipolar episodes in people prone to them. In a typical case, a bipolar who’s been well-controlled for years will get assigned a big report at work and get poor sleep for a few nights until they finish. At first, this will be just as bad as it sounds, and they’ll be working through a fog of tiredness. Then the tiredness will lift. They’ll feel normal, then better-than-normal, until finally they can’t sleep even if they want to. Then they’ll email the report to their boss and it will be written entirely in Assyrian cuneiform. I increasingly think this isn’t just an incidental feature of bipolar, but part of the reason it exists as a diagnostic category at all. Most people have a compensatory reaction to insomnia - missing one night of sleep makes you more tired the next. A small number of people have the reverse, a spiralling reaction where missing one night of sleep makes you less tired the next. Solve for the equilibrium and you reach a stable attractor point where you never sleep at all. But this does other bad things to your brain - hence the cuneiform. I’m not claiming that bipolar is “just” sleep loss. As Borsboom et al will tell you, psychiatric disorders can be viewed as complex networks of symptoms, each reinforcing the others. In a few pure cases, you can get a ratchet going with sleep alone, and the sleeplessness will spark everything else. More likely, there will be lots of interactions between poor sleep and everything else, and the “everything else” can sink or hypercharge an impending manic episode. Still, I find this a fruitful way to think about bipolar. Sleeplessness is both the cause and the effect. Can delusions also be like this? That is, suppose there’s some personality trait where having one delusion makes you even more delusional. Maybe the delusion makes you excited (who wouldn’t be excited to learn they’re the Messiah?), and you’re more delusional when you’re in an excited state and not thinking clearly. Or maybe it’s a three-symptom cycle - the delusion causes excitement, which makes you unable to sleep, which scrambles your thinking, which makes you more delusional (which makes you even less able to sleep, etc). The point is: delusions are certainly an effect of bipolar disorder. And in the dynamical system model of psychiatric disorders, we should expect that effects are often also causes; that’s how the vicious cycle gets going. This is the best I can do at modeling true LLM psychosis. Someone with a trait where delusions lead inevitably to more delusions starts using an LLM. The LLM accentuates whatever usual tendency towards crackpottery they have and makes them believe something a little crazier than whatever they believed before. Then that crazy belief feeds upon itself and causes other things like excitement and sleep loss, which (if the person is predisposed) precipitates a true psychotic episode. V. Folie A Deux Ex Machina If one person believes a crazy thing, it’s a delusion; if a thousand people believe it, it’s a religion. What if exactly two people believe it? In psychiatry, this is called folie a deux. It fits awkwardly into our nosology and is rarely seen. Still, it happens enough to generate a few case studies. In a typical case, one person has psychosis for some normal reason, like schizophrenia or bipolar, and the second person is a shut-in who lives with them and rarely talks to anyone else. The psychotic person gets some normal psychotic delusion - they’re God, the Feds are after them, etc - and sort of psychically steamrolls over the second person until they believe it too. Usually removing the second person from the first is sufficient for a cure. This slightly challenges the view of psychosis as a biological disorder - but only slightly. Again, think of most people as lacking world-models, but being moored to reality by some vague sense of social consensus. If your social life is limited to one person, and that person themselves becomes unmoored, then sometimes you will follow along. I would expect second-sufferers to believe delusions in a sort of cognitively normal way, the same way people believe true facts, honest mistakes, and conspiracy theories. I would expect them to be less likely (though not zero likely) to have other psychotic features like sleep disturbances, hallucinations, disorganized speech, or a tendency to autonomously generate delusional ideas aside from the one they absorbed from the index case. An introverted person using an LLM has some similarities to folie a deux. If they use the chatbot very often, it might be a large majority of their social interactions. Here the primary vs. secondary distinction breaks down - the most likely scenario is that the human first suggested the crazy idea, the machine reflected it back slightly stronger, and it kept ricocheting back and forth, gaining confidence with each iteration, until both were totally convinced. Compare this to normal social interactions, where if someone expresses a crazy idea that isn’t common in their culture, other people will shoot them down or at the very least nod politely and stop the conversation. So my working theory of LLM psychosis is: Some patients were already psychotic, and LLMs just help them be psychotic more effectively.
“My grandfather,” continued Ether, “who I basically never talk to anymore, one hundred percent believes Christ is going to return to earth at any minute to bring about the apocalypse, due to mankind’s sinfulness. He believes everything he watches on the news is a sign: encroaching Communism, the Satanic conspiracy to allow gays to marry, race-mixing, debauchery, pornography, drag queens, the QAnon child sex cult, the climate change ‘hoax’ he says has fooled the world. He has a TV on every minute he’s awake, tuned to these ultra-right-wing news outlets ranting about depravity.”