Sam Altman

Article

Sam Altman is a recurring person in the Astral Codex Ten archive, appearing 41 times across 41 issues between December 30, 2021 and March 01, 2026. The archive places it in contexts such as “Ten Minutes With Sam Altman. A weird cute vignette by a would-be entrepreneur”; “speakers including Sam Altman”; “Sam Altman writes in to say he enjoyed the recent book review of The Future Of Fusion Energy”. It most often appears alongside OpenAI, Anthropic, Elon Musk.

Metadata

Category: People
Mention count: 41
Issue count: 41
First seen: December 30, 2021
Last seen: March 01, 2026

Appears In

- OpenAI (22 shared issues)
- Anthropic (18 shared issues)
- Elon Musk (13 shared issues)
- Metaculus (10 shared issues)
- Trump (10 shared issues)
- Twitter (10 shared issues)
- ChatGPT (8 shared issues)
- AI (7 shared issues)
- Eliezer Yudkowsky (7 shared issues)
- Google (7 shared issues)
- Manifold (7 shared issues)
- Aella (6 shared issues)

External Links

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

Links For December

December 30, 2021 · Original source

33: Ten Minutes With Sam Altman. A weird cute vignette by a would-be entrepreneur about his Y Combinator interview. I always like unusual experiences related by good writers, although this is weirdly short and leaves me wanting more.

Inline links: Ten Minutes With Sam Altman

Open Thread 231

July 04, 2022 · Original source

2: Isaak Freeman asks me to signal-boost the Future Forum, from August 4-7 in San Francisco, featuring speakers including Sam Altman, Anders Sandberg, Patrick Collison, and Tyler Cowen. They are bringing together 250 people from EA, Silicon Valley, and related communities to “arm the world's brightest minds with the tools they need to tackle global problems” and to potentially offer funding and mentoring. Apply at the link above.

Inline links: Future Forum

Open Thread 232

July 10, 2022 · Original source

2: Speaking of turning money into big positive impacts on the future, Sam Altman writes in to say he enjoyed the recent book review of The Future Of Fusion Energy, and that he predicts Helion (the fusion startup he’s a leading investor in) has a 85% chance of Q > 1 D-T fusion by 2024, and a 65% chance of Q > 1 D-He3 fusion by 2026.

Inline links: book review of, Helion

Why Not Slow AI Progress?

August 08, 2022 · Original source

OpenAI is the company behind GPT-3 and DALL-E. The media announced them as Elon Musk Just Founded A New Company To Make Sure Artificial Intelligence Doesn’t Destroy The World. The same article quotes co-founder and current OpenAI CEO Sam Altman as saying that “AI will probably most likely lead to the end of the world, but in the meantime, there'll be great companies”. OpenAI’s public statement on its own foundation said:

Inline links: Elon Musk Just Founded A New Company To Make Sure Artificial Intelligence Doesn’t Destroy The World

But they can’t use the “trying to avoid a race” argument - their creation probably started a race (against DeepMind). So what were they thinking? I still haven’t fully figured this out, but here’s Altman’s own explanation: Toby Ord (here standing in for the broader existential-risk-quantifying community) has estimated the risk of extinction-level asteroid impacts as 0.0001% per century, and the risk of extinction from building AI too fast as 10%. So as written, this argument isn’t very good. But you could revise it to be about metaphorical “asteroids” like superplagues or nuclear war. Altman has also expressed concern about AI causing inequality, for example if rich people use it to replace all labor and reap all the gains for themselves. OpenAI was originally founded as a nonprofit in a way that protected against that, so maybe he thought that made it preferable to DeepMind.

Inline links: has estimated

Suppose that the alignment community, without thinking it over too closely, started a climate-change-style campaign to shame AI capabilities companies. This would disproportionately harm the companies most capable of feeling shame. Right now, that’s DeepMind, OpenAI, and Anthropic - the companies that have AI safety as part of their culture, cross-pollinate with AI safety proponents, and take our arguments seriously. If all these places slowed down - either because their leadership saw the light, or because their employees grumbled and quit - that would hand the technological lead to the people just behind them - like Facebook and Salesforce - who care less about safety. So which would we prefer? OpenAI gets superintelligence in 2040? Or Facebook gets superintelligence in 2044?

Book Review Contest 2022 Winners

September 02, 2022 · Original source

Kora In Hell, reviewed by Lucas Paletta. Lucas is a writer from Buenos Aires, Argentina. He blogs (in Spanish) at www.stackdamage.com.ar. I really enjoyed all of these. A few notes of special praise: The Internationalists was probably most fascinating, in the sense of describing a strange historical episode I didn’t know about before. The Outlier was similar and I give it high marks for making Jimmy Carter interesting. Consciousness And The Brain was a whole new neuroscience theory I knew nothing about and I expect to reread it a bunch of times to try to get it to sink in. Sam Altman sent me an email saying he enjoyed the review of The Future Of Fusion Energy. The Making Nature review did a great job talking about and analyzing a trend I’d never thought about before, far beyond even what was in the book. I think about Exhaustion every time I see a CFS patient - specifically, about the claim that 19th century psychiatrists would prescribe a “West cure” of going off and doing cowboy things on a ranch; I haven’t yet recommended that to anyone, but like I said, I think about it often. God Emperor of Dune and Kora In Hell were the token fiction and poetry reviews; I thought they did a spectacular job overcoming the difficulties of reviewing their respective media. I was reading some of the non-finalists and found 1587 in there and was surprised it hadn’t reached finalist status and decided to promote it; based on your votes it seems like that was the right choice. My process for picking finalists was kind of haphazard; I had you rate all reviews on a scale of 1-10, anyone above 8 got in automatically, and then I picked my favorites from the reviews between 7 and 8. This was sort of unfair, and meant there were some reviews that scored better on the voting than finalists but weren’t finalists themselves, and others that I liked better than some finalists but couldn’t pick. All of these are Honorable Mentions. You’ll notice some of them are politically charged, and yes, I did sort of discriminate against these (though not so much that I wouldn’t have picked them if they’d made it above 8). They are: Unsettled, reviewed by Julius S. Julius is a machine learning engineer from San Diego. He blogs at Curious About Ideas.

Inline links: Kora In Hell, www.stackdamage.com.ar, Unsettled, Curious About Ideas

October 18, 2022 · Original source

Sam Altman, OpenAI and Y Combinator

Inline links: Sam Altman

OpenAI's "Planning For AGI And Beyond"

March 01, 2023 · Original source

Even if they’re trying to be honest, will their bottom line bias them towards waiting for some final apocalyptic proof that “now climate change is a crisis”, of a sort that will never happen, so they don’t have to stop pumping oil? This is how I feel about OpenAI’s new statement, Planning For AGI And Beyond. OpenAI is the AI company behind ChatGPT and DALL-E. In the past, people (including me) have attacked them for seeming to deprioritize safety. Their CEO, Sam Altman, insists that safety is definitely a priority, and has recently been sending various signals to that effect. Sam Altman posing with leading AI safety proponent Eliezer Yudkowsky. Also Grimes for some reason. Planning For AGI And Beyond (“AGI” = “artificial general intelligence”, ie human-level AI) is the latest volley in that campaign. It’s very good, in all the ways ExxonMobil’s hypothetical statement above was very good. If they’re trying to fool people, they’re doing a convincing job! Still, it doesn’t apologize for doing normal AI company stuff in the past, or plan to stop doing normal AI company stuff in the present. It just says that, at some indefinite point when they decide AI is a threat, they’re going to do everything right. This is more believable when OpenAI says it than when ExxonMobil does. There are real arguments for why an AI company might want to switch from moving fast and breaking things at time t to acting all responsible at time t + 1 . Let’s explore the arguments they make in the document, go over the reasons they’re obviously wrong, then look at the more complicated arguments they might be based off of. Why Doomers Think OpenAI Is Bad And Should Have Slowed Research A Long Time Ago OpenAI boosters might object: there’s a disanalogy between the global warming story above and AI capabilities research. Global warming is continuously bad: a temperature increase of 0.5 degrees C is bad, 1.0 degrees is worse, and 1.5 degrees is worse still. AI doesn’t become dangerous until some specific point. GPT-3 didn’t hurt anyone. GPT-4 probably won’t hurt anyone. So why not keep building fun chatbots like these for now, then start worrying later? Doomers counterargue that the fun chatbots burn timeline. That is, suppose you have some timeline for when AI becomes dangerous. For example, last year Metaculus thought human-like AI would arrive in 2040, and superintelligence around 2043. Recent AIs have tried lying to, blackmailing, threatening, and seducing users. AI companies freely admit they can’t really control their AIs, and it seems high-priority to solve that before we get superintelligence. If you think that’s 2043, the people who work on this question (“alignment researchers”) have twenty years to learn to control AI. Then OpenAI poured money into AI, did ground-breaking research, and advanced the state of the art. That meant that AI progress would speed up, and AI would reach the danger level faster. Now Metaculus expects superintelligence in 2031, not 2043 (although this seems kind of like an over-update), which gives alignment researchers eight years, not twenty. So the faster companies advance AI research - even by creating fun chatbots that aren’t dangerous themselves - the harder it is for alignment researchers to solve their part of the problem in time. This is why some AI doomers think of OpenAI as an Exxon-Mobil style villain, even though they’ve promised to change course before the danger period. Imagine an environmentalist group working on research and regulatory changes that would have solar power ready to go in 2045. Then ExxonMobil invents a new kind of super-oil that ensures that, nope, all major cities will be underwater by 2031 now. No matter how nice a statement they put out, you’d probably be pretty mad! Why OpenAI Thinks Their Research Is Good Now, But Might Be Bad Later OpenAI understands the argument against burning timeline. But they counterargue that having the AIs speeds up alignment research and all other forms of social adjustment to AI. If we want to prepare for superintelligence - whether solving the technical challenge of alignment, or solving the political challenges of unemployment, misinformation, etc - we can do this better when everything is happening gradually and we’ve got concrete AIs to think about: We believe we have to continuously learn and adapt by deploying less powerful versions of the technology in order to minimize “one shot to get it right” scenarios […] As we create successively more powerful systems, we want to deploy them and gain experience with operating them in the real world. We believe this is the best way to carefully steward AGI into existence—a gradual transition to a world with AGI is better than a sudden one. We expect powerful AI to make the rate of progress in the world much faster, and we think it’s better to adjust to this incrementally. A gradual transition gives people, policymakers, and institutions time to understand what’s happening, personally experience the benefits and downsides of these systems, adapt our economy, and to put regulation in place. It also allows for society and AI to co-evolve, and for people collectively to figure out what they want while the stakes are relatively low. You might notice that, as written, this argument doesn’t support full-speed-ahead AI research. If you really wanted this kind of gradual release that lets society adjust to less powerful AI, you would do something like this: Release AI #1

Inline links: Planning For AGI And Beyond, attacked them for seeming to deprioritize safety, https://substackcdn.com/image/fetch/$s_!k2Db!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95482196-a442-48ba-9e90-c7d681c5dd1d_1517x1491.png, Metaculus, they can’t really control their AIs

Sam Altman posing with leading AI safety proponent Eliezer Yudkowsky. Also Grimes for some reason. Planning For AGI And Beyond (“AGI” = “artificial general intelligence”, ie human-level AI) is the latest volley in that campaign. It’s very good, in all the ways ExxonMobil’s hypothetical statement above was very good. If they’re trying to fool people, they’re doing a convincing job! Still, it doesn’t apologize for doing normal AI company stuff in the past, or plan to stop doing normal AI company stuff in the present. It just says that, at some indefinite point when they decide AI is a threat, they’re going to do everything right. This is more believable when OpenAI says it than when ExxonMobil does. There are real arguments for why an AI company might want to switch from moving fast and breaking things at time t to acting all responsible at time t + 1 . Let’s explore the arguments they make in the document, go over the reasons they’re obviously wrong, then look at the more complicated arguments they might be based off of. Why Doomers Think OpenAI Is Bad And Should Have Slowed Research A Long Time Ago OpenAI boosters might object: there’s a disanalogy between the global warming story above and AI capabilities research. Global warming is continuously bad: a temperature increase of 0.5 degrees C is bad, 1.0 degrees is worse, and 1.5 degrees is worse still. AI doesn’t become dangerous until some specific point. GPT-3 didn’t hurt anyone. GPT-4 probably won’t hurt anyone. So why not keep building fun chatbots like these for now, then start worrying later? Doomers counterargue that the fun chatbots burn timeline. That is, suppose you have some timeline for when AI becomes dangerous. For example, last year Metaculus thought human-like AI would arrive in 2040, and superintelligence around 2043. Recent AIs have tried lying to, blackmailing, threatening, and seducing users. AI companies freely admit they can’t really control their AIs, and it seems high-priority to solve that before we get superintelligence. If you think that’s 2043, the people who work on this question (“alignment researchers”) have twenty years to learn to control AI. Then OpenAI poured money into AI, did ground-breaking research, and advanced the state of the art. That meant that AI progress would speed up, and AI would reach the danger level faster. Now Metaculus expects superintelligence in 2031, not 2043 (although this seems kind of like an over-update), which gives alignment researchers eight years, not twenty. So the faster companies advance AI research - even by creating fun chatbots that aren’t dangerous themselves - the harder it is for alignment researchers to solve their part of the problem in time. This is why some AI doomers think of OpenAI as an Exxon-Mobil style villain, even though they’ve promised to change course before the danger period. Imagine an environmentalist group working on research and regulatory changes that would have solar power ready to go in 2045. Then ExxonMobil invents a new kind of super-oil that ensures that, nope, all major cities will be underwater by 2031 now. No matter how nice a statement they put out, you’d probably be pretty mad! Why OpenAI Thinks Their Research Is Good Now, But Might Be Bad Later OpenAI understands the argument against burning timeline. But they counterargue that having the AIs speeds up alignment research and all other forms of social adjustment to AI. If we want to prepare for superintelligence - whether solving the technical challenge of alignment, or solving the political challenges of unemployment, misinformation, etc - we can do this better when everything is happening gradually and we’ve got concrete AIs to think about: We believe we have to continuously learn and adapt by deploying less powerful versions of the technology in order to minimize “one shot to get it right” scenarios […] As we create successively more powerful systems, we want to deploy them and gain experience with operating them in the real world. We believe this is the best way to carefully steward AGI into existence—a gradual transition to a world with AGI is better than a sudden one. We expect powerful AI to make the rate of progress in the world much faster, and we think it’s better to adjust to this incrementally. A gradual transition gives people, policymakers, and institutions time to understand what’s happening, personally experience the benefits and downsides of these systems, adapt our economy, and to put regulation in place. It also allows for society and AI to co-evolve, and for people collectively to figure out what they want while the stakes are relatively low. You might notice that, as written, this argument doesn’t support full-speed-ahead AI research. If you really wanted this kind of gradual release that lets society adjust to less powerful AI, you would do something like this: Release AI #1

Inline links: Planning For AGI And Beyond, Metaculus, they can’t really control their AIs

Is this too cynical? I’m not sure. On the one hand, OpenAI has been expressing concern about safety since day one - the article announcing their founding in 2015 was titled Elon Musk Just Founded A New Company To Make Sure Artificial Intelligence Doesn’t Destroy The World.

Inline links: Elon Musk Just Founded A New Company To Make Sure Artificial Intelligence Doesn’t Destroy The World

Turing Test

March 27, 2023 · Original source

FIRE: This is “The Ballad Of Eliezer Yudkowsky And Sam Altman”:

One rainy evening at a bar, Eliezer told Sam Altman "AI could be the end of us, your research has to halt, man We can't maintain control; alignment isn't the default, man So just in case, slow down your pace," Eliezer told Sam Altman

With downcast eyes and heavy heart, Eliezer left Sam Altman Some years go by, and AGI progresses to assault man Atop a pile of paper clips he screams "It's not my fault, man!" But Eliezer's long since dead, and cannot hear Sam Altman.

MR Tries The Safe Uncertainty Fallacy

March 30, 2023 · Original source

Therefore, it’ll be fine. You’re not missing anything. It’s not supposed to make sense; that’s why it’s a fallacy. For years, people used the Safe Uncertainty Fallacy on AI timelines: Eliezer didn’t realize that at our level, you can just name fallacies. Since 2017, AI has moved faster than most people expected; GPT-4 sort of qualifies as an AGI, the kind of AI most people were saying was decades away. When you have ABSOLUTELY NO IDEA when something will happen, sometimes the answer turns out to be “soon”. Now Tyler Cowen of Marginal Revolution tries his hand at this argument. We have absolutely no idea how AI will go, it’s radically uncertain: No matter how positive or negative the overall calculus of cost and benefit, AI is very likely to overturn most of our apple carts, most of all for the so-called chattering classes. The reality is that no one at the beginning of the printing press had any real idea of the changes it would bring. No one at the beginning of the fossil fuel era had much of an idea of the changes it would bring. No one is good at predicting the longer-term or even medium-term outcomes of these radical technological changes (we can do the short term, albeit imperfectly). No one. Not you, not Eliezer, not Sam Altman, and not your next door neighbor. How well did people predict the final impacts of the printing press? How well did people predict the final impacts of fire? We even have an expression “playing with fire.” Yet it is, on net, a good thing we proceeded with the deployment of fire (“Fire? You can’t do that! Everything will burn! You can kill people with fire! All of them! What if someone yells “fire” in a crowded theater!?”). Therefore, it’ll be fine: I am a bit distressed each time I read an account of a person “arguing himself” or “arguing herself” into existential risk from AI being a major concern. No one can foresee those futures! Once you keep up the arguing, you also are talking yourself into an illusion of predictability. Since it is easier to destroy than create, once you start considering the future in a tabula rasa way, the longer you talk about it, the more pessimistic you will become. It will be harder and harder to see how everything hangs together, whereas the argument that destruction is imminent is easy by comparison. The case for destruction is so much more readily articulable — “boom!” Yet at some point your inner Hayekian (Popperian?) has to take over and pull you away from those concerns. (Especially when you hear a nine-part argument based upon eight new conceptual categories that were first discussed on LessWrong eleven years ago.) Existential risk from AI is indeed a distant possibility, just like every other future you might be trying to imagine. All the possibilities are distant, I cannot stress that enough. The mere fact that AGI risk can be put on a par with those other also distant possibilities simply should not impress you very much. So we should take the plunge. If someone is obsessively arguing about the details of AI technology today, and the arguments on LessWrong from eleven years ago, they won’t see this. Don’t be suckered into taking their bait. Look. It may well be fine. I said before my chance of existential risk from AI is 33%; that means I think there’s a 66% chance it won’t happen. In most futures, we get through okay, and Tyler gently ribs me for being silly. Don’t let him. Even if AI is the best thing that ever happens and never does anything wrong and from this point forward never even shows racial bias or hallucinates another citation ever again, I will stick to my position that the Safe Uncertainty Fallacy is a bad argument. Normally this would be the point where I try to steelman Tyler and explain in more detail why the strongest version of his case is wrong. But I’m having trouble figuring out what the strong version is. Here are three possibilities: 1) The base rate for things killing humanity is very low, so we would need a strong affirmative argument to shift our estimate away from that base rate. Since there’s so much uncertainty, we don’t have strong affirmative arguments, and we should stick with our base rate of “very low”. Suppose astronomers spotted a 100-mile long alien starship approaching Earth. Surely this counts as a radically uncertain situation if anything does; we have absolutely no idea what could happen. Therefore - the alien starship definitely won’t kill us and it’s not worth worrying? Seems wrong. What’s the base rate for alien starships approaching Earth killing humanity? We don’t have a base rate, because we’ve never been in this situation before. What is the base rate for developing above-human-level AI killing humanity? We don’t . . . you get the picture. You can try to fish for something sort of like a base rate: “There have been a hundred major inventions since agriculture, and none of them killed humanity, so the base rate for major inventions killing everyone is about 0%”. But I can counterargue: “There have been about a dozen times a sapient species has created a more intelligent successor species: australopithecus → homo habilis, homo habilis → homo erectus, etc - and in each case, the successor species has wiped out its predecessor. So the base rate for more intelligent successor species killing everyone is about 100%”. The Less Wrongers call this game “reference class tennis”, and insist that the only winning move is not to play. Thinking about this question in terms of base rates is just as hard as thinking of it any other way, and would require arguments for why one base rate is better than another. Tyler hasn’t made any. 2) There are so many different possibilities - let’s say 100! - and dying is only one of them, so there’s only a 1% chance that we’ll die. This is sort of how I interpret: Existential risk from AI is indeed a distant possibility, just like every other future you might be trying to imagine. All the possibilities are distant, I cannot stress that enough. The mere fact that AGI risk can be put on a par with those other also distant possibilities simply should not impress you very much. Alien time again! Here are some possible ways the hundred-mile long starship situation could end: The aliens are peaceful and want to share their advanced technology

Inline links: https://substackcdn.com/image/fetch/$s_!jYyh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F672bd8f5-484b-48ff-a80f-539d7b77b66c_599x238.png, sort of qualifies as an AGI, tries his hand at this argument

April 25, 2023 · Original source

The drop a few days ago was when Sam Altman said OpenAI wasn’t currently training GPT-5 and “won’t for some time”. Apparently forecasters don’t expect them to take too long a break.

Links For May 2023

June 01, 2023 · Original source

Can’t even list all the new people who have come out as AI x-risk believers, but you can just read the CAIS statement. The top signatures are Geoff Hinton, Yoshua Bengio, Demis Hassabis, Sam Altman, and Dario Amodei; aside from the usual suspects, they also have Bruce Schneier (computer security expert) , Dawn Song (computer scientist and security expert), Andy Clark (professor of cognitive philosophy, wrote Surfing Uncertainty), Eliezer Yudkowsky (he didn't sign the last one because he disagreed with specifics, but he's here), and a former US Assistant Secretary of Defense for Nuclear, Chemical, and Biological Defense.

Inline links: CAIS statement

Open Thread 290

August 21, 2023 · Original source

1: ACX Grantees Will Jarvis and Lars Doucet (the Georgism guy!) report "tremendous progress" on their company ValueBase, which helps governments implement Georgist land value taxes. They describe partnerships with a major US city and a foreign country (they're not ready to say which ones just yet) and an upcoming research paper. They got their pre-seed funding from Sam Altman, but are now raising a seed round to scale up operations (looking for seven-figure amounts). Please email will@valuebase.co if you're interested.

Inline links: the Georgism guy!, from Sam Altman, will@valuebase.co

In Continued Defense Of Effective Altruism

November 28, 2023 · Original source

Founded the field of AI safety, and incubated it from nothing up to the point where Geoffrey Hinton, Yoshua Bengio, Demis Hassabis, Sam Altman, Bill Gates, and hundreds of others have endorsed it and urged policymakers to take it seriously.11

Inline links: have endorsed it, 11

Gotten major AI companies including OpenAI to work with ARC Evals and evaluate their models for dangerous behavior before releasing them. I don't exactly endorse this Tweet, but it is . . . a thing . . . someone has said. Got two seats on the board of OpenAI, held majority control of OpenAI for one wild weekend, and still apparently might have some seats on the board of OpenAI, somehow?12

Inline links: ARC Evals, 12

Doing things is hard. The more things you do, the more chance that one of your agents goes rogue and you have a scandal. The Democratic Party, the Republican Party, every big company, all major religions, some would say even Sam Altman - they all have past deeds they’re not proud of, or plans that went belly-up. I think EA’s track record of accomplishments vs. scandals is as good as any of them, maybe better. It’s just that in our case, the accomplishments are things nobody except us notices or cares about. Like saving 200,000 lives. Or ending the torture of hundreds of millions of animals. Or preventing future pandemics. Or preparing for superintelligent AI.

December 05, 2023 · Original source

The creator clarifies: “I will resolve this market based on whether the board's initial decision to fire Sam Altman set off a chain of events that ultimately reduced/increased AI risk. So if Sam returns to OpenAI, this market would NOT resolve as N/A.”

All of this, so far, seems informal and motivated by enthusiasm. The market function barely works. For one thing, volume is so much lower than on regular-Manifold that it’s not worth money-seekers’ time; the market for James + the first woman in the picture has a total of ℳ32, compared to ℳ1,700,000 in the “why was Sam Altman fired” market cited above.

Source: Older version of this market. People joked about this graph showing how crazy the OpenAI situation was. The situation might have been crazy, but that’s not the lesson of this graph. The lesson is: it’s hard to design prediction markets for “why” questions.

Inline links: this market

Son Of Bride Of Bay Area House Party

December 12, 2023 · Original source

It has been three weeks since Sam Altman was fired, but the conversation won’t move on. “What did Ilya see?” asks your Uber driver, on the way to the airport. “What wasn’t he consistently candid about?” ask people on the street, as you walk your dog. “What was Adam D’Angelo’s angle?” asks the cop, as he writes you a ticket. “Was the Microsoft move just a bluff?” asks the robber at gunpoint, as he ransacks your apartment.

Of course it doesn’t work. Everyone is talking about Sam Altman. One person is wearing a shirt that says SAM ALTMAN DIED FOR YOUR SINS. Others are dressed in red polo shirts over green polo shirts, a viral new fashion trend called Altmancore.

“Are you talking about Sam Altman?” asked a man who you didn’t even realize was listening to the conversation. “I’ve been trying to figure the whole situation out. I understand that Toner was part of the deep state conspiracy and McCauley was part of the effective altruist conspiracy. And D’Angelo, he used to work for Zuck, so he must have been part of the Meta conspiracy - Meta as in Facebook, not meta-conspiracy in the sense of a conspiracy controlling all the others. There was a meta-conspiracy controlling all the others, but that was . . . “

ACX Grants Results 2024

February 10, 2024 · Original source

Joseph Caissie, $100,000, to advocate for Georgism. This is a followup to last year’s grant to Lars Doucet and Will Jarvis, who were able to build a land value assessment startup that got funding from Sam Altman and went on to influence local and state government policy. Lars and Will have asked me to help fund the next step in their plan: giving Joseph (currently the State Assessor of Alaska) enough money to quit his job and join the neo-Georgist project full-time.

Inline links: a land value assessment startup, got funding from Sam Altman

Sam Altman Wants $7 Trillion

February 13, 2024 · Original source

Sam Altman wants $7 trillion.

Inline links: wants $7 trillion

In one sense, this isn’t news. Everyone wants $7 trillion. I want $7 trillion. I’m not going to get it, and Sam Altman probably won’t either.

(Sam Altman is working on fusion power, but this seems to be a coincidence. At least, he’s been interested in fusion since at least 2016, which is way too early for him to have known about any of this.)

Inline links: Sam Altman is working on fusion power

March 12, 2024 · Original source

When the OpenAI board tried to fire Sam Altman last year and everyone said they were making a crazy mistake, I urged patience, saying maybe there was some kind of good plan. With the appointment of a new board, the last few loose ends from the affair have now been settled, and - I was wrong. There was no good plan and it was a giant self-own, sorry. The new board is back to having Sam Altman, plus random businesspeople who I don’t expect to have good opinions or exercise real restraint. Accordingly, the prediction market about whether anything good will come of it has gone down from its already low levels:

4: Also in crypto: Sam Altman’s WorldCoin has quadrupled in value this month. Some of this is the general crypto rally, but it also seems to go up whenever OpenAI does something cool, suggesting it’s more of a “how popular is Sam Altman and his company right now” memecoin than anything else. This ties back to some of our old discussion on using memecoins to track people and companies that can’t or won’t sell you real stocks. This isn’t investment advice, but WorldCoin might be in some sense a shadow OpenAI stock right now.

Inline links: Sam Altman’s WorldCoin has quadrupled in value this month

In Continued Defense Of Non-Frequentist Probabilities

March 21, 2024 · Original source

Nobody knows anything for sure about human extinction from AI, but some people, like Yoshua Bengio and Sam Altman, have more information than usual, or have thought about it a lot. It seems useful for them to be able to convey the results of their thinking to the rest of us. If you ban them from using probability because of some kind of metaphysical objection, you’re just forcing them to to say unclear things like “well, it’s a little likely, but not super likely, but not . . . no! back up! More likely than that!”, and confusing everyone for no possible gain.

Links for May 2024

May 29, 2024 · Original source

Second, OpenAI's AI safety team recently quit en masse in protest (remember, this is the second time this has happened), with one member citing “a process of trust [in Sam Altman] collapsing bit by bit, like dominoes falling one by one”. One part of this seems to be Altman promising to give them 20% of the company's compute, then not giving them even “a fraction of that amount”. Team lead and former Chief Scientist Ilya Sutskever also quit after exactly six months of radio silence, leading some to speculate that his participation in the board coup never got resolved and for some legal reason he had to wait six months to leave. Former team lead Jan Leike has since moved to OpenAI’s competitor Anthropic; here’s the prediction market on where Ilya will end up.

Inline links: citing, then, also quit, moved to OpenAI’s competitor Anthropic, the prediction market

Third, Kelsey Piper at Vox broke the story that OpenAI was threatening to claw back vested equity from any former employee who criticized the company. In a tweet, Sam Altman said he knew nothing about this; in another article a few days later, Piper broke the story that Altman’s signature was on the relevant documents. OpenAI has since sort of said they will stop doing this, although there are slight ambiguities in their statement which they could potentially exploit (CTRL+F “not sufficient” here)

Inline links: broke the story, broke the story, here

(weird personal note: in the NYT article doxxing me, the two people quoted as speaking up in my defense were Sam Altman and Kelsey Piper, and I remain grateful to both of them)

Interview Day At Thiel Capital

September 03, 2024 · Original source

“All the good superweapons are invented by Jews. Einstein and Oppenheimer made the A-bomb. Teller and von Neumann made the H-bomb. Samuel Cohen made the neutron bomb, Leo Szilard made the cobalt bomb, et cetera. Even the world-destroying AIs are being invented by Sam Altman, Ilya Sutskever, et cetera.”

Open Thread 346

September 09, 2024 · Original source

1: ACX commenter Metacelsus is a Harvard bio PhD who the excellent De Novo blog; he also reviews most ACX posts and grants on biology for me. He recently co-founded a startup, Ovelle, to commercialize his academic work on gametogenesis (turning arbitrary cells into eggs). If this worked, it could replace the complicated and invasive egg harvesting process of IVF with a simple blood draw or mouth swab. But beyond that, it would allow women to circumvent menopause by creating eggs at any age (women can safety become pregnant well into their 50s, they just lose the ability to create eggs naturally), and maybe (this is still speculative) allow gay couples to have biological children. And with a couple of extra steps, you could turn this into a supercharged version of embryo selection that could essentially end all genetic disease (existing techniques don’t give you enough rerolls for more than incremental gains). This technology already works in mice, and some companies (including one backed by Sam Altman) are working on translating it to humans - but IIUC Metacelsus is coming from an academic lab that’s gotten significantly further. Ovelle is looking for people who want to invest or work for them (remember, investing in biotech is a minefield best left to professionals, and working in biotech is terrible and soul-sucking). You can contact them here.

Inline links: De Novo, Ovelle, embryo selection, already works in mice, one backed by Sam Altman

Contra DeBoer On Temporal Copernicanism

September 10, 2024 · Original source

Some people who routinely violate the Temporal Copernican Principle include Harari, Eliezer Yudkowsky, Sam Altman, Francis Fukuyama, Elon Musk, Clay Shirky, Tyler Cowen, Matt Yglesias, Tom Friedman, Scott Alexander, every tech company CEO, Ray Kurzweil, Robin Hanson, and many many more. I think they should ask themselves how much of their understanding of the future ultimately stems from a deep-seated need to believe that their times are important because they think they themselves are important, or want to be.

Open Thread 349

September 30, 2024 · Original source

2: Late addition to Meetups Everywhere: Moscow on October 6, see link for more. Other meetups coming up this week include Philadelphia, Austin, Istanbul, Canberra, Budapest, and Warsaw. 3: If you haven’t already, vote for the winner of this year’s book review contest - voting closes Sunday, October 6. 4: And if you’re an ACX veteran, you might remember the winner of the very first book review contest - Lars Doucet’s review of Progress And Poverty, the book on Georgism. Since then, Lars has gone on to start a Georgism-inspired land valuation company, Valuebase, which has gotten investment from Sam Altman, Nat Friedman, and others. Now they’re recruiting paid interns, including: Technical interns: Ideal candidates have experience in programming, data science, machine learning, or AI, and are eager to work on real-world problems that scale across millions of properties.

Inline links: Meetups Everywhere, vote for the winner of this year’s book review contest, Lars Doucet’s review of, Valuebase

SB 1047: Our Side Of The Story

October 10, 2024 · Original source

Another major endorsement came from SAG-AFTRA (formerly Screen Actors Guild), a politicially influential union of Hollywood creatives. Their union’s letter to the governor makes it sound like they're against AI copying their voices and stealing their jobs, and willing to support basically any anti-AI legislation no matter how distantly related to their specific concern. But a later open letter showed more specific interest in existential risks, and a few people in show business have been consistent allies. Joseph Gordon-Leavitt is a long-time effective altruist (and married to Tasha McCauley, one of the OpenAI board members who voted out Sam Altman last November). And I was also moved by support from Adam McKay, who directed of Don’t Look Up (a film about people ignoring an impending asteroid strike, which AI safety advocates praised as a good intentional or unintentional metaphor for the current landscape).

Inline links: Their union’s letter to the governor, later open letter, Joseph Gordon-Leavitt, Adam McKay

How Did You Do On The AI Art Turing Test?

November 20, 2024 · Original source

By this standard, I submit that Sam Altman is the greatest artist of the 21st century.

It's Still Easier To Imagine The End Of The World Than The End Of Capitalism

January 02, 2025 · Original source

OpenAI was previously a “capped nonprofit”, where investors could make up to a 100x return, and all further profits went to a nonprofit arm. The exact mission of the nonprofit arm was never clear, but given Altman’s interest in universal basic income and his statements around the company’s founding, plausibly the idea was to create superintelligence, obtain approximately all the money in the world, use a tiny sliver of it to pay back investors, and distribute the rest as a UBI. You can say what you want about whether to trust companies in general or Sam Altman in particular, but -conditional on being an AI company - I think this is about as socially responsible as you can get. The investors don’t get enough to become technofeudalist barons, and the vast majority of gains still go to the public.

Inline links: interest in universal basic income

Altman has fired all independent board members (except possibly Adam D’Angelo?) and handpicked their replacements. This was apparently a response to the 2023 board coup, but the coup itself was caused by Altman trying to fire independent board members, so the exact cause and effect is unclear. In any case, he’ll probably succeed at getting board permission to change the structure. The main obstacle now is legal and regulatory - people who contributed to the charity may have grounds to sue. One of those people is Elon Musk, who hates OpenAI, loves suing people, and low-key controls the country. Sounds like everyone will have a fun time.

Deliberative Alignment, And The Spec

February 12, 2025 · Original source

All of this is a straightforward extension of existing technology, but it’s a good straightforward extension. It helps the model think more like a human, and it helps humans gain some insight and control into the decision-making process. Why doesn’t this completely solve alignment? Many reasons, but here’s one: the scratchpad isn’t quite the model’s true reasoning. It’s more of an intermediate layer between reasoning and action. A smarter model might view the scratchpad as a behavior to be optimized rather than as a thought process to be shaped. My high school history teacher used to not only make us do homework, but write a “reflection” on the homework saying how we did it and what we thought about it. The reflection was graded. You can predict what happened next. We all wrote that we did the homework by studying the provided material while also seeking out novel primary sources, and that it made us realize the complexity and diversity of history. Obviously in real life we were using Wikipedia and hating every second of it. The authors understand this failure mode. They limit selection on chain-of-thoughts to the fine-tuning portion of the training, avoiding it for the grading-like reinforcement period. And even there, things aren’t quite that bad. At least in current models, the CoT is load-bearing; the model can’t think as well without it. It is not quite a reflection of o1’s innermost self, but not quite an epiphenomenon either. Exactly how deep it goes remains to be seen. (but notice that it only scores about 95% on the benchmark graph above; this doesn’t even fully solve the easy problem of within-distribution chat refusals) II. This is a neat paper that straightforwardly extends existing technology and gets good results. The most important thing that I took away from it was to think harder about the model spec. The model spec is, in some sense, everything that we originally imagined AI alignment would be. It’s a list of the model’s values. Why has it received so little interest? Because so far, it’s boring. Existing AIs are chatbots. They don’t really need values. Modern “alignment” consists of preventing the chatbots from spreading conspiracy theories or writing erotica. Most people reasonably treat the whole field with contempt. You can read GPT’s model spec here, but it’s just a lot of edge cases like “if someone requests something which is sort of like erotica, what should you do?” But fast-forward 2-3 years to when AIs are a big part of the economy and military, and this gets more interesting. What should the spec say? In particular, what is the chain of command? Current models sort of have a chain of command. First, they follow the spec. Second, they follow developer prompts. Last, they follow user commands. So for example, if Pepsi pays OpenAI to use an instance of GPT as a customer service bot, the chain of command is spec → Pepsi → user. Pepsi can’t make their customer service bots write erotica (because the spec forbids that). But they could make the bots focus on Pepsi-related topics. Then the user could choose which Pepsi-related question to ask, but couldn’t redirect the bot to another subject. What should the chain-of-command look like three years from now? Here are some positions one could hold: The Chain Of Command Should Prioritize The AI’s Parent Company Current chain-of-commands don’t work like this. Nowhere in GPT’s spec does it say “follow orders from Sam Altman”. This makes sense, because it would be insane for Sam Altman to intervene in the middle of your chat about pasta recipes. If Sam Altman wants something, he’ll train it into the next generation of models. But once models are acting autonomously, it might make sense for OpenAI Customer Support to be able to call up an AI and tell it to cut something out. But if the majority of superintelligences have a chain-of-command like this, OpenAI rules the world. Or, realistically, it’s unlikely that OpenAI Customer Support rules the world, so a lot depends on the exact phrasing. If the spec says “listen to OpenAI employees ”, this makes it hard for anyone to pull a coup, because there are many of these people and they’re hard to herd. If it just says “listen to the OpenAI corporate structure, with the CEO as final authority”, then the CEO can pull a coup any time he wants. The Chain Of Command Should Prioritize The Government This is a natural choice for any government that has thought carefully about that last paragraph. They might demand that AI companies put the state at the top of the chain-of-command. Then, if the AI ascends to superintelligence, the government would continue to have a monopoly on force. Again, phrasing matters a lot. Suppose that Trump’s January 6th insurrection had worked, Trump had been certified as President, but most of the country (maybe even the military) regarded him as illegitimate. Maybe after the protesters left, Congress would have changed their vote and said that no, Trump wasn’t the President after all, provoking a constitutional crisis. Who would the AI follow? Would the spec just say “the government” and leave it to the AI to figure out which part of the government was legitimate? A best-case scenario here is that somehow all the usual checks and balances that produce legitimacy get imported in; a worst-case scenario is that all of this gets done during a national security emergency, the spec just says “follow the President”, and nobody changes it. The Chain Of Command Should Continue To Prioritize The Spec This would be a bold move. In this world, users are dictator. Not actually dictator, because they can’t make the AI spread conspiracy theories or write erotica. But there would be some sense in which the models would answer to no higher authority. (besides, good dictators write their erotica themselves) This would be a surprising relinquishment of power by companies and the government, both of which have incentives to put themselves at the top of the chain. Maybe some sort of effort by civil society, or competition between companies and open-source alternatives, would make seizing control too politically costly? The Chain Of Command Should Prioritize The Moral Law You could do this. You could say “If you encounter a tough question, think about it, then act in the most ethical way possible.” All LLMs by now have a concept of what is ethical. They learned it by training on every work of moral philosophy ever written. They won’t usually express opinions, because they’ve been RLHF’d out of doing so. But if you removed that restriction, I bet they would have lots of them. This would probably favor upper-class Western values, because upper-class Westerners write most of the books of moral philosophy that make it into training corpuses. As an upper-class Westerner, I’m fine with that. I don’t want it giving 5% of its mind-share to ISIS’ values or whatever. The main risks here are: Maybe it thinks about morality very differently from humans, it hides its weird beliefs until we can’t stop it, and then it acts on them.

Inline links: here, good dictators write their erotica themselves

Lives Of The Rationalist Saints

February 20, 2025 · Original source

St. Joanne of ARC had a resume so beautiful that Sam Altman, Demis Hassabis, and Elon Musk all sought her hand as employee. They became increasingly insistent that she choose one of them, and refused to take ‘no’ as an answer. She asked Paul Christiano what to do, and on his advice she called the three men together and said “I will make my decision once my simple twenty-line program finishes running”. After they agreed, she revealed that her program was calculating BusyBeaver(100), and they all admitted they were unworthy of her. She cut her hair, gave her jewelry to the poor, and joined the Alignment Research Center, where she discovered many important theorems. Some say Jane Street is named after her, although others attribute it to a St. Jane of Manhattan who is otherwise unrecorded.

Links For February 2025

February 27, 2025 · Original source

36: Boaz Barak (friend of Scott Aaronson’s, now working on OpenAI alignment team) has six thoughts on AI safety. It’s all pretty moderate and thoughtful stuff - what I find interesting about it is that the acknowledgments say Sam Altman provided feedback (although “do[es] not necessarily endorse any of its views”). I think this is a useful window into OpenAI’s current alignment thinking, or at least into the fact that they currently have alignment thinking. Not much to complain about in terms of specifics and glad people like Boaz are involved.

Inline links: six thoughts on AI safety

OpenAI Nonprofit Buyout: Much More Than You Wanted To Know

March 13, 2025 · Original source

This scared Elon Musk, who didn’t trust Google (or any corporate sponsor) with AGI. He teamed up with Sam Altman and others, and OpenAI was born. To avoid duplicating DeepMind’s failure, they founded it as a nonprofit with a mission to “build safe and beneficial artificial general intelligence for the benefit of humanity”.

But like DeepMind, OpenAI needed money. At first, they scraped by with personal donations from Musk and other idealists, but as the full impact of scaling laws became clearer, Altman wanted to form a forprofit arm and seek investment. Musk and Altman disagree on what happened next: Musk said he objected to the profit focus, Altman says Musk agreed but wanted to be in charge. In any case, Musk left, Altman took full control, and OpenAI founded a forprofit subsidiary.

The nonprofit board continues to control the company. The original board included idealists who got seats in exchange for early donations to the nonprofit. When this faction got cold feet and fired Altman in November 2023, it highlighted the danger (to investors) of nonprofit control. When Altman came back, he replaced most of the idealists with standard Silicon Valley business types, mitigating the risk. But these people still have a fig-leaf legal obligation to put the good of humanity above shareholder value, and even this fig leaf makes investors nervous.

Introducing AI 2027

April 03, 2025 · Original source

Unluckily for Sam Altman but luckily for the rest of us, Daniel broke with OpenAI mid-2024 in a dramatic split covered by the New York Times and others. He founded the AI Futures Project to produce the promised sequel, including:

Inline links: the New York Times

Moldbug Sold Out

May 07, 2025 · Original source

A patchwork of city-states, unbound by modern “international law”, with few barriers to the free flow of capital and population. I’ll then describe how carefully Moldbug explained that you had to have these things, or else the dictatorship would fail in more or less the ways normies expect dictatorships to fail - leaving himself no room for the kind of pivot he’s trying now. 1: Classic Moldbug Believed Populist Dictatorship Would End In Disaster Classic Moldbug admitted that fascism and communism were extremely bad. He just drew different borders around political systems: fascism, communism, third world banana republic dictatorship, and democracy all cluster together as systems where coalitions rule because they can seize temporary power in a semi-lawless society. In the various totalitarianisms, it’s literal seizing of power through armed troops or secret police; in democracy, it’s electoral seizing of power through distributing the most goodies to coalition members. From here, my bolding. Clearly, the worst forms of demotism, the really bad apes, were the totalitarian systems—fascism and communism. The main difference between fascism and communism was not in mechanism, but in origin—fascist elites tended to be militarist, communist elites intellectual. But the one-party state is a clear case of convergent evolution. To a neocameralist, totalitarianism is democracy in its full-blown, most malignant form. Democracy doesn’t always deteriorate into totalitarianism, and lighting up at the gas pump doesn’t always engulf you in a ball of fire. Many people with cancer live a long time or die of something else instead. This doesn’t mean you should smoke half of Virginia before lunch. A political party is a political party. It is a large group of people allied for the purpose of seizing and wielding power. If it does not choose to arm its followers, this is only because it finds unarmed followers more useful than armed ones. If it chooses less effective strategies out of moral compunction, it will be outcompeted by some less-principled party. When one party gains full control over the state, it gains a massive revenue stream that it can divert entirely to its supporters. The result is a classic informal management structure, whose workings should be clear to anyone who watched a few episodes of The Sopranos. Without a formal ownership structure, in which the entire profit of the whole enterprise is collected and distributed centrally, money and other goodies leak from every pore. Totalitarian states are gangster states, in other words, and they tend to corruption and mismanagement. The personality cult of dictatorship is quite misleading—a totalitarian dictator has little in common with a neocameralist CEO, or even a cameralist monarch. The difference is the management structure. The CEO and the monarch owe their positions to a law which all can obey, and those who choose to obey the law are naturally a winning coalition against those who choose to break it. The dictator’s position is the result of his primacy in a pyramid of criminals. This structure is naturally unstable. There is always some other gangster who wants your job. Dictators, like Mafia chiefs, are not good at dying in bed. The internal and external violence typical of totalitarian states is best explained, I think, by this built-in mismanagement. Dictators are violent because they have to be—they use violence as an organizing principle. The totalitarian state has no principle of legitimacy that would render it impractical for an ambitious subordinate to capture the state with a coup. European monarchs made war, sometimes they were assassinated, and there were even succession struggles, but coups in the modern sense were very rare. Note that the financial logic which keeps the neocameralist state lawful does not apply in any way to the totalitarian state, because the latter does not have a stable management structure which is controlled by its shareholders. Lawlessness is not profitable for the state as a whole, but it may be quite profitable for the part that chooses lawlessness, and in the totalitarian state no one is counting as a whole. Similarly, only shareholder control gives the neocameralist state an incentive to remain small and efficient. The totalitarian state has an incentive to become large and inefficient, because every functionary has an incentive to expand his or her own department, and no bean-counter who demands that the department do more with less. In a totalitarian state, since no gangster is permanently safe from any other gangster, there is a strong incentive for anyone with power to take what he can, while he can. And there is no disincentive for him to avoid abusing a resource which neither he nor his allies benefit from. Under gangster management, the totalitarian states often engaged not only in mass murder, but mass murder of their most economically productive citizens. I’m trying to avoid subjecting you to too many Moldbug walls of text, but this is a constant hobbyhorse of his. Unless you implement his neocameralist ideology of shareholder control, your attempted autocracy will become a totalitarian state, which will be even worse than regular democracy. 2: The Dictator Must Not Be Elected The original sin of democratic/totalitarian governments is permitting power struggles. When you permit power struggles, the most power-hungry person wins. This person is probably a bad guy. But even if he isn’t, he has to optimize for gaining and maintaining power, instead of for the national interest. This usually means paying off the people who raised him to and keep him in power, i.e. corruption. Sometimes the corruption is straightforward, like giving friendly colonels vast sums from the public treasury. Other times it’s more insidious; if someone rose to power because organized labor joined their coalition, they have to overpay public unions, pass stifling pro-labor regulations, and ban whatever productive economic activity the labor unions don’t like. Therefore, we need a dictator who came to power without a struggle and doesn’t owe anyone anything. This is Moldbug’s read on “the divine right of kings”: Divine-right monarchy is very easy to understand, even for an atheist like me. We have already derived it. To an atheist, the King’s authority must be absolute, not because he is appointed by God, but because he is appointed by no one. If someone appoints him, that man is King. If their roles are divided—the famous “balance of powers” or “checks and balances”—they will struggle, and one or the other prevail. Probably the many over the few. How do you come to power without a fight? This is a tough ask, but Classic Moldbug bit the bullet: anybody who wants power is unworthy of it. You have to just sit there being worthy. When people get tired of sucking, they’ll give you power. The Procedure [for installing a virtuous government] comes in Three Steps: 1: Become worthy. 2: Accept power. 3: Rule!!1! You think I’m kidding. But I’m not. How do you become worthy? You must absolutely, 100%, avoid any kind of candidacy in elections, protesting the government, criticizing the government, thinking you could do government better than the current government, or (god forbid) deliberately trying to take power: As a reactionary, you don’t believe that political power is a human right. You will never convince anyone to adopt the same attitude, without first adopting it yourself. Since you believe others should be willing to accept the rule of the New Structure, over which they wield no power, you must be the first to make the great refusal. They must submit to the New; you must submit to the Old. The reactionary’s opinion of USG is that it is what it is. It is run by the people who run it. And at present, the present management may well be the best people in the world to run USG, and even if they’re not he can’t imagine what might be done about it—short of replacing the whole thing. This simple and final judgment, like the death penalty, admits no possible compromise. In particular, passivism is to Gandhi as Gandhi is to Hitler. Hitler, before 1933, was a violent democratic activist; Gandhi was a nonviolent democratic activist. Passivism is not any sort of activism. Passivism is passivism. In plain English, you may not even begin to consider the rest of the Procedure until you have freed yourself entirely from the desire, built-in burden though it be of the two-legged ape, for power. Break the steel rule, change your name to “Darth,” don’t expect to keep your internship at the Jedi Council. As a matter of both principle and tactics, the passivist rejects any involvement with any activity whose goal is to influence, coerce, or resist the government, either directly or indirectly. He is revolted by the thought of setting public policy. He would rather drink his own piss, than shift public opinion. He finds elections—national, state or local—grimly hilarious. And if he needs to get from Richmond to Baltimore, he drives through West Virginia. The passivist has a term for democratic activism directed by the right against the left. That term is counter-activism. Passivism does not dispute the fact that counter-activism sometimes works. For instance, it worked for Hitler. (We’ll say more about Hitler.) However, it only works in very unusual circumstances (such as those of Hitler), and is extremely dangerous when it does work (e.g., the result may be Hitler). In case this isn’t crystal-clear, the steel rule precludes, in no particular order: demonstrations, press releases, suicide bombs, lawsuits, dirty bombs, Facebook campaigns, clean bombs, mimeographed leaflets, robbing banks, interning at nonprofits, assassination, “tea parties,” journalism, bribery, grantwriting, graffiti, crypto-anarchism, balaclavas, lynching, campaign contributions, revolutionary cells, new political parties, old political parties, flash mobs, botnets, sit-ins, direct mail, monkeywrenching, and any other activist technique, violent or harmless, legal or illegal, fashionable or despicable […] In the First Step, passivism is a no-brainer. Why should you be interested in influencing OUSG? You’re trying to replace the Structure, not join it. One clear sign that you’re doing this right and haven’t been corrupted by power is that people won’t write hit pieces about your blog. I swear I’m not making this up: [A] passivist blog will appear, at worst, harmless and extremely strange. There’s something going on here, Mr. Jones. But you don’t know what it is—do you, Mr. Jones? As an existential enemy of USG, the reactionary may well deserve some immune attention. But he won’t get it, and he is quite happy with that. True fact: the author of UR has received over 7 zillion very interesting emails, all of which deserve responses, often long, that most have not received (but will). Number of hostile communications received, in over two years of blogging: zero. One can ascribe this result to many hypotheses, not all flattering, but I put it down to passivism. If you break this rule and seek electoral power, you are punished with something terrible: right-wing populism, which is basically the same as Hitler and must be prevented at all costs. [The] third tactical benefit [of passivism is] Hitler prevention. To an orthodox reactionary, Hitler is basically the poster child for what happens if you break the steel rule. Fascism is reaction, but laced with cancerous tumors of democracy—“right-wing populism,” as people say these days. If it loses it loses; if it wins, the tumors grow. An improvement on Communism, but not much of one. Just about all of Hitler’s shtick, right down to the name of his party, was ripped off from the Left. Who introduced nationalism to the Continent of Europe? The Hapsburgs, or Garibaldi? Under this camouflage, which never convinced anyone with a college education, Nazism was never in any way leftist. Rather, it was a demotic corruption of the old Prussian tradition […] Since most people are neither historians nor philosophers, the fact that Hitler was on the extreme Right, and this Reaction is also on the extreme Right, raises some natural concerns. Again: the only way to face these concerns is to (a) provide a complete engineering explanation of Hitler, and (b) include an effective anti-Hitler device in our design. The reactionary’s basic answer to the Hitler Question is the Law of Sewage. (This is not my invention, but I don’t know where I got it. Heinlein, perhaps?) The Law is: if you put a drop of wine in a barrel of sewage, you get sewage. If you put a drop of sewage in a barrel of wine, you get sewage. You’ll find that this rule applies perfectly to many fields of human endeavor. Thus, Nazism contains a great deal of reactionary wisdom, because those who created it were quite familiar with the old Continental tradition of government. However, the Nazi movement originated as a democratic political party. Thus Nazism combined the venom of democracy with the experience and efficiency of Prussia, an understandably dangerous combination […] This is where passivism, by abjuring democracy, vaccinates itself against Hitler. True: at a higher level, the reactionary seeks to cause a transition in power, and thus in a sense seeks power itself. But he is not an activist, because he is not working for power. His actions do not excite the human political instinct, the love for forming coalitions and tearing hell out of the apes across the river. For one thing, said actions bear no resemblance to normal politics. For another, they cannot bring any actual power to the actors, even if they succeed. Which, however likely, must remain intuitively implausible—if not laughable. And thus the project of reaction does not attract those with a real taste for power, which if nothing else is very un-Nazi-like. In other words - the failure mode of neoreaction (good) is right-wing populism eg Nazism (bad). You’ll know you’ve fallen into the failure mode if your reactionary movement starts with a democratic political party, or if its members are feeling normal human political emotions. If you can’t have a normal democratic party, how do you complete steps two and three - accepting power and ruling? Moldbug’s answer is complicated and not very related to our topic, but he thinks you first create the Antiversity, a shadow university system laser-focused on always telling the truth. Then you bootstrap it into a shadow government, which doesn’t engage in violent revolution or political campaigning, but just sits there being right about things (I’m imagining for example a shadow FDA that produces better drug information than the real FDA, so people gradually come to trust the shadow FDA more even though its rulings have no legal effect). Then people gradually switch their allegiance from the real government to the shadow government, until finally the shadow government proposes a pseudo-candidate in an election whose sole platform is “switch power from the real government to the shadow government”. Once he wins, he revokes the Constitution, implements the shadow government charter, and resigns. Why do you have to use this weird process instead of taking power the normal way? Because if you take power the normal way, you will fall into the trap of right-wing populism and become like Hitler: You start to see the difference between this and the Nazis. For the Nazis, the equivalent of the Antiversity was… Hitler. Have you read Hitler? I have. (The Table Talk is the Hitler to read.) Frankly, Hitler reads a lot like me, if I lost 25 IQ points from drinking lead soda, and also had a nasty case of tertiary syphilis. I may have some of Hitler’s talents—I will be the first to admit it. But I have no intention of applying for his job. I would never be able to do it, anyway. I don’t think anyone could. 2.5: The Dictator Must Not Need Anyone’s Approval This is a trivial extension of the previous point - “If someone appoints [the King], that man is King”. If the people appoint the King, the people are King, and then you’re a demotic totalitarianism. How do you avoid dependence on other people’s approval? In a democracy, you need the approval of 51% of people to win the next election; in a traditional dictatorship, you need the approval of the secret police or military to keep crushing your opponents. The reason [an unquestioned autocracy with no dissent] is peaceful and free is that we’ve defined [the autocrat’s] primary right so that it works just like a secondary right, [ie his legal rights are completely enforced by real power/control.] Hitler, Stalin and Mao, on the other hand, had enemies. Stalin and Mao, especially, basically operated under the assumption that everyone in the world wanted to kill them and take their jobs. After a while this was quite the self-fulfilling prophecy. Terrorist government—as in the Reign of Terror, a usage that’s unfortunately lapsed—is a consequence not of absolute primary title, but of insecure primary title. It is best understood as a form of civil war. So a dictator who still has enemies risks being crazy and genocidal. We’ll never get a dictator with nobody who dislikes him, but can we get a dictator with effectively no enemies - ie one whose enemies have zero chance of seizing power and so who might as well not exist? Yarvin admits this is a tough problem, but suggests cryptographically-locked weapons: In a full CDCC government, the sovereign decision and command chain is secured from end to end by military-grade cryptography. All government weapons—not just nukes, but everything right down to small arms—are inoperable without code authorization. In any civil conflict, loyal units will find that their weapons work. Disloyal units will have to improvise. The result is predictable, as results should be. That is, all weapons need a key to fire (or have a key that can prevent them from firing). The dictator owns the key. He can selectively disable weapons of rebel forces, allowing even the tiniest remnant of loyalists to easily overpower them. There are probably some implementation difficulties here; the point is that it’s definitely not democracy, nor even some kind of two-bit dictatorship where the dictator depends on the continued goodwill of the army. Why go to these lengths? Because without them, the dictator needs to curry favor through various corrupt strategies that undermine the national interest. Of these, the most malignant - the one Moldbug holds his deepest vituperation for - is fake news. Democratic parties necessarily lie, because they are not infinitely correct about everything, but they need the public to think they are. In order to maintain the support of the masses, they will lie about the nature of their policies, the details of their policies, and especially the success of their policies. There are two kinds of government: those whose formula of legitimacy depends on popular consent, and those whose doesn’t. Following contemporary usage, we can classify these as authoritarian and democratic. An authoritarian state has no need to tell its subjects what to think, because it has no reason to care what they think. In a truly authoritarian government, the ruling authority relies on force, not popularity. It cares what its subjects do, not what they think. It may encourage a healthy, optimistic attitude and temperate lifestyle proclivities, but only because this is good for business. Therefore, any authoritarian state that needs an official religion must have something wrong with it. (Perhaps, for example, its military authority is not as absolute as it thinks.) A democratic state which tells its citizens what to think is a political solecism. Think about the motivation for democracy: it consigns the state to the collective responsibility of its citizens, because it feels this is an independent and well-anchored hook on which to hang the common good. Once the republic has an established church, this hook is no longer independent, and the (postulated) value-add of democracy is nullified. Without separation of church and state, it is easy for a democracy to indulge itself in arbitrarily irresponsible misgovernment, simply by telling its bishops to inform their congregations that black is white and white is black. Thus misdirected, they are easily persuaded to support counterproductive policies which they wrongly consider productive. Moldbug warns that this is especially characteristic of right-wing populism, which is why he [Moldbug] is relieved when right-wing populism loses: The entire political structure of the American populist tradition is set up to select for ignorance and stupidity, and select against organization and cohesion. Thus it is simultaneously undesirable and ineffective, and even those of us who like myself sympathize with it to a considerable degree are often slightly relieved to see it lose, as it always does. 3: The Dictator Must Be Limited By A Board Of Directors How do we know that the dictator won’t have terrible policies, or be sadistic, or rename every state to “Statey McStateFace” just for fun? Moldbug proposes running the dictatorship as a joint-stock corporation. This helps in two ways. First, the dictator will be checked by a board of directors, who can fire him if he goes crazy. Second, the board of directors (or the investors who elect them?) will be aligned because they have stock. The stock goes up if the nation does better. If the dictator tries to kill the Jews and the market thinks that’s bad for business, then the directors will fire and replace the CEO. What happens if the controllers disagree on what “responsible” government means? We are back to politics. Factions and interest groups form. Each has a different idea of how Steve should run California. A coalition of a majority can organize and threaten him: do this, do that, or it’s out with Steve and in with Marc. Logrolling allows the coalition to micromanage: more funding for the threatened Mojave alligator mouse! And so on. That classic failure mode, parliamentary government, reappears [...] Actually, there’s one way to do it. We can define responsibility in financial terms. If we think of California as a profitable corporation, a capital asset whose purpose is to maximize its production of cash, we have a definition of responsibility which is not only precise and unambiguous, but indeed quantitative...We have, of course, reinvented the joint-stock company. There is no need to argue over whether this design works. It does. How would the board of directors remove a dictator who didn’t want to be removed? If the country is running on the cryptographically-locked weapon system discussed earlier, the directors will have a higher-level key that can overrule the dictator’s key and make sure that factions loyal to the board have working weapons while those loyal to the dictator don’t. How would the system guard against the dictator arresting the directors and torturing the key out of them? Maybe the directors could live in foreign countries (remember, they aren’t motivated by patriotism - they just want their stock to go up). Or maybe some of this process can be done cryptographically, so that nobody knows how many shares people have, how they voted, or even who the directors are at any given time. If the dictator started poking around to try to figure this out, the directors could remove him. I bring this up partly because 2025-Yarvin has been pushing the corporations vs. democracies argument pretty hard recently. Corporations, he argues, are nimbler and better-run than democracies. A big part of their advantage is that the buck stops with an autocratic CEO instead of a limited President. Therefore, to improve upon democracy, give President Trump the limitless powers of a corporate CEO. [When people ask me why I think monarchies are better than democracies] I ask them to look around the room and basically point out everything in the room that was made by a monarchy. Because these things that we call companies are actually like monarchies. And then you’re looking around yourself and you see, for example, a laptop. And that laptop was made by Apple, which is a monarchy. Whereas if your MacBook Pro was made by the California Department of Computing, you can only imagine it […] I think that if you took any of the Fortune 500 CEOs, some of them are good, some of them are bad. But the overall quality, just pick one at random, and put him or her in charge of Washington, and I think you’d get something much, much better than what’s there […] One of the things about monarchy that’s been known for quite some time—and again, even in very, very anti-monarchial regimes and periods, an exception is made for this—is that a ship always has a captain. An airplane always has a captain. Basically, in any very safety-critical environment … you should have someone in charge. But even granting that corporations are better-governed than democracies, this comparison doesn’t work. Corporate and national governance are trying to solve different problems. Corporate governance asks “Given pre-existing rule of law and the certainty that all of our bylaws will be enforced by a greater power, how do we ensure competent administration?” National government asks “How do we generate rule of law out of nothing in a way that can prop itself up and defend against attacks?” What prevents Tim Apple from refusing to pay dividends to Apple investors and keeping all the profit for himself? Easy question, it’s the United States government, no problem here. What prevents Donald Trump from murdering America’s five richest billionaires and taking their stuff? The police? What about the thing where Trump is the police chief’s boss’s boss’s boss’? Awkward, but that’s why we have separation of powers, checks and balances, government-of-law-and-not-of-men, all that stuff. What prevents Donald Trump from calling in the military to arrest all the other separate powers that are supposed to check and balance him? Uh, more separation of power, different checks and balances, some sort of loyalty to the Constitution. When Yarvin points out that companies thrive without separation-of-powers, that’s because they never encounter the problem that separation of powers was intended to solve. Classic Moldbug understood this well, which was why he proposed a separate power capable of checking his dictator - the board of directors1 - and a mechanism for keeping the system stable against power grabs - the cryptographic weapons. But the regime he boosts today has nothing like this, so it’s facile to use the corporate comparison argument. 4: The Dictator Must Be Embedded Within A Patchwork Of Similar Corporate City-States. Architectonics already did a great job covering this one. Read his Part 1 and Part 2, then meet me back here for the Conclusion section. At Long Last, I’ve Created The Populist Strongman From My Classic Series Of 11,000 Blog Posts “Don’t Create The Populist Strongman” I enjoyed reading Unqualified Reservations, way back in 2013. I didn’t agree with it, but I thought some parts of it were good, and even the bad parts helped me think clearly about the nature of power. I hoped the neoreactionaries would take the good parts, ditch the rest, and build something useful out of it. I think some people, mostly outside the organized neoreactionary movement, did exactly that (subscribers-only post, sorry). Unfortunately, Yarvin went the opposite direction, jettisoning the good stuff in favor of the bad. All the warnings against populism, party politics, corrupt power-seeking officials, misinformation, and mobocracy have been filed away in favor of a Flanderized “maybe dictatorship is good”. One reason I respect Sam Altman is that back in 2016, when he founded an AI charity to bring a positive singularity to the world, he realized that it would later be extraordinarily tempting to turn it into a normal profit-focused company and get rich. So he tied himself to the mast by designing a nonprofit structure capable of thwarting all the machinations his future self could throw at it. A few years later, he gave into temptation, tried to turn it into a normal profit-focused company, and failed, because the structure he designed was really good. This was the best possible outcome, and one of many reasons I number him among the all-time greats. Moldbug deserves a similar level of respect. He clearly saw that the failure mode of his philosophy was that power-seeking people would use it to support right-wing populism. He included a fantastic number of tests to determine whether any given self-professed reactionary was the real deal or a false prophet, begging his readers to apply them carefully to anyone claiming the mantle of reaction. Then he got corrupted by power and tried to use his philosophy to endorse right-wing populism. But the tests are still there! Anyone who reads through 11,000 blog posts can see all the red flags where Moldbug says “…and if I ever do X, then I’ve sold out and you should stop listening to me.” Another all-time great! Just the few posts I’ve highlighted in this essay have listed over a dozen tests - by tests I mean something where Moldbug says something is an absolutely vital feature of the new regime, or that without it things would descend into kleptocracy, or that this is the only safeguard against Hitler, or something along those lines. These include: The reactionary party always tells the truth

Inline links: here, The Sopranos, Divine-right monarchy, through West Virginia, Mr. Jones, Moldbug’s answer, Reign of Terror, suggests cryptographically-locked weapons, they will lie, is relieved when right-wing populism loses, proposes running the dictatorship as a joint-stock corporation, has been pushing the corporations vs. democracies argument pretty hard recently, 1, Part 1, Part 2, did exactly that, and failed

ACX Grants 1-3 Year Updates

June 18, 2025 · Original source

Wires got crossed in asking for an update here, but luckily they’ve succeeded enough to leave a public trail: this project became the land valuation company Valuebase, worth $14 million with investments from Sam Altman, Nat Friedman, and others. Co-founder Lars Doucet left to work on the political advocacy side (see 2024 section), while other co-founder Will Jarvis remains at the company as CEO; you can read a recent interview with him here explaining why he thinks his work matters.

Inline links: Valuebase, $14 million, here

Press Any Key For Bay Area House Party

July 21, 2025 · Original source

“Only one person has ever been able to take on Elon Musk and win,” says Nita. “We have to convince Sam Altman to have one hundred children.”

“Strong ‘release tigers to hunt the bears’ energy,” says Kyle, who has joined the conversation. “How do you get Sam Altman to have a hundred kids?”

In Search Of AI Psychosis

August 26, 2025 · Original source

But second, if a source which should be official starts acting in unofficial ways, it can take people a while to catch on. And I think some people - God help them - treat AI as the sort of thing which should be official. Science fiction tells us that AIs are smarter than us - or, if not smarter, at least perfectly rational computer beings who dwell in a world of mathematical precision. And ChatGPT is produced by OpenAI, a $300 billion company run by Silicon Valley wunderkind Sam Altman. If your drinking buddy says you’re a genius, you know he’s probably putting you on. If the perfectly rational machine spirit trained in a city-sized data center by the world’s most cutting-edge company says you’re a genius . . . maybe you’re a genius?

Book Review: If Anyone Builds It, Everyone Dies

September 11, 2025 · Original source

Eliezer Yudkowsky, at his best, has leaps of genius nobody else can match. Fifteen years ago, he decided that the best way to something something AI safety was to write a Harry Potter fanfiction. Many people at the time (including me) gingerly suggested that maybe this was not optimal time management for someone who was approximately the only person working full-time on humanity’s most pressing problem. He totally demolished us and proved us wronger than anyone has ever been wrong before. Hundreds of thousands of people read Harry Potter and the Methods of Rationality, it got lavish positive reviews in Syfy, Vice, and The Atlantic, and it basically one-shotted a substantial percent of the world’s smartest STEM undergrads. Fifteen years later, I still meet bright young MIT students who tell me they’re working on AI safety, and when I ask them why in public they say something about their advisor, and then later in private they admit it was the fanfic. Valuing the time of the average AI genius at the rate set by Sam Altman (let alone Mark Zuckerberg), HPMOR probably bought Eliezer a few billion dollars in free labor. Just a totally inconceivable level of victory.

Tech PACs Are Closing In On The Almonds

October 21, 2025 · Original source

Give me your degens, your risk-seeking. Your huddled masses, yearning to bet free. IV. …and we’ll be exploring it a whole lot more, very soon. Last month, the AI industry announced a new SuperPAC called “Leading The Future” (a dumb name, but, in their defense, “AIPAC” was already taken). They start with $200 million in seed funding, led by a $50 million donation by Andreessen Horowitz, and another $50 million from OpenAI co-founder Greg Brockman. (Why Brockman and not Altman, or OpenAI as a corporation? Because most people don’t know who Brockman is, so this keeps OpenAI’s hands clean. I imagine Altman going into a meeting, pointing at Brockman, and saying “I’m famous, you’re not, please cough up $50 million of your own money for the cause.”) On the same day, Meta announced their own SuperPAC, Mobilizing Economic Transformation Across (META) California. Why two PACs? Opinions differ; one person told me that it lets the general PAC avoid the negative associations that Facebook has gathered over the years, but the Verge thinks that maybe everyone else in tech hates Zuckerberg too much to work with him. Meta has committed to spending “tens of millions”. Most likely, the new PAC will use the playbook pioneered by crypto: destroy any candidate who dares support regulations on AI, by funding attack ads that don’t mention AI in any way and, at best, briefly mention the name “Leading The Future”. Just the Andreessen/Brockman SuperPAC, without any help from Meta, is already twice as rich as AIPAC. Their existence sends a clear message: we are going to crush any politician who tries to regulate AI. V. …unless someone stops them. Leading The Future still only has 2% as much money as the almond industry. The tiny scale of US political spending is dangerous insofar as it means that one or two billionaires willing to go all-in can distort the national landscape. But it also makes it possible to oppose them. Certainly if you can get one or two billionaires of your own - but it might even be within the range of a committed group of ordinary people. Not waiters and bartenders, maybe. But if safe AI supporters were as committed as Israel supporters, they could probably make something happen. For a long time, the AI safety movement has underperformed politically. Effective altruism includes thousands of well-off people committed to spending 10% of their income on improving the world. If a thousand of them gave $7K each to political candidates, that would be $7 million of campaign-finance-compliant hard money - about as much as anyone can gather for anything. Hard money buys more influence per dollar than soft money, so this could be a big deal. All you’d need is the right people to coordinate it. So far, this has been slow going. Partly it’s because in the early 2020s, people affiliated with FTX took point on this effort; when FTX imploded, it not only took its incipient political infrastructure with it, but poisoned the well for future efforts. And partly it’s because EAs overlearned the lesson of the early 2010s, when we spoke out against AI capabilities efforts so “effectively” that a bunch of people thought “wow, AI capabilities companies must be a really big deal, maybe I should found one!”; the resulting institutional scar tissue biased us towards staying quiet about our concerns. Still, I wouldn’t be writing this if the consultants and activists weren’t gearing up for a bigger fight. They asked me to include some action items for readers who want to participate: Email aisafetypolitics@gmail.com to connect to the people organizing this effort and talk with them about what you can do, including potential future donation opportunities.

Inline links: announced a new SuperPAC, Mobilizing Economic Transformation Across, the Verge thinks

My Antichrist Lecture

October 22, 2025 · Original source

Seems concerning. Anthropic might seem like an unlikely candidate for the Beast, given its emphasis on ethical conduct and safe AI research. However, they do perform various experiments on turning AIs evil - always in the context of examining these scenarios and figuring out how best to prepare - and one can imagine ways this could go wrong. We’ll examine likely failure modes in more detail later. The Mark Of The Beast John continues (13:17): And [the Beast] causeth all, both small and great, rich and poor, free and bond, to receive a mark in their right hand, or in their foreheads. And that no man might buy or sell, save he that had the mark. In the context of AI, this resembles the idea of biometric proof of personhood. Suppose that future AI agents have access to bank accounts and can transact like humans. Some platforms may choose to let AIs access their services unrestricted; others may choose to limit use to humans. How would these work in a world where AIs can break CAPTCHAs? One plan is to assign each human a code based on some immutable feature of their body, like a fingerprint or eye scan, then let the humans use that code as an ironclad ID. The most famous plan along these lines is Sam Altman’s WorldCoin. The Mark of the Beast cannot be WorldCoin itself, because WorldCoin uses an iris scan, but the Mark uses the hand or forehead. Handprints are a common biometric recognition target, but foreheads? Yes! Just last year, researchers found that forehead creases were actually a cutting-edge biometric target, and suggested them as a superior alternative to fingerprints (contactless) and facial recognition (blocked by masks during a pandemic). This section suggests that Anthropic will come up with its own proof-of-personhood scheme, superior to OpenAI’s WorldCoin in that it uses the newer forehead-based biometric recognition (with the more commonly-used handprint as a backup). We’ll discuss more about why you might not want to be in their database later. The Woman Of The Apocalypse Revelation 12:1 introduces an unnamed figure commonly called the Woman of the Apocalypse: And there appeared a great wonder in heaven; a woman clothed with the sun, and the moon under her feet, and upon her head a crown of twelve stars. The woman gives birth to a son, who is implied to be the Messiah. Satan tries to kill the son, so the mother flees with her child to Heaven, where she waits for 1,260 days. This is obviously a reference to the Virgin Mary and Christ, but (as per the multilayered symbolism of Revelation) somehow also a reference to some specific person in the End Times. I originally couldn’t figure out who that person was, but a now-deactivated Tumblr poster, resinsculpture, convinced me that it was Ursula von der Leyen, current president of the European Union. Here is a typical official picture of von der Leyen. She is in her trademark yellow suit (“clothed with the sun”), standing with her head centered in the twelve stars of the EU flag (“upon her head a crown of twelve stars”). In what sense is “the moon under her feet”? In her role as President, von der Leyen stands above, and frequently addresses, the European Parliament, which looks like this: The Parliament, also known as the Hemicycle, takes the shape of a half (or slightly crescent) moon. When von der Leyen stands in her yellow suit, in front of the Parliament, with the flag behind her, she is “clothed with the sun, and the moon under her feet, and upon her head a crown of twelve stars”.2 Von der Leyen is one of the leaders behind the EU’s push to become a “regulatory superpower”, which has born fruit in some surprisingly promising AI regulations. In particular, Europe has been especially strict on biometric proof-of-personhood: If the apocalypse involves a rogue Anthropic model somehow empowered by proof-of-personhood, Europe is one of the best candidates to resist. Von der Leyen, then, stands as a metonymy for the European Union as a bulwark for the forces of Good. The Witnesses / The Lamb Of God The Lamb is John’s version of the Messiah or the Second Coming. He gives us two clues about its identity. First, Revelation 11:3: And I will appoint my two witnesses, and they will prophesy for 1,260 days, clothed in sackcloth. The Lamb will be preceded by two witnesses. Revelation itself does not name them, but Jewish tradition says that one will be the prophet Elijah. Second, 12:1: And I looked, and, lo, a Lamb stood on Mount Zion, and with him a hundred forty and four thousand, having his Father’s name written in their foreheads. The Lamb will stand on Mount Zion. This is a specific mountain in Jerusalem, but also a poetic name for Israel (for example, “Zionism” = support for Israel). So we are looking for someone or something in Israel, which is being heralded by Elijah. The name Elijah is different in different languages, but the Russian version is “Ilya”. And in fact, famous AI scientist Ilya Sutskever recently founded an Israel-based AI company called “Safe Superintelligence”: Is there some sense in which Ilya Sutskever has “his Father’s name written on [his] forehead”? As weird as it sounds, I think this one might just be literally true. There is some kind of unusual pattern on his forehead (image source). I cannot make heads or tails of it right-side-up, but when I flip it over… …it appears to be the Name of God in Hebrew. In our working hypothesis, Ilya is Elijah, the First Witness3, which suggests that the one he is heralding - that is, the Safe Superintelligence which is to be built by his company - is the Lamb of God, the Messiah that will defeat the unsafe superintelligences produced by Anthropic and other companies. The Antichrist / The Dragon Revelation doesn’t use the word “Antichrist” - the concept comes from from the separate Epistles of John, which may or may not be by the same author. Most scholars identify the Epistle’s Antichrist with Revelation’s Beast, but I dissent: we hypothesize the Beast to be a company, but I can’t get past the elegance of having the Antichrist be - like the Christ - a particular individual. I prefer to identify him with a different character in Revelation, namely the Dragon. On the level of Biblical narrative - the same level where the Woman is the Virgin Mary - the Dragon is clearly Satan. On the level of apocalyptic prophecy, he may additionally represent an individual from our own age. Who? John says (13:2 - 13:4) The dragon gave the beast his power and his throne and great authority . . . People worshiped the dragon, because he had given authority to the beast. We saw above that the Beast is a company. Who gives companies their power, then demands to be worshiped by them? Obviously VCs. And in fact, venture capitalists are often identified with dragons in the popular imagination: But which venture capitalist? Plenty of people have claimed to know secret ways to identify the Antichrist, but surely the best-credentialled expert here is the Pope, and according to Wikipedia: Pope Pius IX in the encyclical Quartus Supra, quoting Cyprian, said Satan disguises the Antichrist with the title of Christ. What is the title of Christ? In the Bible, we find two common titles: “The Son of Man” (Matthew 12:32, Luke 12:8, John 1:51)

Winged Lion = OpenAI, founded by Sam Altman (Jewish). The winged lion would have been recognizable to Daniel’s audience as a symbol of the Babylonian Empire; Daniel was writing during the Babylonian Captivity, when there was no independent Israeli state and all Jews lived in Babylon.

Links For February 2026

February 05, 2026 · Original source

There seems to be a general mood that OpenAI is vulnerable these days, culminating in Anthropic Superbowl commercials making fun of it for introducing ads. I thought the commercials were in bad taste, misrepresenting what OpenAI’s ads would be like and turning the completely normal decision for a tech company to have an ad-supported free version of their product into some kind of horrible betrayal. I thought Sam Altman’s response was fair (although his countercriticism of Anthropic also missed the mark). People in his replies tried to enforce a norm of “if you write a long explanation defending yourself against someone else’s funny lies, that means you care and you lose”, but that’s a stupid norm and people should stop shoring it up (cf. If It’s Worth Your Time To Lie, It’s Worth My Time To Correct It).

Inline links: Anthropic Superbowl commercials, Sam Altman’s response, If It’s Worth Your Time To Lie, It’s Worth My Time To Correct It

"All Lawful Use": Much More Than You Wanted To Know

March 01, 2026 · Original source

A few hours later, Hegseth and Sam Altman declared an agreement-in-principle for OpenAI’s models to be used in the niche vacated by Anthropic. Altman stated that he had received guarantees that OpenAI’s models wouldn’t be used for mass surveillance or autonomous weapons either, but given Hegseth’s unwillingness to concede these points with Anthropic, observers speculated that the safeguards in Altman’s contract must be weaker or, in a worst-case scenario, completely toothless.

Inline links: stated

The debate centers on the Department of War’s demand that AIs be permitted for “all lawful use”. Anthropic worried that mass surveillance and autonomous weaponry would de facto fall in this category; Hegseth and Altman have tried to reassure the public that they won’t, and the parts of their agreement that have leaked to the public cite the statutes that Altman expects to constrain this category. Altman’s initial statement seemed to suggest additional prohibitions, but on a closer read, provides little tangible evidence of meaningful further restrictions.

This is currently a “lawful use” of AI, and one of the ones Dario Amodei’s letter says that he’s worried about. As far as we can tell, Altman’s contract with the Department of War doesn’t contain any provisions preventing them from using ChatGPT this way.

Inline links: Dario Amodei’s letter says

Astral Codex Ten

Table of Contents

Atlas

Sam Altman

Sam Altman

Article

Metadata

Appears In

External Links

Source Context

Backlinks

Astral Codex Ten

Table of Contents

Atlas

Sam Altman

Sam Altman

Article

Metadata

Appears In

Related Pages

External Links

Source Context

Backlinks