Astral Codex Ten

rationalist community

14 min read

rationalist community

Article

rationalist community is a recurring concept in the Astral Codex Ten archive, appearing 11 times across 11 issues between January 29, 2021 and June 18, 2025. The archive places it in contexts such as “Then it draws on vague rumors of the “rationalist community""; “I think the rationalist and EA communities”; “my grasp of the views of the rationalist community”. It most often appears alongside Scott, Harvard, California.

Metadata

  • Category: Concepts
  • Mention count: 11
  • Issue count: 11
  • First seen: January 29, 2021
  • Last seen: June 18, 2025

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

Contra Weyl On Technocracy
January 29, 2021 · Original source
I am not defending technocracy. But I do like evidence-based policy. So I read with interest Glen Weyl's Why I Am Not A Technocrat. It starts with a short summary of Seeing Like A State. It ties this into modern "evidence-based policy" and "mechanism design". It talks about how technocrats will always have their own insular culture and biases and paradigms, which prevent them from seeing the real world in its full complexity. Therefore, we should be careful about supposedly "objective" policies, and make sure they are always heavily informed by real people's real knowledge. Then it draws on vague rumors of the "rationalist community" and a shadowy figure named "Eliezer Yudkowsky" to create a completely fictional reimagination of us as a group of benighted people who don't understand any of these things, and just go around saying "hurr durr top-down systems are great, no way there could possibly be anything our models don't capture."
Inline links: Why I Am Not A Technocrat
The effective altruism movement, which largely grew directly out of the rationalist movement, seeks to maximize the efficacy with which charitable donations are directed using standard rationalist methods. It is a tight-knit community that strongly privileges rationalist approaches over all other forms of knowledge-making (such as from the humanities, continental philosophy, or humanistic social sciences) and tends to dismiss input not formulated in rationalist terms. The community also has a strong and explicitly stated view that its activities uniquely contribute to the achievement of “the good”: of their top five recommendations of most productive careers by a leading community organization, two suggest being a researcher or support staff within the movement, and two others recommend working on the AI alignment problem (see the next point). Until recently, much of the analysis and funding emerging from the community has pointed towards a focus on extremely unlikely but potentially catastrophic risks, such as alien, asteroid or biological catastrophes.
Weyl wrote this essay a few months before COVID, so his pooh-poohing of the idea that there might be a biological catastrophe is an unfortunate anachronism. But I think it's important to note that we got this right (and he got it wrong) precisely because we "privilege rationalist approaches over all other forms of knowledge-making". People like Toby Ord tried to calculate the risk of every kind of disaster and how bad it would be - and at the same time Weyl was making fun of us for caring about biological catastrophes, Ord was writing about how the numbers suggested zoonotic diseases from bats could cause catastrophic pandemics. This kind of work ultimately led to EA flagship group Open Philanthropy Project spending almost $50 million on its Biosecurity And Pandemic Preparedness Program between 2014 and 2019; if other people had taken a few minutes to read our arguments instead of chiding us for how naive it is to prioritize things based on rational methods, maybe the world would have been more prepared.
Inline links: like Toby Ord, Biosecurity And Pandemic Preparedness Program
Weyl Contra Me On Technocracy
January 29, 2021 · Original source
1. Let me start with concessions. There are many points where @slatestarcodex correctly highlights various areas where my grasp of beliefs and facts are limited or wrong, especially in the depth of my grasp of the views of the rationalist community. I freely admit that there are serious limits to how much I've been able to research the views of people in this community and I certainly hope they are not as I characterized them, though as I will point out below many elements of Scott's response confirm my concerns.
2. Given the last point, I fully acknowledge the danger of throwing stones lest I shatter my own glass house. However, leaving aside any blame, I think to make sense of my piece and Scott's response requires a bit of context that clearly he lacked. First, outside some specific blog post I wrote, I am best known as a mechanism designer. To cast me as a general opponent of technology and mechanisms runs against literally everything I am known for and have worked on my whole career. The piece was as much a self-critique and caution about taking the sorts of work I do in the wrong, over-zealous spirit as I had seen many in the rationalist community doing as anything else. Second, if responding to anything directly, it was this review of my book by https://www.aeaweb.org/articles?id=10.1257/jel.20191533 I think that piece perfectly exemplifies the spirit I am responding to and critiquing. I think we need to avoid that spirit of mechanism design.
Inline links: https://www.aeaweb.org/articles?id=10.1257/jel.20191533
4. Furthermore, the positive examples of technocracy @slatestarcodex refers to are...surprising. Two examples. To call school desegregation a technocratic invention papers over decades of community activism for desegregation. Perhaps even more dramatically looking at the coronavirus as an example of the success of technocracy runs against pretty much any reasonable reading of the international data. Danielle Allen and I have a piece coming out on this (we were both deeply involved in developing a response plan here https://ethics.harvard.edu/Covid-Roadmap that significantly influenced now-President Biden's response), but perhaps the sharpest point here is that the country, Taiwan, which performed best in the virus was led in part by Audrey Tang who moved back to Taiwan after being immersed in and repulsed by the rationalist movement in Silicon Valley - see e.g. https://www.wired.com/story/how-taiwans-unlikely-digital-minister-hacked-the-pandemic/) and dedicated herself to doing things differently in Taiwan (see her amazing poetic job description here:
Inline links: https://ethics.harvard.edu/Covid-Roadmap, https://www.wired.com/story/how-taiwans-unlikely-digital-minister-hacked-the-pandemic/
If You Can Be Bad, You Can Also Be Good
May 03, 2021 · Original source
Spotted on Reddit about the rationalist community:
Inline links: on Reddit
Open Thread 183
August 01, 2021 · Original source
3: The rationalist community is always experimenting with new institutions, and now we're trying an old one - official community matchmaker. My friend Elena has stepped up to the plate and is doing good work interviewing lots of people about their preferences and trying to set them up on dates with other people she's interviewed (poly vs. mono is optional, she has good matches available in both categories). She finds we need more of three particular demographics: men who want kids, women who don't necessarily want kids, and people who are open to dating a transgender partner. If you fall into one of these three groups, we're expanding the definition of "rationalist community" wide enough to cover you as an ACX reader, whether you identify as "rationalist" in any other way or not, and you’re eligible for Elena’s (free) service. Consider checking out her website ; she'll also be around to answer any questions in the comments here. Currently most available matches are in the SF Bay Area, so though in theory the service isn't geographically limited she'll probably be best positioned to help people who live near there. [EDIT: All Elena’s free appointments are now full, she is trying to figure out some kind of better business model, you can sign up for her wait list if you want]
Inline links: her website
Book Review: The Scout Mindset
September 29, 2021 · Original source
But Galef earned her celebrity status honestly, through long years of hard labor in the rationality mines. Back in ~2007, a bunch of people interested in biases and decision-making joined the “rationalist community” centered around the group blogs Overcoming Bias and Less Wrong. Around 2012, they mostly left to do different stuff. Some of them went into AI to try to save the world. Others went into effective altruism to try to revolutionize charity. Some, like me, got distracted and wrote a few thousand blog posts on whatever shiny things happened to catch their eyes. But a few stuck around and tried to complete the original project. They founded a group called the Center For Applied Rationality (aka “CFAR”, yes, it’s a pun) to try to figure out how to actually make people more rational in the real world.
Inline links: Overcoming Bias, Less Wrong
A lot of the best rationalists I know instinctively apply these tests to everything they think. One technique for cultivating this practice (not the book’s recommendation) is to go on Twitter, where the adage is “there’s always an old tweet”. Argue that people who say racist things should be cancelled, and someone will dig up your old racist tweet and make you defend why you shouldn’t face the same consequences. Argue that it’s disgraceful when the other party uses extreme violent language about their outgroup, and someone will dig up an old tweet where you used even more extreme language about yours. Demand that the Republican senator resign for sexual misconduct, and someone will find the old tweet where you said the Democratic senator should tough it out. Eventually, if you want to maintain any dignity at all, you learn to double-check whether your beliefs are consistent with one another or with what you’d believe in vaguely similar situations.
I know some rationalists who aren’t very nice people (I also know others who are great). There are lots of other facets of nice-person-ness beyond just an ability to acknowledge your mistakes (for example, you have to start out thinking that being mean to other people is a mistake!) But all these skills about “what tests can you put your thoughts through to see things from the other person’s point of view?” or “how do you stay humble and open to correction?” are non-trivial parts of the decent-human-being package, and sometimes they carry over.
ACX Grants Results
December 28, 2021 · Original source
…via the Long Term Future Fund: This is an EA grants program that volunteered to evaluate and judge all applications that had anything to do with AI or the rationalist and effective altruist communities. They have more grant-making expertise and more money than I do, so I was happy to send those applications their way without considering them further. If you sent in an AI or rationalist/EA community-related grant and didn’t see your name above, don’t despair! LTFF hasn’t made their decisions yet, so I’m not able to announce these at the same time as the others. When they’re done, I’ll make sure you know.
Why Do I Suck?
February 02, 2022 · Original source
2: The rationalist community was really great
It still is! But in the same sense that I was clearing a personal backlog of unwritten-up ideas, the rationalist community was clearing a backlog of scientific and philosophical ideas sitting in journals or obscure old books that it turned out were really interesting to a lot of people. The early Internet provided a critical mass where people interested in cognition and math and the future could suddenly all share the parts of the puzzle they knew about with each other and make rapid progress. Eliezer Yudkowsky, Robin Hanson, Nick Bostrom, and other intellectuals all had their own backlog of stuff which had probably been published in journals or something but which the wider world had yet to appreciate. I was the biggest-name blogger who was sitting around listening to them talk about it, so I got access to a stream of amazing content that most people didn’t know about.
There was a time when “bets are a tax on bullshit” or “words are cluster-structures in thingspace” were new and exciting ideas. There was a time when nobody had heard of the replication crisis unless they happened to be reading the medical journals where John Ioannidis was publishing. The rationalist community scooped all this stuff up, broke it down into easily digestible bits, and put it in one place. I happened to be sitting in that place, which meant I had the privilege of transmitting it to many of you.
Slightly Against Underpopulation Worries
August 04, 2022 · Original source
Even this isn’t quite right, because a lot of Orthodox Jews do leave Orthodoxy, so along with those 100 million devout Orthodox there will probably be a few dozen million extra Reform Jews with a confused relationship to religion and lots of emotional baggage. It’ll be a great time for the rationalist community.
23
February 14, 2023 · Original source
The rationalist community has many advantages here - it’s a well-bounded, closely connected group of people who are all interested in experimenting with weird social technology. Our version of the matching checkbox site, https://www.reciprocity.io/, has solved the onboarding problem; most rationalists have accounts, though you’ll need to be Facebook friends to see some of them.
Inline links: https://www.reciprocity.io/
Bayes For Everyone
May 30, 2025 · Original source
There’s a lot wrong about that summary1, but it’s at least a place to begin. The “new tools” are the ones created in the last couple thousand years, like careful concepts, finicky definitions, the scientific method, analysis & synthesis, and the quantification of everything. We here in the rationalist community love these things. Inculcating students in them is what much of the academic curriculum was built to do.
Inline links: 1
This, I think, is actually the deepest value of teaching kids Bayes: it’s a way to get them to converse with people whose views they think are stupid. And it’s only through actually doing that that we have any chance of helping people become rational. Such conversations (done with checking each other’s math) are the way to inculcate an openness to being wrong, a detached self-worth, comfort with uncertainty, and all the other aspects of what Julia Galef has so winsomely dubbed scout mindset. Approached this way, Bayes isn’t the weirdo, quant-y capstone to scout mindset — it’s the publicly-accessible front door. 6: Where are we, now? I.I.: You claim to be promoting something new, but it seems to me like you’re just describing the rationalist community. Indeed I am! My claim wasn’t that this method of explaining Bayes goes beyond what’s practiced in the community — just that it’s likely to do a better job pulling kids into the sorts of intense, curiosity-fueled relationships that our community excels at.
ACX Grants 1-3 Year Updates
June 18, 2025 · Original source
Helped create Manifold Markets, a prediction market site with thousands of satisfied users, whose various spinoffs play a central role in the rationalist/EA community.