Astral Codex Ten

Chesterton's Fence

8 min read

Chesterton’s Fence

Article

Chesterton’s Fence is a recurring concept in the Astral Codex Ten archive, appearing 2 times across 2 issues between July 14, 2023 and April 15, 2025. The archive places it in contexts such as ““Chesterton’s Fence” is a Rationalist shorthand for the idea that we should expect the world to be more complex than our models”; “You can think of it as a bit like Chesterton’s fence”. It most often appears alongside Chesterton’s Fence, !Kung San, aboriginal people on the west coast of Canada.

Metadata

  • Category: Concepts
  • Mention count: 2
  • Issue count: 2
  • First seen: July 14, 2023
  • Last seen: April 15, 2025

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

Your Book Review: The Educated Mind
July 14, 2023 · Original source
I think you see it, for example, in the community’s penchant for yelling “Chesterton’s Fence!” whenever anyone criticizes something they don’t understand. Philosophic understanding is obsessed with things making sense; Ironic understanding says, reality is always a few steps beyond you. “Chesterton’s Fence” is a Rationalist shorthand for the idea that we should expect the world to be more complex than our models; if something looks stupid, you should consider that the stupidity may be in you.
Highlights From The Comments On POSIWID
April 15, 2025 · Original source
Biden supports Xi’s campaign to prevent Chinese people from getting addicted to video games, and wants to keep video-game-enabling GPUs out of the country. …and design experiments to distinguish between these, or wait for more chip sanctions to see how they pan out. But in real life, we can be very sure some of these (like 2 and 7) weren’t intended, and others (like 4) were. Why? Some combination of trusting Biden’s stated goals, psychoanalyzing Biden’s plausible goals, checking who lobbied Biden to do this, and reading enough international relations journals to get a sense of what policymakers are thinking about. I think it’s fine to do black box systems analysis, just like it’s fine to do behaviorism. But we should view these as methodological commitments for a specific group, rather than good strategies for normal people. Jared Peterson (blog) writes: This originally struck me as rather silly and as an obvious misinterpretation of an idea that has nothing to do with human intentions...then I read the comments and saw many people claiming exactly that! Donella Meadows is an important figure in the field of Systems Thinking, and says by definition (whether human designed or not), systems have a purpose. "A system’s function or purpose is not necessarily spoken, written, or expressed explicitly, except through the operation of the system. The best way to deduce the system’s purpose is to watch for a while to see how the system behaves. Purposes are deduced from behavior, not from rhetoric or stated goals” One way to think about this is that Meadows would be OK talking about Molochs purpose as something coherent. Is changing the climate the purpose of modern capitalism? In one sense, no. But simultaneously, it is perfectly coherent to talk about the system as having that exact purpose because the system seems to work towards that goal. Even if you push against the system, the system seems to adapt and continue with that goal anyways. There is something almost intelligent about systems where they seem to work towards goals that no one ever intended. But the phrase isn't about human goals at all! Oh! I agree this makes sense if you need to talk about the “purpose” of an un-designed system with no humans in it. Moonshadow writes: This sentiment is grasping towards the same sort of place as your "Meditations on Moloch" essay. No-one involved in the system wants what the system actually ends up doing. But whatever their individual intents, /the system as a whole/, if allowed to grow naturally, inevitably ends up doing what Moloch wants. Of course the purpose we intended for the system isn't really that, any more than Moloch really exists. But you can't begin the meta level fight - of designing the system's high level organisational structures and incentives to try to reduce this effect, instead of letting it emerge organically like it always does - unless you first admit the problem. I agree this is a useful thing to talk about, I just don’t think “purpose” is the right word for it. I’m not even sure “system” is the right word for it. A good example of Moloch would be two countries having a nuclear arms race. But how is this POSIWID? The purpose of the . . . system of two countries . . . is to . . . have a nuclear arms race? This is pretty different from how I usually hear it used. But here is a dissenting voice. Ajb writes: POSIWID was not originally an antagonistic political snark. It's perfectly sensible to notice that a system may be fulfilling other purposes than it does officially, and this is not incompatible with it operating in good faith. You can think of it as a bit like Chesterton's fence: * to reform a system you should understand what purposes it fulfills, not just what it is officially supposed to do * These additonal or alternative purposes may in fact be desirable ones that you should avoid breaking. Cybernetics (where the phrase originated) drew a lot of inspiration from biology, and there obviously nothing has an 'official purpose' at all. But it nevertheless has organisation and is functional. Rob writes: The problem with quoting aphorisms like this is that it misses the context - specifically the context of a management consultant (viz. Stafford Beer) who spends his entire life being told about systems his clients have put in place, with some stated purpose in mind. Those systems do not achieve their stated purposes, but can be continually defended against change by re-stating the purpose - this shouldn't work, but in practice it often does, because most people aren't great at decoupling intent from outcome. "The purpose of a system is what it does" is a good rhetorical counter, because it acknowledges that, in practice, any continuation of a system with known outcomes is a tacit acceptance of those outcomes as the system's real purpose. You don't get to claim some other "real" purpose once you know what the outcomes are. My interpretation has always been in the spirit of this tweet: https://x.com/primawesome/status/1178671690261286918?lang=en > My neighbor told me coyotes keep eating his outdoor cats so I asked how many cats he has and he said he just goes to the shelter and gets a new cat afterwards so I said it sounds like he’s just feeding shelter cats to coyotes and then his daughter started crying. I agree this makes more sense in the context of some supposed person claiming that “the system has good intentions” means they should never have to change the system. I don’t think I really see this failure mode. I bet a lot of you are going to yell at me and say that, I don’t know, homelessness or something is like this. But defenders of the current homelessness system never say you can’t change it because it had good intentions when it started. I predict they would say that their own group is doing good work, and it’s everyone else who needs to change. Or that the current system works a little and just needs to be funded more. Or that the current system is better than nothing, and your proposed attempt to “change” it is secretly a plan to gut it and leave homeless people without help. I definitely don’t think they’d say “Yes, your proposed change would improve the system, but you’re not allowed to make it because the people who designed the current system had good intentions”. Leah Libresco Sargeant writes: I think the Catholic principle of double effect is helpful here. This often comes up in the case of eg delivering a baby pre-viability because the mom has an infection that will progress to sepsis and death if she and the baby aren’t separated. The three criteria are: the nature of the act is itself good, or at least morally neutral;
Inline links: blog, writes, https://substackcdn.com/image/fetch/$s_!Dy01!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6929f8eb-51df-403a-8666-268f9f1f034f_529x185.png, writes, writes, https://x.com/primawesome/status/1178671690261286918?lang=en, writes