Astral Codex Ten

cybernetics

14 min read

cybernetics

Article

cybernetics is a recurring concept in the Astral Codex Ten archive, appearing 2 times across 2 issues between March 23, 2021 and April 15, 2025. The archive places it in contexts such as “Engineers were already designing systems along cybernetic principles long before Wiener invented theoretical cybernetics”; “Cybernetics (where the phrase originated) drew a lot of inspiration from biology”. It most often appears alongside 2008 crisis, A Failure, But Not Of Prediction, Affordable Housing Bureau.

Metadata

  • Category: Concepts
  • Mention count: 2
  • Issue count: 2
  • First seen: March 23, 2021
  • Last seen: April 15, 2025

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

Book Review: Antifragile
March 23, 2021 · Original source
I've grown used to this in medicine, but I was surprised to see Taleb point out the same effect in fields like physics and engineering. For example, he argues that jet engines just sort of happened when engineers played around with airplane engines enough; physicists didn't explain how they worked until later. Engineers were already designing systems along cybernetic principles long before Wiener invented theoretical cybernetics. Medieval European architecture was done essentially without mathematics - Roman numerals (the only numerals anyone had at the time) were too unwieldy to add or subtract, and "according to the medieval science historian Guy Beaujouan, before the thirteenth century no more than five persons in the whole of Europe knew how to perform division."
Highlights From The Comments On POSIWID
April 15, 2025 · Original source
Homeowners want to preserve or increase the value of their houses. Of these, I think 6 is one of the less important ones - if this were the dominating factor, people would support upzoning, since it usually raises the value of properties in the upzone (if developers can build skyscrapers on your land, then your land value goes up relative to the profitability of skyscrapers). But part of the problem is that people don’t support upzoning. So 6 can’t be the dominating factor. Without POSIWID, people could think about all of these possibilities and come to their own conclusions. POSIWID tries to ban thinking about 1-5 by fiat, insisting that 6 is the only possible explanation and anyone considering the others is naive. I think this makes it a bad heuristic. But there are two more concerning things about how Negating is using POSIWID. First, he’s picking out one particularly salient thing the system does (raise house prices) and claim that’s “the” purpose. He could equally well pick any of the other results - preserve neighborhood character, protect the environment, help Chinese people escape currency controls. Like I said in the original post, in practice POSIWID serves as justification for paranoia - whatever effect you like least, whatever possibility would be most sinister - that’s the one that the system is intentionally aiming for. Second, he’s saying it’s the purpose of “the” system. Which system? I bet whatever government he’s talking about has some organization called the Affordable Housing Bureau, or whatever. And I bet that the Affordable Housing Bureau really does make housing slightly more affordable, relative to the counterfactual where it doesn’t exist. It’s just that lots of other government, market, and social forces conspire to make it much less affordable. If Negating were to claim “The purpose of the Affordable Housing Bureau is to make housing less affordable”, this would be false even if the overall picture (the government is deliberately raising real estate prices) were true. Brad writes: I have to toss in Pournelle's Iron Law. The purpose of a system - when it is first established - may be dramatically different from the purpose it assumes after a few years. Consider: You establish a system to solve a problem. That could be homelessness, or asylum, or drug abuse, or any of a number of other things. This system employs people, who then have an automatic interest - not in solving the problem - but in prolonging it, even in making it worse. After all, without the problem, the organization would not need to exist. And hwold writes: I see it used as "if you have a complex system/bureaucracy to solve X, then the incentives inside it is for X to get worse, and incentives will not have 0 influence on outcomes" For example : https://x.com/Devon_Eriksen_/status/1906042672499864034 I think this sounds profound on first glance, and it’s probably true in some cases. But it’s not nearly true enough to be an Iron Law. Try to think about it in specific Near Mode cases: If you eliminated police, would crime go down, because the police have an incentive to preserve crime? If you eliminated the fire department, would fires go down, because the fire department has an incentive to preserve fire? If you eliminated doctors, would cancer deaths go down, because doctors have an incentive to preserve cancer deaths? If you eliminated the FDA, would dangerous drug side effects go down, because the FDA has an incentive to preserve dangerous drug side effects? If you eliminated the Federal Reserve, would bank runs go down, because the Federal Reserve has an incentive to preserve bank runs? Brad’s original comment mentions homelessness and drug abuse, but I know some drug abuse doctors, and they’re (mostly) good people who do their best in a tough situation. Drug abuse doesn’t continue because drug abuse doctors are secretly ensuring it continues to help their bottom line. Drug abuse continues because fentanyl is really, really addictive. Even good conspiracy theories don’t work like this. Was there a conspiracy among pain pill manufacturers to addict people? Yeah, kinda, although I think the degree to which this caused the opioid crisis is pretty overblown. But the pain pill manufacturers weren’t a system dedicated to preventing addiction. They did their job (reduce pain) fine, then ran an unrelated evil conspiracy on the side! Breb writes: This way of thinking may result from taking a strategy for predicting the motives of individuals, and using it to predict the motives of organisations. "Cui bono?" works when you're considering a single action carried out by a single person at a single moment in time, but it doesn't really work when you're considering the behaviour of hundreds of people who are incentivised to somewhat-but-not-perfectly cooperate over a long period to somewhat-but-not-perfectly implement a goal that was established by someone who somewhat-but-not-perfectly understands that that goal is just an instrument to attain a larger, more complex goal set by somebody else. I’m against this for individuals too! There are a million self-help gurus who try to convince you that that if you procrastinate - let’s say you always do term papers the night before and get terrible grades and it’s threatening your ability to complete college - then it must be because this secretly benefits you in some way. Maybe your overly-strict father wants you to complete college, and you’re deliberately trying to fail as a secret act of rebellion against him hidden even from yourself. Although something like this might sometimes be true, more often a clearer understanding of the circuitry involved (in this case, hyperbolic discounting) saves you from these labyrinths and lets you think about things straightforwardly again. Tom J writes: In the original Stafford Beer sense, the slogan POSIWID means that you can't tell from outside the system whether any given behaviour was *intended* or not. For the purposes of objective analysis, you have to treat your system as a black box that *does* whatever it's observed to do, as opposed to what people *claim* the point of the system is. This may be true in cybernetics. Or it may be an interesting methodological commitment, in the same way that the behaviorists’ “assume there is no such thing as human interiority” was an interesting methodological commitment. But I don’t think it’s common or valuable in normal-life analysis of social systems. When Biden bans NVIDIA from sending advanced chips to China, black box analysis would have to be ambivalent between explanations like: Biden personally hates Jensen Huang and wants his company to suffer
Inline links: writes, writes, https://x.com/Devon_Eriksen_/status/1906042672499864034, writes, hyperbolic discounting, writes
Biden supports Xi’s campaign to prevent Chinese people from getting addicted to video games, and wants to keep video-game-enabling GPUs out of the country. …and design experiments to distinguish between these, or wait for more chip sanctions to see how they pan out. But in real life, we can be very sure some of these (like 2 and 7) weren’t intended, and others (like 4) were. Why? Some combination of trusting Biden’s stated goals, psychoanalyzing Biden’s plausible goals, checking who lobbied Biden to do this, and reading enough international relations journals to get a sense of what policymakers are thinking about. I think it’s fine to do black box systems analysis, just like it’s fine to do behaviorism. But we should view these as methodological commitments for a specific group, rather than good strategies for normal people. Jared Peterson (blog) writes: This originally struck me as rather silly and as an obvious misinterpretation of an idea that has nothing to do with human intentions...then I read the comments and saw many people claiming exactly that! Donella Meadows is an important figure in the field of Systems Thinking, and says by definition (whether human designed or not), systems have a purpose. "A system’s function or purpose is not necessarily spoken, written, or expressed explicitly, except through the operation of the system. The best way to deduce the system’s purpose is to watch for a while to see how the system behaves. Purposes are deduced from behavior, not from rhetoric or stated goals” One way to think about this is that Meadows would be OK talking about Molochs purpose as something coherent. Is changing the climate the purpose of modern capitalism? In one sense, no. But simultaneously, it is perfectly coherent to talk about the system as having that exact purpose because the system seems to work towards that goal. Even if you push against the system, the system seems to adapt and continue with that goal anyways. There is something almost intelligent about systems where they seem to work towards goals that no one ever intended. But the phrase isn't about human goals at all! Oh! I agree this makes sense if you need to talk about the “purpose” of an un-designed system with no humans in it. Moonshadow writes: This sentiment is grasping towards the same sort of place as your "Meditations on Moloch" essay. No-one involved in the system wants what the system actually ends up doing. But whatever their individual intents, /the system as a whole/, if allowed to grow naturally, inevitably ends up doing what Moloch wants. Of course the purpose we intended for the system isn't really that, any more than Moloch really exists. But you can't begin the meta level fight - of designing the system's high level organisational structures and incentives to try to reduce this effect, instead of letting it emerge organically like it always does - unless you first admit the problem. I agree this is a useful thing to talk about, I just don’t think “purpose” is the right word for it. I’m not even sure “system” is the right word for it. A good example of Moloch would be two countries having a nuclear arms race. But how is this POSIWID? The purpose of the . . . system of two countries . . . is to . . . have a nuclear arms race? This is pretty different from how I usually hear it used. But here is a dissenting voice. Ajb writes: POSIWID was not originally an antagonistic political snark. It's perfectly sensible to notice that a system may be fulfilling other purposes than it does officially, and this is not incompatible with it operating in good faith. You can think of it as a bit like Chesterton's fence: * to reform a system you should understand what purposes it fulfills, not just what it is officially supposed to do * These additonal or alternative purposes may in fact be desirable ones that you should avoid breaking. Cybernetics (where the phrase originated) drew a lot of inspiration from biology, and there obviously nothing has an 'official purpose' at all. But it nevertheless has organisation and is functional. Rob writes: The problem with quoting aphorisms like this is that it misses the context - specifically the context of a management consultant (viz. Stafford Beer) who spends his entire life being told about systems his clients have put in place, with some stated purpose in mind. Those systems do not achieve their stated purposes, but can be continually defended against change by re-stating the purpose - this shouldn't work, but in practice it often does, because most people aren't great at decoupling intent from outcome. "The purpose of a system is what it does" is a good rhetorical counter, because it acknowledges that, in practice, any continuation of a system with known outcomes is a tacit acceptance of those outcomes as the system's real purpose. You don't get to claim some other "real" purpose once you know what the outcomes are. My interpretation has always been in the spirit of this tweet: https://x.com/primawesome/status/1178671690261286918?lang=en > My neighbor told me coyotes keep eating his outdoor cats so I asked how many cats he has and he said he just goes to the shelter and gets a new cat afterwards so I said it sounds like he’s just feeding shelter cats to coyotes and then his daughter started crying. I agree this makes more sense in the context of some supposed person claiming that “the system has good intentions” means they should never have to change the system. I don’t think I really see this failure mode. I bet a lot of you are going to yell at me and say that, I don’t know, homelessness or something is like this. But defenders of the current homelessness system never say you can’t change it because it had good intentions when it started. I predict they would say that their own group is doing good work, and it’s everyone else who needs to change. Or that the current system works a little and just needs to be funded more. Or that the current system is better than nothing, and your proposed attempt to “change” it is secretly a plan to gut it and leave homeless people without help. I definitely don’t think they’d say “Yes, your proposed change would improve the system, but you’re not allowed to make it because the people who designed the current system had good intentions”. Leah Libresco Sargeant writes: I think the Catholic principle of double effect is helpful here. This often comes up in the case of eg delivering a baby pre-viability because the mom has an infection that will progress to sepsis and death if she and the baby aren’t separated. The three criteria are: the nature of the act is itself good, or at least morally neutral;
Inline links: blog, writes, https://substackcdn.com/image/fetch/$s_!Dy01!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6929f8eb-51df-403a-8666-268f9f1f034f_529x185.png, writes, writes, https://x.com/primawesome/status/1178671690261286918?lang=en, writes