Astral Codex Ten

NEPA

25 min read

NEPA

Article

NEPA is a recurring organization in the Astral Codex Ten archive, appearing 3 times across 3 issues between April 17, 2023 and April 15, 2025. The archive places it in contexts such as ""People have joked about applying NEPA review to AI capabilities research""; “Trump administration may remove all NEPA regulations”; “For NEPA: I think not”. It most often appears alongside FDA, China, OpenAI.

Metadata

  • Category: Organizations
  • Mention count: 3
  • Issue count: 3
  • First seen: April 17, 2023
  • Last seen: April 15, 2025

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

Highlights From The Comments On IRBs
April 17, 2023 · Original source
Finally, most of the surveys in question are just a series of basic psychology scales or tasks both the worker and average SSC reader are very familiar with. I suspect many of them are administered by students as practice rather than 'serious' research. As the other poster said, rejected HITs are just any task the requestor declines for any reason. A worker's acceptance rate is extremely important - one of the few pieces of advice Amazon seems to give requestors is to filter for 98% or 99% acceptance rate. It's probably pretty reasonable for surveys - if you can't get 99 out of 100 of those filled out acceptably (assuming good faith by the requestors), maybe you should be filtered. It's also worth noting that Amazon makes communication difficult, and that rejected HITs can only be reversed for like a month - after that, they're permanently on your record. It's also probably worth restating: if a worker goes below the high 90s, they'll have access to fewer tasks, likely from less reputable requestors, and they'll need to do 100 of these to offset every rejection. And the worker is at much greater risk of being dug deeper into that hole by requestors rejecting their work in bad faith with no recourse - part of why surveys are popular is because the IRB can bludgeon requestors into accountability. Most of the surveys in question are also are the crumbs that filter through the grasping pedipalps of the hordes of workers (and their scripts). If people are seriously using MTurk to monetize their time, they're likely looking for 'batch HITs' - the sort of thing where there's hundreds or thousands of tasks that can be quickly repeated (moderating images, 3 cents for a sentiment analysis, a couple quarters to outline a car in an image, etc.) Of course, this mana from heaven rarely lasts long, and the worker always takes a risk - 'if I do 100 of these, and this is an unscrupulous requestor, well - I better have ten thousand accepted HITs under my belt.' That's why workers are so protective of their acceptance rate. Back to surveys - again as the other poster replied, most of what the average MTurk worker will see is probably a psychology study questionnaire with a series of whatever common scales, attention checks, and other tricks the worker has probably seen at least dozens if not hundreds of times by now. They often pay Amazon's princely sum of about 10 cents per (expected) minute - based on the minimum wage in whatever benighted 00s year Amazon Mechanical Turk launched. Anecdotally, it also seems like a lot of these are from students - probably just practice research by someone who likely has less experience with the platform than the worker themselves. The problem the requestor has - at least as of ~2018 - is that there is a lot of fraud with foreign workers getting access to MTurk accounts and submitting totally garbo data, often very quickly. Based purely on a 'time to complete' metric, this is hard to distinguish from a legit worker who has filled out hundreds of these and is looking to maximize how many pennies they get for their minutes. It also wasn't uncommon for workers to 'cook' such a survey - letting it sit at the end screen before submitting - just to avoid getting pinged for finishing it quickly. As for how this all ties back into Institutional Review Boards - well, yeah, griping to the IRB is often the MTurk worker's only recourse. Amazon just doesn't care, and as I recall a lot of requestors don't even know workers can contact them - and as mentioned there's a narrow time window to discuss rejected HITs before they become permanent. On the other hand, in a lot of cases this is basically a reddit mob complaining that a student doling out dimes screwed up their understanding of MTurk's arcane inner workings, and that's in the case that the workers aren't actually trying to defraud them for said dimes. 5. Comments About Regulation, Liability, and Vetocracy CatCube writes: I think the fundamental problem is that you cannot separate the ability to make a decision from the ability to make a *wrong* decision. However, our society--pushed by the regulator/lawyer/journalist/administrator axis you discuss--tries to use detailed written rules to prevent wrong decisions from being made. But, because of the decision/wrong decision inseparability thing, the consequences are that nobody has the ability to make a decision. This is ultimately a political question. It's not wrong, precisely, or right either. It's a question of value tradeoffs. Any constraint you put on a course of action is necessarily something that you value more than the action, but this isn't something people like to admit or hear voiced aloud. If you say, "We want to make sure that no infrastructure project will drive a species to extinction", then you are saying that's more important than building infrastructure. Which can be a defensible decision! But if you keep adding stuff--we need to make sure we're not burdening certain races, we need to make sure we're getting input from each neighborhood nearby, etc.--you can eventually end up overconstraining the problem, where there turns out to be no viable path forward for a project. This is often a consequence of the detailed rules to prevent wrong decisions. But because we can't admit that we're valuing things more than building stuff (or doing medical research, I guess?), we as a society just end up sitting and stewing about how we seemingly can't do anything anymore. We need to either: 1) admit we're fine with crumbling infrastructure, so long as we don't have any environmental, social, etc., impacts; or 2) decide which of those are less important and streamline the rules, admitting that sometimes the people who are thus able to make a decision are going to screw it up and do stuff we ultimately won't like. Darwin on why safetyism expanded just as the neoliberals were trying to decrease government regulation: Without the excuse of 'we were following all of the very strict and explicit regulations, so the bad thing that happened was a freak accident and not our fault' to rely on, companies had to take safety and caution and liability limitation and PR management into their own hands in a much more serious way. And without the confidence in very strict and explicit regulations to limit the bad things companies might do, and without democratically-elected regulators as a means to bring complaint and affect change, we became much more focused on seeking remedy for corporate malfeasance by suing companies into oblivion and destroying them in the court of public opinion. Basically, government actually *can* do useful things, as it turns out. One of the useful things it can do is be a third party to a dispute between two people or entities, such as 'corporations' and 'citizens', and use it's power to legibly and credibly ensure cooperation by explicitly specifying what will be considered defection and then punishing it harshly. This actually allows the two parties, which might otherwise be in conflict, to trust each other much more and cooperate much better, because their incentives have been shifted by a third party to make defection more costly. Without government playing that role, you can fall back into bad equilibrium of distrust and warring, which in this case might look like a wary populace ready to sue and decry at the slightest excuse, and paranoid corporations going overboard on caution and PR to shield from that. Meadow Freckle writes: Why can’t you sue an IRB for killing people for blocking research? You can clearly at least sometimes activist them into changing course. But their behavior seems sue-worthy in these examples, and completely irresponsible. We have negligence laws in other areas. Is there an airtight legal case that they’re beyond suing, or is it just that nobody’s tried? I don’t know, and this seems like an important question. And Donald writes: Why do we need special rules for medicine? The law has rules about what dangerous activities people are allowed to consent to, for example in the context of dangerous sports or dangerous jobs. Criminal and civil trials in this context seem to be a fairly functional system. If Doctors do bad things, they can stand in the accused box in court and get charged with assault or murder, with the same standards applied as are applied to everyone else. If there need to be exceptions, they should be exceptions of the form "doctors have special permission to do X". I do want to slightly defend something IRB-like here. When a doctor asks you to be part of a study, they’re implicitly promising that they did their homework, this is a valuable thing to study, and that there’s no obvious reason it should be extremely unsafe. As a patient (who may be uneducated) you have no way of knowing whether or not this promise is true. Every so often, someone does everything right, and something goes wrong anyway. A drug that everyone reasonably thought would be safe and effective turns out to have unpredictable side effects - this is part of why we have to do studies in the first place. If every time this happened, a doctor had to stand trial for assault/murder, nobody would ever study new drugs. Trials are a crapshoot, and juries tend to rule against doctors on the grounds that the disabled/dead patient is very sympathetic and everyone knows doctors/hospitals are rich and can give them infinite money as damages. There is no way for an average uneducated jury to distinguish between “doctor did their homework and got unlucky” and “doctor did an idiotic thing”. Either way, the prosecution can find “expert witnesses” to testify, for money, that you were an idiot and should have known the study would fail. In order to remove this risk, you need some standards for when a study is safe, so that if people sue you, you can say “I was following the standards and everyone else agreed with me that this was good” and then the lawsuit will fail. Right now those standards are “complied with an IRB”. This book is arguing that the IRB’s standards are too high, but we can’t cut the IRB out entirely without some kind of profound reform of the very concept of lawsuits, and I don’t know what that reform would look like. 6. Comments About The Act/Omission Distinction jumpingjacksplash writes: I think you've unintentionally elided two distinct points: first, that IRBs are wildly inefficient and often pointless within the prevailing legal-moral normative system (PLMNS); second, that IRBs are at odds with utilitarianism. Law in Anglo-Saxon countries, and most people's opinions, draw a huge distinction between harming someone and not helping them. If I cut you with a knife causing a small amount of blood loss and maybe a small scar, that's a serious crime because I have an obligation not to harm you. If I see a car hurtling towards you that you've got time to escape from if you notice it, but don't shout to warn you (even if I do this because I don't like you), then that's completely fine because I have no obligation to help you. This is the answer you'd get from both Christianity and Liberalism (in the old-fashioned/European sense of the term, cf. American Right-Libertarianism). Notably, in most Anglo-Saxon legal systems, you can't consent to be caused physical injury. Under PLMNS, researchers should always ask people if they consent to using their personal data in studies which are purely comparing data and don't change how someone will be treated. For anything that affects what medical treatment someone will or won't receive, you'd at least have to give them a full account of how their treatment would be different and what the risks of that are. If there's a real risk of killing someone, or permanently disabling them, you probably shouldn't be allowed to do the study even if all the participants give their informed consent. This isn't quite Hans Jonas' position, but it cashes out pretty similarly. That isn't to say the current IRB system works fine for PLMNS purposes; obviously there's a focus on matters that are simply irrelevant to anything anyone could be rationally concerned with. But if, for example, they were putting people on a different ventilator setting than they otherwise would, and that risked killing the patient, then that probably shouldn't be allowed; the fact that it might lead to the future survival of other, unconnected people isn't a relevant consideration, and nor is "the same number of people end up on each ventilator setting, who cares which ones it is" because under PLMNS individuals aren't fungible. Under utilitarianism, you'd probably still want some sort of oversight to eliminate pointless yet harmful experiments or reduce unnecessary harm, but it's not clear why subjects' consent would ever be a relevant concern; you might not want to tell them about the worst risks of a study, as this would upset them. The threshold would be really low, because any advance in medical science could potentially last for centuries and save vastly more people than the study would ever involve. The problem is, as is always the case for utilitarianism, this binds you to some pretty nasty stuff; I can't work out whether the Tuskegee experiment's findings have saved any lives, but Mengele's research has definitely saved more people than he killed, and I'd be surprised if that didn't apply to Unit 731 as well. The utilitarian IRB would presumably sign off on those. More interestingly, it might have to object to a study where everyone gives informed consent but the risk of serious harm to subjects is pretty high, and insist that it be done on people whose quality of life will be less affected if it goes wrong (or whose lower expected utility in the longer term makes their deaths less bad) such as prisoners or the disabled. The starting point to any ideal system has to be setting out what it's trying to achieve. Granted, if you wanted reform in the utilitarian direction, you probably wouldn't advocate a fully utilitarian system due to the tendency of the general public to recoil in horror. I want to stress how far we are away from “do experiments without patient’s consent” here - a much more common problem is that patients really want to be in experiments, and the system won’t allow it. This is most classic in studies on cancer, where patients really want access to experimental drugs and IRBs are constantly coming up with reasons not to give it to them. Jonas argued that all cancer studies should be banned because it’s impossible to consent when you’re desperate to survive, which isn’t the direction I would have taken that particular example in. But there are other examples - during COVID, lots of effective altruists stepped up to be in human challenge trials that would have gotten the vaccines tested faster, but the government wouldn’t allow them to participate. I would honestly be happy with a system that counts the harm of denying a patient’s ability to consent to an experiment they really want to be in as a negative, forget about any lives saved. And JDK writes: I haven't finished reading by felt compelled to comment on this: "the stricter IRB system in place since the '90s probably only prevents a single-digit number of deaths per decade, but causes tens of thousands more by preventing lifesaving studies." No. It does NOT "cause" deaths. We can't go down this weird path of imprecision about what "causing" means. I've been examining Ivan Illich, "Medical Nemesis" recently. By claiming IRBs which stop research ostensibly CAUSE death strikes me as cultural iatrogenesis masquerading as a cure for clinical iatrogenesis. […] "Might have been saved if" is not the same as "death was caused by". This seems to me to be a weird and overly metaphysical nitpick. Suppose a surgeon is operating on someone. In the process, they must clamp a blood vessel - this is completely safe for one minute, but if they leave it clamped more than one minute, the patient dies. They clamp it as usual, but I rush into the operating room and forceably restrain the surgeon and all the staff. The surgeon is unable to remove the clamp and the patient dies. I (and probably the legal system) would like to be able to say I caused the patient’s death in this scenario. But it sounds like JDK is saying I have to say the surgeon caused the patient's death and I was only tangentially involved. Here’s another example; suppose the US government bans all food production - farmers, hunters, fishermen, etc are forbidden from doing their jobs. After a few months, everyone starves to death. I might want to say something like “the US government’s ban on food production killed people”. But by JDK’s reasoning, this is wrong - the government merely prevented farmers and fishermen from saving people (by giving them food so they didn’t starve). I might want to say something like “Mao’s collective farming policy killed lots of people”. But since this is just a weaker version of hypothetical-Biden’s ban on food, by JDK’s reasoning I can’t do this. This seems contrary to common usage, common sense, and communicating information clearly. I have never heard any philosopher or dictionary suggest this, so what exactly is the argument? (JDK has a response here, but I didn’t find it especially enlightening) 7. Comments About The Applications For AI Metaphysiocrat writes: People have joked about applying NEPA review to AI capabilities research, but I wonder if some kind of IRB model might have legs (as part of a larger package of capabilities-slowing policy.) It’s embedded in research bureaucracies, we sort of know how to subject institutions to it, and so on. I can think of seven obvious reasons this wouldn’t work, but at this point I’m getting doomery enough that I feel like we may just have to throw every snowball we have at the train on the off chance one has stopping power. Zach Stein-Perlman writes: A colleague of mine is interested in 'IRBs for AI'-- he hasn't investigated it but has thought about IRB-y stuff in the context of takeaways for AI (https://wiki.aiimpacts.org/doku.php?id=responses_to_ai:technological_inevitability:incentivized_technologies_not_pursued:vaccine_challenge_trials). He's interested in people's takes on the topic. My take: my understanding is that the US can’t technically demand all doctors use IRBs. (Almost) al doctors use IRBs for a combination of a few reasons : The US government demands that everyone who receives federal funding use an IRB, and most doctors get some federal funding.
Inline links: writes, Darwin, writes, writes, writes, writes, has a response here, writes, writes, https://wiki.aiimpacts.org/doku.php?id=responses_to_ai:technological_inevitability:incentivized_technologies_not_pursued:vaccine_challenge_trials
Links For February 2025
February 27, 2025 · Original source
24: Claim: Trump administration may remove all NEPA regulations. I think this would most likely be very good. Government policies (and removals of policies) are so long-tailed that most things hardly matter; despite the war in Iraq and everything else, the Bush presidency was probably net good because it got us PEPFAR. Part of my plan to resist despair is to hope that Trump is doing so many crazy things that he might hit on one or two extremely long-tailed good things like this one and make up lost ground.
Inline links: Trump administration may remove all NEPA regulations
Highlights From The Comments On POSIWID
April 15, 2025 · Original source
the good effect outweighs the bad effect in circumstances sufficiently grave to justify causing the bad effect and the agent exercises due diligence to minimize the harm And I think it’s the second that’s most relevant to POSIWID. If the system could switch to doing the good without the bad, would it happily make the switch? For the cancer hospital: yes! For NEPA: I think not. This is an interesting test, thanks. My only concern is that “if the system could switch” is kind of meaningless. When we ask whether the purpose of a charity is to help the poor, or just to give high salaries to its CEO, the test urges us to ask “If it could switch to helping the poor just as much without paying its CEO anything, would it do that?” What if the answer is “the board, low-level staff, and donors would support this, but the CEO wouldn’t,” and the charity’s actions come from compromises negotiated among these groups? What is the purpose of the charity then? TimG writes: I've seen reports (don't know how true) that NGOs in San Fran get paid a lot of money to solve homelessness. But after billions spent, homelessness is worse. I thought this saying was a kinda reference to that sort of thing: the NGOs are there to collect money by virtue of the fact that there are homeless. Which is not what they are purported to do. My understanding of the situation is that there are many groups. Some are traditional anti-homelessness groups that try to build homeless shelters or something. Others are homeless-rights-advocacy groups that try to prevent the police from doing things which they think violate homeless people’s rights, like forcing them to go to shelters. It’s true that these two purposes are at odds, and that this conflict prolongs homelessness in San Francisco. But I think that thinking of this as a “system” whose “purpose” is to preserve homelessness (because systems actually act in ways contradictory to their goals) makes you less able to understand the dynamics, not more! The build-shelter groups are mostly building shelters! The fight-against-shelters groups are mostly fighting against shelters! Both of them are doing what they claimed to do, and it’s all canceling out. The more you are tempted to think of [the set of both these groups] as a single “system” fulfilling a single “purpose”, the more confused you’re making yourself. Brett writes: I've always thought of the phrase as an argument against the "no true Scotsman" fallacy when it's used in an organisational setting. When there are significant failings of an organisation, the response (within the organisation) can sometimes be: "there are some bad apples working against the purpose of our system: our system is not supposed to do this and the failings are due to individuals and not the system itself". POSIWID then is applicable: you can't claim a system "isn't supposed to" do something, if it's repeatedly doing it on a large enough scale. I don’t think this works. Often failures are because of incompetent individuals. For example, one reason that UK intelligence agencies did such a bad job fighting Communism in the ‘40s and ‘50s was that lots of their staff, including some leaders, were Soviet spies. When those people were replaced, results improved! And there are plenty of stories of companies that turn around once a few bad executives get fired and replaced (eg Apple after Jobs came back). So why would we want a phrase saying that the failure of systems is never because of incompetent individuals? David Henry (blog) writes: It makes the most sense if you don't take it as having anything to do with intentions. The truth at which it gestures is "This system can be relied upon to consistently produce this outcome, just as if it were designed to do so." The point is to suggest that the "unintended side effects" are a direct result of the "rules" of the system, intentionally so or not, and therefore you can't ignore them as one-off incidents, or hope a minor patch will fix it. The system needs to be abolished, or else given a complete overhaul. Obviously the ambiguous phrasing also allows you to assign insanely hostile and nonsensical motives to the outgroup. I would like to think this was not intention of the people who came up with the phrase, but whether it is or not, it can be relied upon to consistently produce that outcome. I agree this is one of many possible meanings it could have which there are much better ways to phrase. Joost de Wit writes: I’d say the hospital is precisely designed to cure 66% of people because it operates within constraints (financial, #doctors, approved meds). A “system” designed to cure let’s say 99% of people would look wholly different. I have occasionally been a low-level representative in hospital administration meetings. I’m trying to to think of what suggestions I could given to “redesign” the hospital to cure 99% of people. “Hey, guys, have you considered having more money?” I guarantee the hospital has considered this. The reason they don’t have more money is that insurance companies won’t pay more for care and donors won’t donate more. Maybe you could bring it up a level, to the US health care system as a whole? But insofar as anyone is in charge here (maybe the Secretary of Health and Human Services), I guarantee that person has also considered getting more money. The reason they don’t have more money is that Congress and the President set their budget and balance it off against their other priorities. Maybe the system is America as a whole? In this case yeah, you could imagine an America redesigned completely around cancer care, where there are sky-high taxes and all the money goes to cancer hospitals, so much so that bridges collapse and the military can’t defend the country anymore because we’re spending all the money on hospitals. But what does it mean to have a “systems analysis” principle which is incapable of accurately analyzing any system smaller than the whole country? Also, shouldn’t we expect a good theory to yield true predictions? My theory is that cancer hospitals want to cure as many patients as possible (given other constraints). If I recommended them a new policy that would increase their cure rate, they might worry about cost or hassle - but if it were low-cost and low-hassle, they’d eventually implement it. But if you recommended a new policy that brought them closer to 66% (“We’re on track to rise to 70% next year, but if we get Dr. Smith to relapse back into alcoholism, we can go back to 66%!”) they would call you insane and fire you immediately and definitely not agree. Since “make cure rates as high as possible” accurately predicts the hospital’s behavior, but “keep cure rates at exactly 66%” doesn’t, why would you describe the second one as the “purpose”? What use is it to accuse them of having a “purpose” which they will never take any action to achieve? But also, what are even we doing here? In real life, nobody says things like “the purpose of a cancer hospital is to keep cure rates at 66%”. Why are people defending this inane statement so hard? This reminds me of the old atheism-religion debates, where some atheist would bring up an awkwardly-phrased Bible statement, and the religious people would contort themselves to say that nooooooo, it’s totally true that the world was created in seven days, as long as you define day to mean “any time period of an indeterminate length”. But at least their motives make sense to me; lots of other things depend on whether Bible verses are true or false. POSIWID was first coined in 2001. Why should people contort themselves to defend this extremely poorly-phrased thing? In this comment thread, people have claimed that the real meaning of POSIWID is: Chesterton’s Fence
Inline links: writes, writes, blog, writes, writes