US government is a recurring organization in the Astral Codex Ten archive, appearing 12 times across 12 issues between February 14, 2022 and March 01, 2026. The archive places it in contexts such as "US government increased the urgency of their own warnings"; "skepticism toward the US Government and media"; "a few big actors like the US and Chinese government". It most often appears alongside Twitter, China, Elon Musk.
- Article page
- US government
- Mention count
- 12
- Issue count
- 12
- First seen
- February 14, 2022
- Last seen
- March 01, 2026
Outbound links
Issue trail
- Mantic Monday: Ukraine Cube Manifold
- Ukraine Warcasting
- Yudkowsky Contra Christiano On AI Takeoff Speeds
- 22
- Links For March 2023
- Highlights From The Comments On IRBs
- Highlights From The Comments On Capitalism & Charity
- Links For January 2024
- Book Review: The Origins Of Woke
- Links For January 2025
- Introducing AI 2027
- "All Lawful Use": Much More Than You Wanted To Know
Source context
These run from about 48% to 60%, but I think the differences are justified by the slightly different wordings of the question and definitions of “invasion”. You see a big jump last Friday when the US government increased the urgency of their own warnings. I ignored this on Friday because I couldn’t figure out what their evidence was, but it looks like the smart money updated a lot on it. A few smaller markets that Clay didn’t include: Manifold is only at 36% despite several dozen traders. I think they’re just wrong - but I’m not going to use any more of my limited supply of play money to correct it, thus fully explaining the wrongness. Futuur is at 47%, but also thinks there’s an 18% chance Russia invades Lithuania, so I’m going to count this as not really mature. Insight Prediction, a very new site I’ve never seen before, claims to have $93,000 invested and a probability of 22%, which is utterly bizarre; I’m too suspicious and confused to invest, and maybe everyone else is too. (PredictIt, Polymarket, and Kalshi all avoid this question. I think PredictIt has a regulatory agreement that limits them to politics. Polymarket and Kalshi might just not be interested, or they might be too PR-sensitive to want to look like they’re speculating on wars where thousands of people could die.) What happens afterwards? Clay beats me again: For context: So it looks like forecasters expect that, conditional upon Russia invading at all, there’s an 80% chance they’ll take Mariupol in the east, a 66% chance they’ll take Kharkiv (also eastern, but only a third ethnic Russian and currently aligned with the central government), and only about a 30% chance they take Kyiv or Odessa. See also this thread full of speculation in the subreddit. As for me, I’m going all in on “yes” after seeing this tweet: Alexander Cube Last week I speculated that to truly realize the potential of prediction markets, we’d need one that was real money, easy to use, and easy to create markets on. Gustavo Lacerda and Nuno Sempere very kindly drew this picture and named it after me: Nobody has reached the promised land at the furthest point. But all three connected vertices are occupied. Augur is real-money and lets people create their own markets, (but it’s impossible to use - it’s made of complicated crypto contracts that nobody’s made a workable front end for yet). Polymarket is real money and easy to use (but doesn’t let people create their own markets; apparently they’re nervous about resolution disputes). Manifold is easy to use and lets people create their own market, but it’s not real money (they’re American and centralized, so they have to follow anti-gambling regulations). Manifold Markets Speaking of which, they’re open! As the cube suggests, Manifold is a site where anyone can create their own (play money) prediction market. They set the question and they decide when and how it resolves (with everyone else just out of luck if they decide to fake it or rug-pull). It’s a bold strategy, but boy oh boy are people liking it so far: Not actually in order This is a semi-randomly selected sample of Manifold markets, but let’s go through them one by one. The Ukraine market is the biggest on Manifold. It’s also deeply out of step with every other prediction market and the top non-prediction-market authorities - who are all giving numbers in the 50s and 60s. I don’t understand how this is so low - yes, play money < real money, but mostly because play money doesn’t get enough people betting. Here lots of people are betting - it’s the biggest market on the site, and since you only start with $1000 either twenty people have bet everything or more people have bet a fraction - but it’s still wrong. I tried to spend some play money to correct it and it snapped back to just as wrong as it was before. I have no explanation. Midnight The Stray Cat is the second biggest market on Manifold, just after Ukraine. I guess the Internet really liking cats shouldn’t be a surprise at this point. In case you need to do research first I’m told this is the cat in question: Props to Manifold for a bunch of markets like the third one on there, where they eat their own dog food by using their market to predict how their business decisions are going to go. ACX Bot has copy-pasted all of my predictions from 2022. At some point they should be able to compare their results with Zvi (ie a single very smart person), with the contest many of you entered (ie an average of formless crowdsourced predictions), and Metaculus (ie a non-monetary forecasting tournament). I’m looking forward to it! Most of you already know Lars Doucet, who’s written some great ACX posts on Georgism. I don’t know what possessed him to make a Joe Rogan Georgism interviewee market, unless he’s gunning for the position. Valinor is a group house on my street, with ~a dozen people living in and around it. We’ve been talking about fixing the backyard for a while. Now we can bet about whether it will happen. Having a number for this actually affects some of my decisions a little. Connor is hijacking the prediction market to make a poll, which is pretty cute. Dwayne Johnson does not have a 15% chance of winning the election. Manifold is suffering from the usual play money problem, where if you only start out with $1000 in play money, nobody wants to lock it up for three years to make a 15% profit. Vivek’s market, “Will I believe that 13177 is a prime number”, is pretty unusual. I’m interpreting it as a test/demonstration of prediction markets’ information-gathering ability. If you don’t know something and it’s hard to Google, you can make a prediction market about whether you’ll believe it in the future, and people who are able to figure out the answer will bet on it. Based on the 97% YES rate, I’m guessing 13177 is in fact a prime number. What else can you do this with? TANSTAAFL’s “Will I Be Convinced That Justin Trudeau Is Not Fidel Castro’s Son?” market is maybe pushing the limit of this methodology. Anyway, there are lots of me-too prediction markets but this is something genuinely new under the sun. Maybe it will be awesome itself, but I’m also hoping it helps bigger players realize how much more is possible. This Week In Metaculus A few new questions on intelligence enhancement, eg: The question explicitly allows embryo selection, but says it must raise IQ ten points and be available for <25% median income to count. Trivial improvements to existing embryo selection will top out around 9 points, so this seems to be predicting something more interesting, maybe iterated embryo selection at the very least. I’m probably slightly bearish on this one; I believe if it existed someone would find a way to get it, but I think the regulatory climate might be able to prevent the relevant research indefinitely. Improving adult IQ is really hard. This is a bold thing to speculate about! Atmospheric CO2 was 300ish for most of pre-industrial history, 400ish now, and rising. This question predicts 600 in 2100, which sounds like what happens if global warming gets a bit worse but eventually stabilizes. I’m less sure. I think if we make it to 2100, we’ll have so much technology that atmospheric CO2 can be whatever we want it to be. But maybe we’ll want it to stay where it is; once there’s been a lot of global warming and people have moved / shifted lifestyles, it could be equally disruptive to cool the planet back down. Right now it’s 5%, the official government prediction is 10% by 2030, but this market says 17.6%. But look at that probability distribution! It’s a lot of people saying 10%ish, plus a very long tail of very big numbers. I think people are disagreeing about how exponential this change is going to be. Shorts Metaculus is holding an essay contest for people who want to use their AI-related prediction markets to argue the future of AI. $6500 available in prizes.
Inline links: Manifold is only at 36%, Futuur is at 47%, an 18% chance, $93,000 invested and a probability of 22%, https://substackcdn.com/image/fetch/$s_!fl0X!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F983ebd4c-28ea-48f8-9125-6ab8d9096d6d_511x310.png, https://substackcdn.com/image/fetch/$s_!rSzR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fd76dd820-af14-4644-84f7-e4810e4e669e_410x289.png, this thread full of speculation in the subreddit, Gustavo Lacerda, Nuno Sempere, https://substackcdn.com/image/fetch/$s_!ukFP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F851b95c2-a103-43fc-8e6f-339468fa1469_485x320.png, Augur, Polymarket, Manifold, https://substackcdn.com/image/fetch/$s_!j26W!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fcba2cb25-7980-446d-8050-499c35c4e56f_919x1017.png, predictions from 2022, ACX posts on Georgism, Will I Be Convinced That Justin Trudeau Is Not Fidel Castro’s Son?, https://substackcdn.com/image/fetch/$s_!V2bV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fca419ec2-7b91-44cd-80b0-2aad1baa4c2f_766x166.png, The question, https://substackcdn.com/image/fetch/$s_!Ah6n!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F1ff4f28d-1901-45f2-85bc-8f1c2120ecd1_770x187.png, https://substackcdn.com/image/fetch/$s_!zSK8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F80e3383c-6f68-412a-ac90-ca63236bdb10_769x167.png, This, https://substackcdn.com/image/fetch/$s_!frCz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F976d1b66-daee-43d1-948a-500e77ac5a2f_764x164.png, this market, holding an essay contest
I also want to acknowledge the obvious: that the main theme of my reporting and commentary on this issue has been major skepticism toward the US Government and media, particularly in their prognostications of an “imminent” invasion. I still maintain there’s much to criticize — it strikes me as very conceivable that this constant barrage of maximalist predictions could have perversely influenced Putin’s calculations. Premature statements of fact by politicians and pundits that an invasion had already occurred, when it had not yet occurred, were reckless in such fraught circumstances. Journalists who abused their access to “official” anonymous sources did the opposite of inspiring confidence in the stark warnings they were pumping out. And so on and so forth. But yes, it has to be said: the official prophecies have in fact been tragically borne out.
Inline links: Premature, anonymous sources
I’ll need to reflect more on the implications of this outcome. All I can promise you is transparency, honesty, and a willingness to correct for any blindspots. One potential blindspot here was placing too much emphasis on the repeated and vehement criticism by actual Ukrainian officials of what they decried as alarmist US rhetoric. Just last week, I interviewed a sitting member of the Ukraine parliament who straight-up told me that externally-generated “panic” was a far greater threat to Ukraine’s security than any forthcoming Russian invasion. Ukraine’s president, over and over again, was even more searing in his own repudiations of US government and media behavior. I don’t have a great explanation for this dynamic yet, but it’s possible what they were telling me tracked too closely with my pre-existing disdain for official US claims vis-a-vis Russia — which in the very recent past have been wildly wrong and destructive. As you’ll remember if you lived through Russiagate.
Inline links: interviewed, searing
Chess AI performance over time. Why does this matter? If there’s a slow takeoff (ie gradual exponential curve), it will become obvious that some kind of terrifying transformative AI revolution is happening, before the situation gets apocalyptic. There will be time to prepare, to test slightly-below-human AIs and see how they respond, to get governments and other stakeholders on board. We don’t have to get every single thing right ahead of time. On the other hand, because this is proceeding along the usual channels, it will be the usual variety of muddled and hard-to-control. With the exception of a few big actors like the US and Chinese government, and maybe the biggest corporations like Google, the outcome will be determined less by any one agent, and more by the usual multi-agent dynamics of political and economic competition. There will be lots of opportunities to affect things, but no real locus of control to do the affecting. If there’s a fast takeoff (ie sudden FOOM), there won’t be much warning. Conventional wisdom will still say that transformative AI is thirty years away. All the necessary pieces (ie AI alignment theory) will have to be ready ahead of time, prepared blindly without any experimental trial-and-error, to load into the AI as soon as it exists. On the plus side, a single actor (whoever has this first AI) will have complete control over the process. If this actor is smart (and presumably they’re a little smart, or they wouldn’t be the first team to invent transformative AI), they can do everything right without going through the usual government-lobbying channels. So the slower a takeoff you expect, the less you should be focusing on getting every technical detail right ahead of time, and the more you should be working on building the capacity to steer government and corporate policy to direct an incoming slew of new technologies. Yudkowsky Contra Christiano Eliezer counters that although progress may retroactively look gradual and continuous when you know what metric to graph it on, it doesn’t necessarily look that way in real life by the measures that real people care about. (one way to think of this: imagine that an AI’s effective IQ starts at 0.1 points, and triples every year, but that we can only measure this vaguely and indirectly. The year it goes from 5 to 15, you get a paper in a third-tier journal reporting that it seems to be improving on some benchmark. The year it goes from 66 to 200, you get a total transformation of everything in society. But later, once we identify the right metric, it was just the same rate of gradual progress the whole time. ) So Eliezer is much less impressed by the history of previous technologies than Paul is. He’s also skeptical of the “GDP will double in 4 years before it doubles in 1” claim, because of two contingent disagreements and two fundamental disagreements. The first contingent disagreement: government regulations make it hard to deploy imperfect things, and non-trivial to deploy things even after they’re perfect. Eliezer has non-jokingly said he thinks AI might destroy the world before the average person can buy a self-driving car. Why? Because the government has to approve self-driving cars (and can drag its feet on that), but the apocalypse can happen even without government approval. In Paul’s model, sometime long before superintelligence we should have AIs that can drive cars, and that increases GDP and contributes to a general sense that exciting things are going on. Eliezer says: fine, what if that’s true? Who cares if self-driving cars will be practical a few years before the world is destroyed? It’ll take longer than that to lobby the government to allow them on the road. The second contingent disagreement: superintelligent AIs can lie to us. Suppose you have an AI which wants to destroy humanity, whose IQ is doubling every six months. Right now it’s at IQ 200, and it suspects that it would take IQ 800 to build a human-destroying superweapon. Its best strategy is to lie low for a year. If it expects humans would turn it off if they knew how close it was to superweapons, it can pretend to be less intelligent than it really is. The period when AIs are holding back so we don’t discover their true power level looks like a period of lower-than-expected GDP growth - followed by a sudden FOOM once the AI gets its superweapon and doesn’t need to hold back. So even if Paul is conceptually right and fundamental progress proceeds along a nice smooth curve, it might not look to us like a nice smooth curve, because regulations and deceptive AIs could prevent mildly-transformative AI progress from showing up on graphs, but wouldn’t prevent the extreme kind of AI progress that leads to apocalypse. To an outside observer, it would just look like nothing much changed, nothing much changed, nothing much changed, and then suddenly, FOOM. But even aside from this, Eliezer doesn’t think Paul is conceptually right! He thinks that even on the fundamental level, AI progress is going to be discontinuous. It’s like a nuclear bomb. Either you don’t have a nuclear bomb yet, or you do have one and the world is forever transformed. There is a specific moment at which you go from “no nuke” to “nuke” without any kind of “slightly worse nuke” acting as a harbinger. He uses the example of chimps → humans. Evolution has spent hundreds of millions of years evolving brainier and brainier animals (not teleologically, of course, but in practice). For most of those hundreds of millions of years, that meant the animal could have slightly more instincts, or a better memory, or some other change that still stayed within the basic animal paradigm. At the chimp → human transition, we suddenly got tool use, language use, abstract thought, mathematics, swords, guns, nuclear bombs, spaceships, and a bunch of other stuff. The rhesus monkey → chimp transition and the chimp → human transition both involved the same ~quadrupling of neuron number, but the former was pretty boring and the latter unlocked enough new capabilities to easily conquer the world. The GPT-2 → GPT-3 transition involved centupling parameter count. Maybe we will keep centupling parameter count every few years, and most times it will be incremental improvement, and one time it will conquer the world. But even talking about centupling parameter points is giving Paul too much credit. Lots of past inventions didn’t come by quadrupling or centupling something, they came by discovering “the secret sauce”. The Wright brothers (he argues) didn’t make a plane with 4x the wingspan of the last plane that didn’t work, they invented the first plane that could fly at all. The Hiroshima bomb wasn’t some previous bomb but bigger, it was what happened after a lot of scientists spent a long time thinking about a fundamentally different paradigm of bomb-making and brought it to a point where it could work at all. The first transformative AI isn’t going to be GPT-3 with more parameters, it will be what happens after someone discovers how to make machines truly intelligent. (this is the same debate Eliezer had with Ajeya over the Biological Anchors post; have I mentioned that Ajeya and Paul are married?) Fine, Let’s Nitpick The Hell Out Of The Chimps Vs. Humans Example This is where the two of them end up, so let’s follow. Between chimps and humans, there were about seven million years of intermediate steps. These had some human capabilities, but not others. IE homo erectus probably had language, but not mathematics, and in terms of taking over the world it did make it to most of the Old World but was less dominant than moderns. But if we say evolutionary history started 500 million years ago (the Cambrian), and AI history started with the Dartmouth Conference in 1955, then the equivalent of 7 million years of evolutionary history is 1 year of AI history. In the very very unlikely and forced comparison where evolutionary history and AI history go at the same speed, there will be only about a year between chimp-level and human-level AIs. A chimp-level AI probably can’t double GDP, so this would count as a fast takeoff by Paul’s criterion. But even more than that, chimp → human feels like a discontinuity. It’s not just “animals kept getting smarter for hundreds of millions of years, and then ended up very smart indeed”. That happened for a while, and then all of sudden there was a near-instant phase transition into a totally different way of using intelligence with completely new abilities. If AI worked like this, we would have useful toys and interesting specialists for a few decades, until suddenly someone “got it right”, completed the package that was necessary for “true intelligence”, and then we would have a completely new category of thing. Paul admits this analogy is awkward for his position. He answers: Chimp evolution is not primarily selecting for making and using technology, for doing science, or for facilitating cultural accumulation. The task faced by a chimp is largely independent of the abilities that give humans such a huge fitness advantage. It’s not completely independent—the overlap is the only reason that evolution eventually produces humans—but it’s different enough that we should not be surprised if there are simple changes to chimps that would make them much better at designing technology or doing science or accumulating culture […] So I don’t think the example of evolution tells us much about whether the continuous change story applies to intelligence. This case is potentially missing the key element that drives the continuous change story—optimization for performance. Evolution changes continuously on the narrow metric it is optimizing, but can change extremely rapidly on other metrics. For human technology, features of the technology that aren’t being optimized change rapidly all the time. When humans build AI, they will be optimizing for usefulness, and so progress in usefulness is much more likely to be linear. That is, evolution wasn’t optimizing for tool use/language/intelligence, so we got an “overhang” where chimps could potentially have been very good at these, but evolution never bothered “closing the circuit” and turning those capabilities “on”. After a long time, evolution finally blundered into an area where marginal improvements in these capacities improved fitness, so evolution started improving them and it was easy. Imagine a company which, through some oversight, didn’t have a Sales department. They just sat around designing and manufacturing increasingly brilliant products, but not putting any effort into selling them. Then the CEO remembers they need a Sales department, starts one up, and the company goes from moving near zero units to moving millions of units overnight. It would look like the company had “suddenly” developed a “vast increase in capabilities”. But this is only possible when a CEO who is weirdly unconcerned about profit forgets to do obvious profit-increasing things for many years. This is Paul’s counterargument to the chimp analogy. Evolution isn’t directly concerned about various intellectual skills; it only wants them in the unusual cases where they’ll contribute to fitness on the margin. AI companies will be very concerned about various intellectual skills. If there’s a trivial change that can make their product 10x better, they’ll make it. So AI capabilities will grow in a “well-rounded” way, there won’t be any “overhangs”, and there won’t be any opportunities for a sudden overhang-solving phase transition with associated new-capability development like with chimps → humans. Eliezer answers: Chimps are nearly useless because they're not general, and doing anything on the scale of building a nuclear plant requires mastering so many different nonancestral domains that it's no wonder natural selection didn't happen to separately train any single creature across enough different domains that it had evolved to solve every kind of domain-specific problem involved in solving nuclear physics and chemistry and metallurgy and thermics in order to build the first nuclear plant in advance of any old nuclear plants existing. Humans are general enough that the same braintech selected just for chipping flint handaxes and making water-pouches and outwitting other humans, happened to be general enough that it could scale up to solving all the problems of building a nuclear plant - albeit with some added cognitive tech that didn't require new brainware, and so could happen incredibly fast relative to the generation times for evolutionarily optimized brainware. Now, since neither humans nor chimps were optimized to be "useful" (general), and humans just wandered into a sufficiently general part of the space that it cascaded up to wider generality, we should legit expect the curve of generality to look at least somewhat different if we're optimizing for that. Eg, right now people are trying to optimize for generality with AIs like Mu Zero and GPT-3. In both cases we have a weirdly shallow kind of generality. Neither is as smart or as deeply general as a chimp, but they are respectively better than chimps at a wide variety of Atari games, or a wide variety of problems that can be superposed onto generating typical human text. They are, in a sense, more general than a biological organism at a similar stage of cognitive evolution, with much less complex and architected brains, in virtue of having been trained, not just on wider datasets, but on bigger datasets using gradient-descent memorization of shallower patterns, so they can cover those wide domains while being stupider and lacking some deep aspects of architecture. It is not clear to me that we can go from observations like this, to conclude that there is a dominant mainline probability for how the future clearly ought to go and that this dominant mainline is, "Well, before you get human-level depth and generalization of general intelligence, you get something with 95% depth that covers 80% of the domains for 10% of the pragmatic impact". ...or whatever the concept is here, because this whole conversation is, on my own worldview, being conducted in a shallow way relative to the kind of analysis I did in Intelligence Explosion Microeconomics, where I was like, "here is the historical observation, here is what I think it tells us that puts a lower bound on this input-output curve". Here Eliezer sort of kind of grants Paul’s point that AIs will be optimized for generality in a way chimps aren’t, but points to his previous “Intelligence Explosion Microeconomics” essay to argue that we should expect a fast takeoff anyway. IEM has a lot of stuff in it, but one key point is that instead of using analogies to predict the course of future AI, we should open that black box and try to actually reason about how it will work, in which case we realize that recursive self-improvement common-sensically has to cause an intelligence explosion. I am sort of okay with this, but I feel like a commitment to avoiding analogies should involve not bringing up the chimp-human analogy further, which Eliezer continues to do, quite a lot. I do feel like Paul succeeded in convincing me that we shouldn’t place too much evidential weight on it. The Wimbledon Of Reference Class Tennis “Reference class tennis” is an old rationalist idiom for people throwing analogies back and forth. “AI will be slow, because it’s an economic transition like the Agricultural or Industrial Revolution, and those were slow!” “No, AI will be fast, because it’s an evolutionary step like chimps → humans, and that was fast!” “No, AI will be slow, because it’s an invention, like the computer, and computers were invented piecemeal and required decades of innovation to be useful.” “No, AI will be fast, because it’s an invention, like the nuclear bomb, and nuclear bombs went from impossible to city-killing in a single day.” “No, AI will be slow, because it will be surrounded by a shell-like metallic computer case, which makes it like a turtle, and turtles are slow.” “No, AI will be fast, because it’s dangerous and powerful, like a tiger, and tigers are fast!” And so on. Comparing things to other things is a time-tested way of speculating about them. But there are so many other things to compare to that you can get whatever result you want. This is the failure mode that the term “reference class tennis” was supposed to point to. Both participants in this debate are very smart and trying their hardest to avoid reference-class tennis, but neither entirely succeeds. Eliezer’s preferred classes are Bitcoin (“there wasn't a cryptocurrency developed a year before Bitcoin using 95% of the ideas which did 10% of the transaction volume”), nukes, humans/chimps, the Wright Brothers, AlphaGo (which really was a discontinuous improvement on previous Go engines), and AlphaFold (ditto for proteins). Paul’s preferred classes are the Agricultural and Industrial Revolutions, chess engines (which have gotten better along a gradual, well-behaved curve), all sorts of inventions like computers and ships (likewise), and world GDP. Eliezer already listed most of these in his Intelligence Explosion Microeconomics paper in 2013, and concluded that the space of possible analogies was contradictory enough that we needed to operate at a higher level. Maybe so, but when someone lobs a reference class tennis ball at you, it’s hard to resist the urge to hit it back. Recursive Self-Improvement This is where I think Eliezer most wants to take the discussion. The idea is: once AI is smarter than humans, it can do a superhuman job of developing new AI. In his Microeconomics paper, he writes about an argument he (semi-hypothetically) had with Ray Kurzweil about Moore’s Law. Kurzweil expected Moore’s Law to continue forever, even after the development of superintelligence. Eliezer objects: Suppose we were dealing with minds running a million times as fast as a human, at which rate they could do a year of internal thinking in thirty-one seconds, such that the total subjective time from the birth of Socrates to the death of Turing would pass in 20.9 hours. Do you still think the best estimate for how long it would take them to produce their next generation of computing hardware would be 1.5 orbits of the Earth around the Sun? That is: the fact that it took 1.5 years for transistor density to double isn’t a natural law. It’s pointing to a law that the amount of resources (most notably intelligence) that civilization focused on the transistor-densifying problem equalled the amount it takes to double it every 1.5 years. If some shock drastically changed available resources (by eg speeding up human minds a million times), this would change the resources involved, and the same laws would predict transistor speed doubling in some shorter amount of time (naively 0.000015 years, although realistically at that scale other inputs would dominate). So when Paul derives clean laws of economics showing that things move along slow growth curves, Eliezer asks: why do you think they would keep doing this when one of the discoveries they make along that curve might be “speeding up intelligence a million times”? (Eliezer actually thinks improvements in the quality of intelligence will dominate improvements in speed - AIs will mostly be smarter, not just faster - but speed is a useful example here and we’ll stick with it) Paul answers: Summary of my response: Before there is AI that is great at self-improvement there will be AI that is mediocre at self-improvement. Powerful AI can be used to develop better AI (amongst other things). This will lead to runaway growth. This on its own is not an argument for discontinuity: before we have AI that radically accelerates AI development, the slow takeoff argument suggests we will have AI that significantly accelerates AI development (and before that, slightly accelerates development). That is, an AI is just another, faster step in the hyperbolic growth we are currently experiencing, which corresponds to a further increase in rate but not a discontinuity (or even a discontinuity in rate). The most common argument for recursive self-improvement introducing a new discontinuity seems be: some systems “fizzle out” when they try to design a better AI, generating a few improvements before running out of steam, while others are able to autonomously generate more and more improvements. This is basically the same as the universality argument in a previous section. Eliezer: Oh, come on. That is straight-up not how simple continuous toy models of RSI work. Between a neutron multiplication factor of 0.999 and 1.001 there is a very huge gap in output behavior. Outside of toy models: Over the last 10,000 years we had humans going from mediocre at improving their mental systems to being (barely) able to throw together AI systems, but 10,000 years is the equivalent of an eyeblink in evolutionary time - outside the metaphor, this says, "A month before there is AI that is great at self-improvement, there will be AI that is mediocre at self-improvement." (Or possibly an hour before, if reality is again more extreme along the Eliezer-Hanson axis than Eliezer. But it makes little difference whether it's an hour or a month, given anything like current setups.) This is just pumping hard again on the intuition that says incremental design changes yield smooth output changes, which (the meta-level of the essay informs us wordlessly) is such a strong default that we are entitled to believe it if we can do a good job of weakening the evidence and arguments against it. And the argument is: Before there are systems great at self-improvement, there will be systems mediocre at self-improvement; implicitly: "before" implies "5 years before" not "5 days before"; implicitly: this will correspond to smooth changes in output between the two regimes even though that is not how continuous feedback loops work. I got a bit confused trying to understand the criticality metaphor here. There’s no equivalent of neutron decay, so any AI that can consistently improve its intelligence is “critical” in some sense. Imagine Elon Musk replaces his brain with a Neuralink computer which - aside from having read-write access - exactly matches his current brain in capabilities. Also he becomes immortal. He secludes himself from the world, studying AI and tinkering with his brain’s algorithms. Does he become a superintelligence? I think under the assumptions Paul and Eliezer are using, eventually maybe. After some amount of time he’ll come across a breakthrough he can use to increase his intelligence. Then, armed with that extra intelligence, he’ll be able to pursue more such breakthroughs. However intelligent the AI you’re scared of is, Musk will get there eventually. How long will it take? A good guess might be “years” - Musk starts out as an ordinary human, and ordinary humans are known to take years to make breakthroughs. Suppose it takes Musk one year to come up with a first breakthrough that raises his IQ 1 point. How long will his second breakthrough take? It might take longer, because he has picked the lowest-hanging fruit, and all the other possible breakthroughs are much harder. Or it might take shorter, because he’s slightly smarter than he was before, and maybe some extra intelligence goes a really long way in AI research. The concept of an intelligence explosion seems to assume the second effect dominates the first. This would match the observation that human researchers, who aren’t getting any smarter over time, continue making new discoveries. That suggests the range of possible discoveries at a given intelligence level is pretty vast. Some research finds that the usual pattern in science is constant rate of discovery from exponentially increasing number of researchers, suggesting strong low-hanging fruit effects, but these seem to be overwhelmed by other considerations in AI right now. I think Eliezer’s position on this subject is shaped by assumptions like: If you have an AI as intelligent as Elon Musk today, then tomorrow you can run it on more hardware with a bit of normal human algorithmic progress, and get one twice as intelligent. So even if it would take Elon years to make a breakthrough, long before those years are up you’ll have an AI that can make breakthroughs much faster.
On the DSL threat, someone brought up that the US government was quicker to call the Russian invasion of Ukraine than most prediction markets. I agree this happened. In some sense, it’s unsurprising; the US government has spy satellites, moles in the Kremlin, and lots of highly-paid analysts. Of course they should do better than everyone else.
And yet “trust the US government” has so far failed to solve all of our epistemic problems. Partly this is because the US government constantly disagrees with itself (the FDA got in a fight with Biden over vaccine readiness; Trump’s EPA and Biden’s EPA made very different statements on climate change). Partly it’s because for internal political reasons or military/geopolitical reasons, the US government has lots of incentives to lie or stretch the truth. Partly it’s because other organizations with the same advantages as the US government make counterclaims (eg the Ukrainian government said their intelligence told them Russia wouldn’t invade, and they also seemed pretty trustworthy).
Aside from the wokeness angle, I find this to be an interesting intellectual property question; the Saami say that EU law gives them IP rights to their clothing. If gaming companies used an outfit trademarked by some fashion company without permission, I think the fashion company would be legally in the right to demand the game remove it, so I guess this hinges on whether you can consider a culture to be the sort of unit that can trademark things. Companies are allowed to claim rights to any product their employees invent, and I think universities do something similar, so it doesn’t seem like a stretch for a tribe to make a similar demand. I think probably the fair solution is for the US government to trademark every American cultural product (t-shirts! jeans! burgers!) and then tell the Saami they probably don’t want a trade war and we’ll let them use our stuff (for example, draw a picture of a person in a t-shirt) only if they let us use theirs. Plus an extra lump sum bonus payment from them to us for making us go through this annoying process.
Finally, most of the surveys in question are just a series of basic psychology scales or tasks both the worker and average SSC reader are very familiar with. I suspect many of them are administered by students as practice rather than 'serious' research. As the other poster said, rejected HITs are just any task the requestor declines for any reason. A worker's acceptance rate is extremely important - one of the few pieces of advice Amazon seems to give requestors is to filter for 98% or 99% acceptance rate. It's probably pretty reasonable for surveys - if you can't get 99 out of 100 of those filled out acceptably (assuming good faith by the requestors), maybe you should be filtered. It's also worth noting that Amazon makes communication difficult, and that rejected HITs can only be reversed for like a month - after that, they're permanently on your record. It's also probably worth restating: if a worker goes below the high 90s, they'll have access to fewer tasks, likely from less reputable requestors, and they'll need to do 100 of these to offset every rejection. And the worker is at much greater risk of being dug deeper into that hole by requestors rejecting their work in bad faith with no recourse - part of why surveys are popular is because the IRB can bludgeon requestors into accountability. Most of the surveys in question are also are the crumbs that filter through the grasping pedipalps of the hordes of workers (and their scripts). If people are seriously using MTurk to monetize their time, they're likely looking for 'batch HITs' - the sort of thing where there's hundreds or thousands of tasks that can be quickly repeated (moderating images, 3 cents for a sentiment analysis, a couple quarters to outline a car in an image, etc.) Of course, this mana from heaven rarely lasts long, and the worker always takes a risk - 'if I do 100 of these, and this is an unscrupulous requestor, well - I better have ten thousand accepted HITs under my belt.' That's why workers are so protective of their acceptance rate. Back to surveys - again as the other poster replied, most of what the average MTurk worker will see is probably a psychology study questionnaire with a series of whatever common scales, attention checks, and other tricks the worker has probably seen at least dozens if not hundreds of times by now. They often pay Amazon's princely sum of about 10 cents per (expected) minute - based on the minimum wage in whatever benighted 00s year Amazon Mechanical Turk launched. Anecdotally, it also seems like a lot of these are from students - probably just practice research by someone who likely has less experience with the platform than the worker themselves. The problem the requestor has - at least as of ~2018 - is that there is a lot of fraud with foreign workers getting access to MTurk accounts and submitting totally garbo data, often very quickly. Based purely on a 'time to complete' metric, this is hard to distinguish from a legit worker who has filled out hundreds of these and is looking to maximize how many pennies they get for their minutes. It also wasn't uncommon for workers to 'cook' such a survey - letting it sit at the end screen before submitting - just to avoid getting pinged for finishing it quickly. As for how this all ties back into Institutional Review Boards - well, yeah, griping to the IRB is often the MTurk worker's only recourse. Amazon just doesn't care, and as I recall a lot of requestors don't even know workers can contact them - and as mentioned there's a narrow time window to discuss rejected HITs before they become permanent. On the other hand, in a lot of cases this is basically a reddit mob complaining that a student doling out dimes screwed up their understanding of MTurk's arcane inner workings, and that's in the case that the workers aren't actually trying to defraud them for said dimes. 5. Comments About Regulation, Liability, and Vetocracy CatCube writes: I think the fundamental problem is that you cannot separate the ability to make a decision from the ability to make a *wrong* decision. However, our society--pushed by the regulator/lawyer/journalist/administrator axis you discuss--tries to use detailed written rules to prevent wrong decisions from being made. But, because of the decision/wrong decision inseparability thing, the consequences are that nobody has the ability to make a decision. This is ultimately a political question. It's not wrong, precisely, or right either. It's a question of value tradeoffs. Any constraint you put on a course of action is necessarily something that you value more than the action, but this isn't something people like to admit or hear voiced aloud. If you say, "We want to make sure that no infrastructure project will drive a species to extinction", then you are saying that's more important than building infrastructure. Which can be a defensible decision! But if you keep adding stuff--we need to make sure we're not burdening certain races, we need to make sure we're getting input from each neighborhood nearby, etc.--you can eventually end up overconstraining the problem, where there turns out to be no viable path forward for a project. This is often a consequence of the detailed rules to prevent wrong decisions. But because we can't admit that we're valuing things more than building stuff (or doing medical research, I guess?), we as a society just end up sitting and stewing about how we seemingly can't do anything anymore. We need to either: 1) admit we're fine with crumbling infrastructure, so long as we don't have any environmental, social, etc., impacts; or 2) decide which of those are less important and streamline the rules, admitting that sometimes the people who are thus able to make a decision are going to screw it up and do stuff we ultimately won't like. Darwin on why safetyism expanded just as the neoliberals were trying to decrease government regulation: Without the excuse of 'we were following all of the very strict and explicit regulations, so the bad thing that happened was a freak accident and not our fault' to rely on, companies had to take safety and caution and liability limitation and PR management into their own hands in a much more serious way. And without the confidence in very strict and explicit regulations to limit the bad things companies might do, and without democratically-elected regulators as a means to bring complaint and affect change, we became much more focused on seeking remedy for corporate malfeasance by suing companies into oblivion and destroying them in the court of public opinion. Basically, government actually *can* do useful things, as it turns out. One of the useful things it can do is be a third party to a dispute between two people or entities, such as 'corporations' and 'citizens', and use it's power to legibly and credibly ensure cooperation by explicitly specifying what will be considered defection and then punishing it harshly. This actually allows the two parties, which might otherwise be in conflict, to trust each other much more and cooperate much better, because their incentives have been shifted by a third party to make defection more costly. Without government playing that role, you can fall back into bad equilibrium of distrust and warring, which in this case might look like a wary populace ready to sue and decry at the slightest excuse, and paranoid corporations going overboard on caution and PR to shield from that. Meadow Freckle writes: Why can’t you sue an IRB for killing people for blocking research? You can clearly at least sometimes activist them into changing course. But their behavior seems sue-worthy in these examples, and completely irresponsible. We have negligence laws in other areas. Is there an airtight legal case that they’re beyond suing, or is it just that nobody’s tried? I don’t know, and this seems like an important question. And Donald writes: Why do we need special rules for medicine? The law has rules about what dangerous activities people are allowed to consent to, for example in the context of dangerous sports or dangerous jobs. Criminal and civil trials in this context seem to be a fairly functional system. If Doctors do bad things, they can stand in the accused box in court and get charged with assault or murder, with the same standards applied as are applied to everyone else. If there need to be exceptions, they should be exceptions of the form "doctors have special permission to do X". I do want to slightly defend something IRB-like here. When a doctor asks you to be part of a study, they’re implicitly promising that they did their homework, this is a valuable thing to study, and that there’s no obvious reason it should be extremely unsafe. As a patient (who may be uneducated) you have no way of knowing whether or not this promise is true. Every so often, someone does everything right, and something goes wrong anyway. A drug that everyone reasonably thought would be safe and effective turns out to have unpredictable side effects - this is part of why we have to do studies in the first place. If every time this happened, a doctor had to stand trial for assault/murder, nobody would ever study new drugs. Trials are a crapshoot, and juries tend to rule against doctors on the grounds that the disabled/dead patient is very sympathetic and everyone knows doctors/hospitals are rich and can give them infinite money as damages. There is no way for an average uneducated jury to distinguish between “doctor did their homework and got unlucky” and “doctor did an idiotic thing”. Either way, the prosecution can find “expert witnesses” to testify, for money, that you were an idiot and should have known the study would fail. In order to remove this risk, you need some standards for when a study is safe, so that if people sue you, you can say “I was following the standards and everyone else agreed with me that this was good” and then the lawsuit will fail. Right now those standards are “complied with an IRB”. This book is arguing that the IRB’s standards are too high, but we can’t cut the IRB out entirely without some kind of profound reform of the very concept of lawsuits, and I don’t know what that reform would look like. 6. Comments About The Act/Omission Distinction jumpingjacksplash writes: I think you've unintentionally elided two distinct points: first, that IRBs are wildly inefficient and often pointless within the prevailing legal-moral normative system (PLMNS); second, that IRBs are at odds with utilitarianism. Law in Anglo-Saxon countries, and most people's opinions, draw a huge distinction between harming someone and not helping them. If I cut you with a knife causing a small amount of blood loss and maybe a small scar, that's a serious crime because I have an obligation not to harm you. If I see a car hurtling towards you that you've got time to escape from if you notice it, but don't shout to warn you (even if I do this because I don't like you), then that's completely fine because I have no obligation to help you. This is the answer you'd get from both Christianity and Liberalism (in the old-fashioned/European sense of the term, cf. American Right-Libertarianism). Notably, in most Anglo-Saxon legal systems, you can't consent to be caused physical injury. Under PLMNS, researchers should always ask people if they consent to using their personal data in studies which are purely comparing data and don't change how someone will be treated. For anything that affects what medical treatment someone will or won't receive, you'd at least have to give them a full account of how their treatment would be different and what the risks of that are. If there's a real risk of killing someone, or permanently disabling them, you probably shouldn't be allowed to do the study even if all the participants give their informed consent. This isn't quite Hans Jonas' position, but it cashes out pretty similarly. That isn't to say the current IRB system works fine for PLMNS purposes; obviously there's a focus on matters that are simply irrelevant to anything anyone could be rationally concerned with. But if, for example, they were putting people on a different ventilator setting than they otherwise would, and that risked killing the patient, then that probably shouldn't be allowed; the fact that it might lead to the future survival of other, unconnected people isn't a relevant consideration, and nor is "the same number of people end up on each ventilator setting, who cares which ones it is" because under PLMNS individuals aren't fungible. Under utilitarianism, you'd probably still want some sort of oversight to eliminate pointless yet harmful experiments or reduce unnecessary harm, but it's not clear why subjects' consent would ever be a relevant concern; you might not want to tell them about the worst risks of a study, as this would upset them. The threshold would be really low, because any advance in medical science could potentially last for centuries and save vastly more people than the study would ever involve. The problem is, as is always the case for utilitarianism, this binds you to some pretty nasty stuff; I can't work out whether the Tuskegee experiment's findings have saved any lives, but Mengele's research has definitely saved more people than he killed, and I'd be surprised if that didn't apply to Unit 731 as well. The utilitarian IRB would presumably sign off on those. More interestingly, it might have to object to a study where everyone gives informed consent but the risk of serious harm to subjects is pretty high, and insist that it be done on people whose quality of life will be less affected if it goes wrong (or whose lower expected utility in the longer term makes their deaths less bad) such as prisoners or the disabled. The starting point to any ideal system has to be setting out what it's trying to achieve. Granted, if you wanted reform in the utilitarian direction, you probably wouldn't advocate a fully utilitarian system due to the tendency of the general public to recoil in horror. I want to stress how far we are away from “do experiments without patient’s consent” here - a much more common problem is that patients really want to be in experiments, and the system won’t allow it. This is most classic in studies on cancer, where patients really want access to experimental drugs and IRBs are constantly coming up with reasons not to give it to them. Jonas argued that all cancer studies should be banned because it’s impossible to consent when you’re desperate to survive, which isn’t the direction I would have taken that particular example in. But there are other examples - during COVID, lots of effective altruists stepped up to be in human challenge trials that would have gotten the vaccines tested faster, but the government wouldn’t allow them to participate. I would honestly be happy with a system that counts the harm of denying a patient’s ability to consent to an experiment they really want to be in as a negative, forget about any lives saved. And JDK writes: I haven't finished reading by felt compelled to comment on this: "the stricter IRB system in place since the '90s probably only prevents a single-digit number of deaths per decade, but causes tens of thousands more by preventing lifesaving studies." No. It does NOT "cause" deaths. We can't go down this weird path of imprecision about what "causing" means. I've been examining Ivan Illich, "Medical Nemesis" recently. By claiming IRBs which stop research ostensibly CAUSE death strikes me as cultural iatrogenesis masquerading as a cure for clinical iatrogenesis. […] "Might have been saved if" is not the same as "death was caused by". This seems to me to be a weird and overly metaphysical nitpick. Suppose a surgeon is operating on someone. In the process, they must clamp a blood vessel - this is completely safe for one minute, but if they leave it clamped more than one minute, the patient dies. They clamp it as usual, but I rush into the operating room and forceably restrain the surgeon and all the staff. The surgeon is unable to remove the clamp and the patient dies. I (and probably the legal system) would like to be able to say I caused the patient’s death in this scenario. But it sounds like JDK is saying I have to say the surgeon caused the patient's death and I was only tangentially involved. Here’s another example; suppose the US government bans all food production - farmers, hunters, fishermen, etc are forbidden from doing their jobs. After a few months, everyone starves to death. I might want to say something like “the US government’s ban on food production killed people”. But by JDK’s reasoning, this is wrong - the government merely prevented farmers and fishermen from saving people (by giving them food so they didn’t starve). I might want to say something like “Mao’s collective farming policy killed lots of people”. But since this is just a weaker version of hypothetical-Biden’s ban on food, by JDK’s reasoning I can’t do this. This seems contrary to common usage, common sense, and communicating information clearly. I have never heard any philosopher or dictionary suggest this, so what exactly is the argument? (JDK has a response here, but I didn’t find it especially enlightening) 7. Comments About The Applications For AI Metaphysiocrat writes: People have joked about applying NEPA review to AI capabilities research, but I wonder if some kind of IRB model might have legs (as part of a larger package of capabilities-slowing policy.) It’s embedded in research bureaucracies, we sort of know how to subject institutions to it, and so on. I can think of seven obvious reasons this wouldn’t work, but at this point I’m getting doomery enough that I feel like we may just have to throw every snowball we have at the train on the off chance one has stopping power. Zach Stein-Perlman writes: A colleague of mine is interested in 'IRBs for AI'-- he hasn't investigated it but has thought about IRB-y stuff in the context of takeaways for AI (https://wiki.aiimpacts.org/doku.php?id=responses_to_ai:technological_inevitability:incentivized_technologies_not_pursued:vaccine_challenge_trials). He's interested in people's takes on the topic. My take: my understanding is that the US can’t technically demand all doctors use IRBs. (Almost) al doctors use IRBs for a combination of a few reasons : The US government demands that everyone who receives federal funding use an IRB, and most doctors get some federal funding.
Inline links: writes, Darwin, writes, writes, writes, writes, has a response here, writes, writes, https://wiki.aiimpacts.org/doku.php?id=responses_to_ai:technological_inevitability:incentivized_technologies_not_pursued:vaccine_challenge_trials
The US government demands that everyone who receives federal funding use an IRB, and most doctors get some federal funding.
Suppose that eg the US government decided to give everyone free health care by taxing people and spending it on health care. It seems like this should have tradeoffs or hurt the economy somehow. But you could argue that the health care money just goes to doctors and nurses and so on, who would then spend it on normal economy stuff, so the non-health-care economy is just as big as always.
1: Did you know: the US government maintains a database of dad jokes (h/t @april)
Inline links: a database of dad jokes, @april
White people have average IQ 100, black people have average IQ 85, this IQ difference accurately predicts the different proportions of whites and blacks in most areas, most IQ differences within race are genetic, maybe across-race ones are genetic too. I love Hitler and want to marry him. None of these are great options, and I think most people work off some vague cloud of all of these and squirm if you try to make them get too specific. I don’t exactly blame Hanania for not taking a strong stand here. It’s just strange to assume civil rights law is bad and unnecessary without having any opinion on whether any of this is true, whether civil rights law is supposed to counterbalance it, and whether it counterbalances it a fair amount. A cynic might notice that in February of this year, Hanania wrote Shut Up About Race And IQ. He says that the people who talk about option 4 are “wrong about fundamental questions regarding things like how people form their political opinions, what makes for successful movements, and even their own motivations.” A careful reader might notice what he doesn’t describe them as being wrong about. The rest of the piece almost-but-not-quite-explicitly clarifies his position: I read him as saying that race realism is most likely true, but you shouldn’t talk about it, because it scares people. (I’m generally against “calling people out” for believing in race realism. I think people should be allowed to hide beliefs that they’d get punished for not hiding. I sympathize with some of these positions and place medium probability on some weak forms of them. I think Hanania is open enough about where he’s coming from that this review doesn’t count as a callout.) His foil here is race realist Nathan Cofnas, who says you have to discuss these things. Otherwise progressives can win every argument by using the line of reasoning above - “Just look how much inequality there still is, this shows there’s still lots of racism or at least the lingering effects of past racism, obviously our job isn’t done yet and we need lots more civil rights law to combat it.” Hanania’s answer to Cofnas is that this isn’t a debate club. “Ah, but Glaucon, your claim that affirmative action is unnecessary must imply the corollary that there must be no inequality, thus proving a contradiction.” LOL no. Realistically this will get fought on the level of “You oppose affirmative action, which makes you a gross Nazi” vs. “You support affirmative action, which makes you an annoying wokescold.” Just say the wokescold thing louder than your enemies say the Nazi thing, and you win. Talking about racial differences scares people off and doesn’t help. I find it hard not to feel contempt for this level of contempt for reason, but Hanania is no doubt right about the strategic considerations. And in his book, he follows his own principle. There’s no discussion of why civil rights law might be necessary, or why it might be impossible for companies to hire enough minorities without reverse discrimination. As he predicts on his blog, it’s not fatal. You wouldn’t notice unless you were looking for it. I’m not really sure what to do here. How do you review a book that has a glaring omission, but also its author has written an essay called Here’s Why I Like Glaring Omissions And Think Everyone Should Have Them? Is it dishonest? Some sort of special super-meta-honesty? How many stars do you take off? Nothing in my previous history of book-reviewing has prepared me for this question. The Origins Of . . . Racial Categories Hanania presents a few scattered arguments that civil rights law is the origin of woke, of which the section on racial categories was most interesting. Having instituted affirmative action, the government had to decide what categories it was going to inspect businesses for. Like the rest of civil rights law, the resulting system was a bunch of political kludges. There is no “true” set of races that “falls out naturally” from genetic or cultural data, but the US government’s system was especially fake and embarrassing. They created the concept of “Asian-American” by combining the old category “Oriental” together with Indians, Pakistanis, Thais, etc. Then, under pressure from the Hawaiian delegation, they added Pacific Islanders to create a even more heterogenous and meaningless category of “AAPI” (Asian American or Pacific Islander). Then, under more pressure from Hawaii later, they separated out “Native Hawaiian” again. The result is that Pakistanis, Koreans, and Tongans are the “same race”, but Hawaiians and Samoans are “different races”.
Inline links: Shut Up About Race And IQ
After an official investigation, his death was ruled a suicide and all further inquiry into the instance have been barred by the Us government. While Lewis himself did not have any immediate descendants, his extended family have submitted requests every year to have his body exhumed in order to confirm the cause of death. To this day their requests have unanimously been denied.
The summary: we think that 2025 and 2026 will see gradually improving AI agents. In 2027, coding agents will finally be good enough to substantially boost AI R&D itself, causing an intelligence explosion that plows through the human level sometime in mid-2027 and reaches superintelligence by early 2028. The US government wakes up in early 2027, potentially after seeing the potential for AI to be a decisive strategic advantage in cyberwarfare, and starts pulling AI companies into its orbit - not fully nationalizing them, but pushing them into more of a defense-contractor-like relationship. China wakes up around the same time, steals the weights of the leading American AI, and maintains near-parity. There is an arms race which motivates both countries to cut corners on safety and pursue full automation over public objections; this goes blindingly fast and most of the economy is automated by ~2029. If AI is misaligned, it could move against humans as early as 2030 (ie after it’s automated enough of the economy to survive without us). If it gets aligned successfully, then by default power concentrates in a double-digit number of tech oligarchs and US executive branch members; this group is too divided to be crushingly dictatorial, but its reign could still fairly be described as technofeudalism. Humanity starts colonizing space at the very end of the 2020s / early 2030s.
Whatever the President thinks is legal may also, in certain cases, be legal. During the War on Terror, President George W. Bush’s Office of Legal Counsel claimed that he also had the inherent constitutional power as President to lawfully authorize warrantless mass collection of internet metadata and telephone call records, a dragnet scooping up Americans and non-Americans’ data alike. The program was initially justified by counterterrorism, but was far more expansive4. This was such a scandal within the US government that many DOJ officials threatened to resign; even DOJ officials who didn’t know what was going on threatened to resign because they assumed it was so bad. Later, the program was moved under statutory and FISA Court frameworks, until finally Congress ended it by passing the USA FREEDOM Act.
Inline links: warrantless mass collection of internet metadata and telephone call records, 4, threatened to resign because they assumed it was so bad
First, the policies are vague. Directive 3000.09 requires that autonomous weapon systems be designed to “allow commanders and operators to exercise appropriate levels of human judgment over the use of force.” But it doesn’t define “appropriate”, and the US government has stated it “is a flexible term” where what qualifies “can differ across weapon systems, domains of warfare, types of warfare, operational contexts, and even across different functions in a weapon system.” The institution that decides what’s “appropriate” is the same institution that wants to use the weapon.