superintelligent AI
Article
superintelligent AI is a recurring concept in the Astral Codex Ten archive, appearing 2 times across 2 issues between January 19, 2022 and July 25, 2023. The archive places it in contexts such as “They both accept that superintelligent AI is coming”; “I’m afraid of the world being destroyed by superintelligent AI”. It most often appears alongside transhumanists, 1992 Presidential debate, ABA.
Metadata
- Category: Concepts
- Mention count: 2
- Issue count: 2
- First seen: January 19, 2022
- Last seen: July 25, 2023
Appears In
- Practically-A-Book Review: Yudkowsky Contra Ngo On Agents
- Highlights From The Comments On Social Model Of Disability
Related Pages
-
- transhumanists (2 shared issues)
-
- 1992 Presidential debate (1 shared issues)
-
- ABA (1 shared issues)
-
- Adesh Thapliyal (1 shared issues)
-
- AGI (1 shared issues)
-
- AI boxing problem (1 shared issues)
-
- AI Safety (1 shared issues)
-
- Alan Smith (1 shared issues)
-
- Alexander Graham Bell (1 shared issues)
-
- AlphaFold (1 shared issues)
-
- AlphaGo (1 shared issues)
-
- AlphaWorldConquest (1 shared issues)
External Links
Source Context
Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.
They both accept that superintelligent AI is coming, potentially soon, potentially so suddenly that we won't have much time to react.
They both accept that a sufficiently advanced superintelligent AI could destroy the world if it wanted to. Maybe it would dream up a bioweapon and bribe some lab to synthesize it. Maybe it would spoof the military into starting nuclear war. Maybe it would invent self-replicating nanomachines that could disassemble everything into component molecules. Maybe it would do something we can't possibly imagine, the same way we can do things gorillas can't possibly imagine. Point is, they both accept as a given that this could happen. I explored this assumption more in this 2015 article; since then, we've only doubled down on our decision to gate trillions of dollars in untraceable assets behind a security system of "bet you can't solve this really hard math problem".
Inline links: this 2015 article
This becomes a starting point for the rest of the discussion. In the unusually good scenario where good smart people have the capability to build AI first, how do they use it without either themselves building the kind of superintelligent AI that will probably blow up and destroy the world, or squandering their advantage until some idiot builds that AI and kills them? Eliezer gives an example:
I feel the same way about “the world was created in seven days”. And I feel the same way about people trying to metaphor their way around my weird beliefs - when I say I’m afraid of the world being destroyed by superintelligent AI, I don’t mean that as a metaphor for the complexity of modern technological life, I don’t mean that capitalism or bureaucracy is like a superintelligence, I mean I’m actually afraid of the world being destroyed by superintelligent AI! I’m a grown adult and I know how to say the specific things I mean!