Neel Nanda
Article
Neel Nanda is a recurring person in the Astral Codex Ten archive, appearing 3 times across 3 issues between September 17, 2024 and October 13, 2025. The archive places it in contexts such as “The commenters, especially Neel Nanda, found that doing knowledge cutoffs properly is hard”; “the famous Neel Nanda mech interp paper from 2023”; “External consultants: Neel Nanda”. It most often appears alongside COVID, Manifold, Polymarket.
Metadata
- Category: People
- Mention count: 3
- Issue count: 3
- First seen: September 17, 2024
- Last seen: October 13, 2025
Appears In
Related Pages
-
- COVID (2 shared issues)
-
- Manifold (2 shared issues)
-
- Polymarket (2 shared issues)
-
- Substack (2 shared issues)
-
- 2023 (1 shared issues)
-
- Aaron Silverbook (1 shared issues)
-
- ACX Grants (1 shared issues)
-
- ACX Grants (1 shared issues)
-
- ACXG (1 shared issues)
-
- Adam Morris (1 shared issues)
-
- Adam Winkel (1 shared issues)
-
- Aerolamp (1 shared issues)
External Links
Source Context
Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.
The basic structure is the same as past forecasting AIs like FutureSearch. A heavily-modified copy of ChatGPT gathers relevant news articles, then prompts itself to think in superforecaster-like ways. The creators say the ChatGPT copy had a knowledge cutoff of October 2023, so they tested it on Metaculus questions from after that date. It got 87.7% accuracy, slightly above Metaculus forecasters’ 87.0%. Manifold is skeptical: The commenters, especially Neel Nanda, found that doing knowledge cutoffs properly is hard, and the ChatGPT base seems to know about news events after October 2023 - upon questioning, it seemed aware of an earthquake in November 2023. When presented with a different set of questions that were all after November 2023, FiveThirtyNine substantially underperformed the Metaculus average. But also, my attempts to play around with the bot haven’t been encouraging: I asked it to predict the chance that Prospera would have a population of at least 1,000 in 2027. Like FutureSearch on the same question, it cited many interesting news articles on Prospera’s chances but failed to do the basic step of figuring out its current population and growth rate. It eventually concluded 35% chance, which is reasonable enough. But when asked whether Prospera would have a population of 100,000 in 2028, it also said 35% chance, which is absurd.
Inline links: FutureSearch, When presented with a different set of questions
if you look carefully at the famous Neel Nanda mech interp paper from 2023 https://arxiv.org/pdf/2301.05217 you'll see that the model "learns" a trigonometric identity; how? because it tries a superposition of lots of functions at once, and its training loss tells it to minimize the difference between its "guess" and the correct answer, so the "guess" gradually goes from a flat distribution over "all" functions to a sharper and sharper peak at the right one. it actually picks a weird-ass linear combination of sines and cosines that no human would think to write down, but which is approximately close to the right answer.
Inline links: https://arxiv.org/pdf/2301.05217
External consultants: Paige Brocidiacono, Jay Lubow, Neel Nanda, John Schilling, Alex Turner, Robert Yaman, et al