Astral Codex Ten

Samotsvety

6 min read

Samotsvety

Article

Samotsvety is a recurring person in the Astral Codex Ten archive, appearing 2 times across 2 issues between April 18, 2022 and March 21, 2024. The archive places it in contexts such as “Samotsvety’s estimate implies a highly confident answer”; “by guessing randomly, or consistently taking some number 5% higher than Samotsvety’s number”. It most often appears alongside 2024 election, 2050, AI.

Metadata

  • Category: People
  • Mention count: 2
  • Issue count: 2
  • First seen: April 18, 2022
  • Last seen: March 21, 2024

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

22
April 18, 2022 · Original source
Last month superforecaster group Samotsvety Forecasts published their estimate of the near-term risk of nuclear war, with a headline number of 24 micromorts per week.
Inline links: estimate of the near-term risk of nuclear war
A few weeks later, J. Peter Scoblic, a nuclear security expert with the International Security Program, shared his thoughts. His editor wrote:
Inline links: shared his thoughts
In other words: the Samotsvety analysis was the best that domain-general forecasting had to offer. This is the best that domain-specific expertise has to offer. Let’s see if they line up:
In Continued Defense Of Non-Frequentist Probabilities
March 21, 2024 · Original source
Consider some object or process which might or might not be a coin - perhaps it’s a dice, or a roulette wheel, or a US presidential election. We divide its outcomes into two possible bins - evens vs. odds, reds vs. blacks, Democrats vs. Republicans - one of which I have arbitrarily designated “heads” and the other “tails” (you don’t get to know which side is which). It may or may not be fair. What’s the probability it comes out heads? The answer to all of these is exactly the same - 50% - even though you have wildly different amounts of knowledge about each. This is because 50% isn’t a description of how much knowledge you have, it’s a description of the balance between different outcomes. Is it bad that one term can mean both perfect information (as in 1) and total lack of information (as in 3)? No. This is no different from how we discuss things when we’re not using probability. Do vaccines cause autism? No. Does drinking monkey blood cause autism? Also no. My evidence on the vaccines question is dozens of excellent studies, conducted so effectively that we’re as sure about this as we are about anything in biology. My evidence on the monkey blood question is that nobody’s ever proposed this and it would be weird if it were true. Still, it’s perfectly fine to say the single-word answer “no” to both of them to describe where I currently stand. If someone wants to know how much evidence/certainty is behind my “no”, they can ask, and I’ll tell them. Likewise, is there a God? Maybe you ask the world’s top philosopher of religion, who has spent his entire life thinking about this question, and he says “I’m not sure”. Then you ask a random teenager who has given it two seconds’ thought, and she also says “I’m not sure”. Neither of these people has done anything wrong. Their identical answers conceal a vastly different amount of thought that’s gone into the question. But it’s your job to ask each person how much thought they put in, not the job of the English language to design a way of saying the words “I’m not sure” that communicates level of effort and expertise. Likewise, if I answer there’s a 0.001% chance vaccines cause autism, and a 0.001% chance monkey blood causes autism, it’s not the job of probability theory to tell you how much effort went into that assessment and how much of an expert I am. If you care about that, you can ask me! 3. What Is Samotsvety Better Than You At? Samotsvety Forecasting is a team of some of the top forecasters in the world. Their job is to assign probabilities to future events. They seem very good at it. They win forecasting contests. They make lots of money on prediction markets. They get featured in media articles. Sometimes people hire them as consultants when they have some forecasting question relevant to their business. Sometimes some client will ask Samotsvety for a prediction relative to their business, for example whether Joe Biden will get impeached, and they will give a number like “it’s 17% likely that this thing will happen”. This number has some valuable properties: It’s well-calibrated. Things that they assign 17% probability to will happen about 17% of the time. If you randomly change this number (eg round it to 20%, or invert it to 83%) you will be less well-calibrated.
Inline links: get featured in media articles
If you were someone whose job involved adjusting for situations like these - for example, an oil trader who thought that Joe Biden getting impeached would change oil prices - you would do better at your job, over the long run, by treating it as 17% likely that Joe Biden would be impeached then by guessing randomly, or consistently taking some number 5% higher than Samotsvety’s number, or using some other method. If there were two oil companies, and one consistently used Samotsvety’s numbers to make decisions, and the other consistently took a random number 1-100 and made decisions with that, the one that used Samotsvety’s numbers would do better.
In other words, there’s something special about the number 17% on this question. It has properties that other numbers like 38% or 99.9999% don’t have. If someone asked you (rather than Samotsvety) for this number, you would give a less good number that didn’t have these special properties. If by some chance you actually were better at finding these kinds of numbers than Samotsvety, you could probably get a job as a forecasting consultant. Or you could make lots of play money on Manifold, or lots of real money on the stock market, or help your preferred political party as a campaign strategist.