Game

Entities classified as game within this archive.

Reference Index

Use the title to open the reference entry. Use the caret to expand a compact inline dossier with source context, issue trail, related pages, and outbound links.

Civilization

Civilization is a recurring game in the Astral Codex Ten archive, appearing 2 times across 2 issues between August 25, 2022 and May 15, 2025. The archive places it in contexts such as "imagine you have two options: play Civilization and lose"; "play Civilization and lose"; "playing classy games, like Civilization". It most often appears alongside ACX survey, AGI, AI-risk.

Article page
Civilization
Mention count
2
Issue count
2
First seen
August 25, 2022
Last seen
May 15, 2025
August 25, 2022 · Original source
For an example of other choices that work kind of like this - imagine you have two options: play Civilization and lose, or go to a moderately interesting museum. It's hard to say that one of these options is better than the other, so you might as well treat them as equal. But now suppose that you also have the option of playing Civ and winning. That's presumably more fun than losing, but it's still not clearly better than the museum, so now "play Civ and win" and "museum" are equal, while "play Civ and lose" is eliminated as an inferior choice.
10: Siberian Fox writes: astralcodexten.substack.com/p/book-review-…\n\nbut I still disagree. I'm open to being wrong because it means I get my eyes pecked by seagulls, but I do believe a galactic civilization with trillions of barely worth living meh lives > a bubble utopia of 5000 people around wasteland","username":"SilverVVulpes","name":"Siberian fox","profile_image_url":"","date":"Wed Aug 24 18:57:30 +0000 2022","photos":[],"quoted_tweet":{},"reply_count":0,"retweet_count":0,"like_count":3,"impression_count":0,"expanded_url":{"url":"https://astralcodexten.substack.com/p/book-review-what-we-owe-the-future","image":"https://substackcdn.com/image/fetch/w_1200,h_600,c_limit,f_jpg,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F051d8840-77a5-4dfa-8a7c-fe6cd0c626da_535x382.png","title":"Book Review: What We Owe The Future","description":"...","domain":"astralcodexten.substack.com"},"video_url":null,"belowTheFold":true}" data-component-name="Twitter2ToDOM">
I’m also sympathetic to the galactic civilization, but only because it’s glorious. This is different from “it has a lot of people experiencing mild contentment”.
May 15, 2025 · Original source
When I try to apply SRTHMK’s lessons, I can’t deny that I’m being exactly the kind of hypocrite who says that my generation was okay but the next generation is destroying society. I chafed against all of my parents’ stupid computer use restrictions as a teenager - why couldn’t they understand that I was only playing classy games, like Civilization, and hanging out on decent sites, like LiveJournal? Now it’s twenty years later and…
Baldur's Gate 3

Baldur's Gate 3 is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "Baldur's Gate 3". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Banerjee.

Reference entry
Baldur's Gate 3
Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...S Census vt/txt-convergence We Should Never Have Gone To Mammoth Caves Which Sports? Why Sports? Summer Camp For Sluts / Young Swingers' Week Zermelo-Fraenkel Set Theory Baldur's Gate 3 Beating Balatro Call of Duty 4: Modern Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Game...
Call of Duty 4: Modern Warfare

Call of Duty 4: Modern Warfare is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "Call of Duty 4: Modern Warfare". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...hould Never Have Gone To Mammoth Caves Which Sports? Why Sports? Summer Camp For Sluts / Young Swingers' Week Zermelo-Fraenkel Set Theory Baldur's Gate 3 Beating Balatro Call of Duty 4: Modern Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many S...
Civilization IV

Civilization IV is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between July 30, 2024 and July 30, 2024. The archive places it in contexts such as "I do really enjoy playing Civilization IV . And the basic structure of Civilization IV is". It most often appears alongside /r/iamverysmart, 4chan, Achilles.

Reference entry
Civilization IV
Mention count
1
Issue count
1
First seen
July 30, 2024
Last seen
July 30, 2024
July 30, 2024 · Original source
I have no real answer to this question - which, in case you missed it, is “what is the meaning of life?” But I do really enjoy playing Civilization IV. And the basic structure of Civilization IV is “you mine resources, so you can build units, so you can conquer territory, so you can mine more resources, so you can build more units, so you can conquer more territory”. There are sidequests that make it less obvious. And you can eventually win by completing the tech tree (he who has ears to hear, let him listen). But the basic structure is A → B → C → A → B → C. And it’s really fun! If there’s enough bright colors, shiny toys, razor-edge battles, and risk of failure, then the kind of ratchet-y-ness of it all, the spiral where you’re doing the same things but in a bigger way each time, turns into a virtuous repetition, repetitive only in the same sense as a poem, or a melody, or the cycle of generations.
Civilization IV: Fall From Heaven

Civilization IV: Fall From Heaven is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between January 07, 2022 and January 07, 2022. The archive places it in contexts such as "I agree this game is not as fun as, say, Civilization IV: Fall From Heaven". It most often appears alongside ACX Discord, Aerojet XLR-132, Aimable.

Mention count
1
Issue count
1
First seen
January 07, 2022
Last seen
January 07, 2022
January 07, 2022 · Original source
Look, there’s a weird game called “movie criticism”, where you take a movie as a jumping-off point to have thoughts on Society or the Human Condition. In the real world, people watch movies because they’re funny, or they have cool action sequences, or because the lead actress is really hot. But the rules of the “movie criticism” game say you have to ignore this stuff and treat them as deep commentary. I agree this game is not as fun as, say, Civilization IV: Fall From Heaven. But I have deliberately limited the amount of time I play that game for the sake of my sanity and my career, which means I need to play other games, and the “movie criticism” game seems okay.
Disco Elysium

Disco Elysium is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "Disco Elysium (1, by EH) Disco Elysium (2, by DC)". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Reference entry
Disco Elysium
Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...Sports? Summer Camp For Sluts / Young Swingers' Week Zermelo-Fraenkel Set Theory Baldur's Gate 3 Beating Balatro Call of Duty 4: Modern Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop...
...Sluts / Young Swingers' Week Zermelo-Fraenkel Set Theory Baldur's Gate 3 Beating Balatro Call of Duty 4: Modern Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The...
...Sports? Summer Camp For Sluts / Young Swingers' Week Zermelo-Fraenkel Set Theory Baldur's Gate 3 Beating Balatro Call of Duty 4: Modern Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us...
Final Fantasy

Final Fantasy is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between August 16, 2023 and August 16, 2023. The archive places it in contexts such as "My favorite video games are ... Final Fantasy". It most often appears alongside Alice, Attack On Titan, Bali.

Reference entry
Final Fantasy
Mention count
1
Issue count
1
First seen
August 16, 2023
Last seen
August 16, 2023
August 16, 2023 · Original source
I’m Jane. My favorite animes are Full Metal Alchemist, Attack On Titan, My Hero Academia, Code Geass, Neon Genesis Evangelion, Gurren Lagann, and Fate: Stay Night. My favorite video games are Super Smash Bros, Final Fantasy, Stardew Valley, Minecraft, and Fortnite. I’m really shy and don’t leave the house a lot but my family says I should get more into dating. Let me know if you want to hang out and play something and get to know each other better.
Fortnite

Fortnite is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between August 16, 2023 and August 16, 2023. The archive places it in contexts such as "My favorite video games are ... Fortnite". It most often appears alongside Alice, Attack On Titan, Bali.

Reference entry
Fortnite
Mention count
1
Issue count
1
First seen
August 16, 2023
Last seen
August 16, 2023
August 16, 2023 · Original source
I’m Jane. My favorite animes are Full Metal Alchemist, Attack On Titan, My Hero Academia, Code Geass, Neon Genesis Evangelion, Gurren Lagann, and Fate: Stay Night. My favorite video games are Super Smash Bros, Final Fantasy, Stardew Valley, Minecraft, and Fortnite. I’m really shy and don’t leave the house a lot but my family says I should get more into dating. Let me know if you want to hang out and play something and get to know each other better.
Mega Man

Mega Man is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between September 28, 2023 and September 28, 2023. The archive places it in contexts such as "Thirty years after the Mega Man video games came out". It most often appears alongside 2020 election, @eigenrobot, @jeremychrysler.

Reference entry
Mega Man
Mention count
1
Issue count
1
First seen
September 28, 2023
Last seen
September 28, 2023
September 28, 2023 · Original source
Thirty years after the Mega Man video games came out, we’ve finally invented the low-level robot enemies you have to kill a few dozen of before you get to the main boss! But (unlike Mega Man enemies) these don’t have weapons, so I wonder what the advantage is supposed to be over having lots of CCTV cameras. Maybe psychological?
Minecraft

Minecraft is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between August 16, 2023 and August 16, 2023. The archive places it in contexts such as "My favorite video games are ... Minecraft". It most often appears alongside Alice, Attack On Titan, Bali.

Reference entry
Minecraft
Mention count
1
Issue count
1
First seen
August 16, 2023
Last seen
August 16, 2023
August 16, 2023 · Original source
I’m Jane. My favorite animes are Full Metal Alchemist, Attack On Titan, My Hero Academia, Code Geass, Neon Genesis Evangelion, Gurren Lagann, and Fate: Stay Night. My favorite video games are Super Smash Bros, Final Fantasy, Stardew Valley, Minecraft, and Fortnite. I’m really shy and don’t leave the house a lot but my family says I should get more into dating. Let me know if you want to hang out and play something and get to know each other better.
Monopoly

Monopoly is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between April 16, 2021 and April 16, 2021. The archive places it in contexts such as "inspiring a popular board game that was shamelessly ripped off and repackaged as Monopoly". It most often appears alongside "The Rent Is Too Damn High!", 16th amendment, 1886.

Reference entry
Monopoly
Mention count
1
Issue count
1
First seen
April 16, 2021
Last seen
April 16, 2021
April 16, 2021 · Original source
Henry George is variously known for leading an early movement that popularized Universal Basic Income, sporting a fancy beard while shouting "The Rent Is Too Damn High!" and inspiring a popular board game that was shamelessly ripped off and repackaged as Monopoly.
By George, this is wealth. Digital though it may be, it's physically encoded on a storage device somewhere, and is thus tangible (it's not a pure abstract concept flitting about in Platonic heaven) and has its origins in nature. Human exertion built the computer that encodes it, and clicking the button that saves it to disk or displays it on your screen is labor. Finally, it directly satisfies human desires (mine, at the very least). It's value may be negligible, but it's wealth. By contrast, the digital bit sitting in some database that says I own a particular eBook or mp3 is just a digital IOU – a claim on the wealth that are the physical bits on my local storage device or remote server that digitally encodes the files. The fact that digital files don't seem particularly physical, and that they can be trivially and endlessly copied, doesn't mean that Henry George, magically transported to today, wouldn't regard them as wealth. Okay, so is there anything else that's not wealth? By George, Bitcoin isn't wealth, in case you were wondering. It's just a (very fancy) financial instrument, a digital claim on wealth. And that goes for most crypto assets – a token on some blockchain that says I own a painting by Banksy is just another IOU, regardless of the technical sophistication of its distributed trustless ledger. What about intellectual property? Copyrights, patents, and trademarks are all different forms of Monopoly – the exclusive, government-granted legal right to do a particular thing (publish a certain book, manufacture a certain product, use a certain name in business, etc). The exclusive right to do or produce a thing, valuable as it may be, is not the thing itself. By George, Monopoly is not wealth. But there is something big that is wealth – the C-word. Capital. By George, Capital is "wealth devoted to procuring more wealth", and it's the next thing he insists everyone is hopelessly confused about. He quotes Adam Smith, agreeing with him thus far: That part of a man's stock which he expects to afford him revenue is called his capital. ...and also gives us a short etymology lesson on the origin of the term: The word capital, as philologists trace it, comes down to us from a time when wealth was estimated in cattle, and a man's income depended upon the number of head he could keep for their increase. ("Per capita" being the Latin for "by head") By George, all capital is wealth, but not all wealth is capital. George notes capital is often described as being "stored up labor", and endorses this view – but what it really means, is capital is stored up production. It's not literally the labor that's stored up but the wealth generated by it, set aside and then dedicated to the purpose of getting more wealth. George insists that it is the owner's intention that transforms wealth into capital. If you buy an old factory to throw parties in for your hipster friends, it's just wealth. But the minute you decide to put it to work to make something useful (or start charging your hipster friends a cover charge at the door), it becomes capital. George therefore further insists that a laborer's daily bread and the clothes on their back do not count as capital, because a person has to eat and wear clothes whether they work or not. The laborer's tools (and arguably their steel-toed work boots) can however be counted as capital, because their purpose is to assist the laborer in getting more wealth by working for wages, and the laborer wouldn't acquire, use, and maintain those things otherwise. George has more exclusions: We must exclude from the category of capital everything that may be included either as land or labor. Human exertion (labor) by itself can never be capital. The products of human labor become capital when they are stored up and set to the purpose of getting more wealth. To muddle this distinction defeats the point of having separate terms for those things at all, and prevents us from reasoning meaningfully about how they relate to one another. Labor is not capital, and neither is labor by itself wealth, it produces wealth – and if it ain't wealth, it ain't capital. And that brings us to land. Land, land, land. By George, land is not wealth. And it's definitely not capital. The unique specialness of land is George's entire schtick and the very core of his philosophy. The term land embraces, in short, all natural materials, forces, and opportunities That means that a field or a meadow is "land", as is a mountain. But so are the fish in the sea, the clouds in the sky, veins of gold in the earth's crust, and the oil deep under ground. These things aren't yet wealth – not until human beings both a) desire them and b) touch them with labor. So... land is not wealth. But... how come? I mean, look: land is tangible, it "comes from nature", humans are always productively applying their labor to it, and it certainly seems capable of gratifying human desires. George sees this reasoning as understandable, but insists it's the root mistake that leads other political economists astray – because for George, land just is nature itself. Come again? Land is the ultimate source of all wealth, but it's most useful to think of it as a generator, acompletely separate entity from the wealth that human labor and desire draws from it. Players of Magic: the Gathering and Settlers of Catan should already have a solid grasp of this distinction: In modern times, George would grant electromagnetic spectrum and orbital real estate for satellites the same status of "land" that already applies to farmland and terrestrial real estate. We don't even need to speculate about whether he'd attach this status to sunlight because he straight-up predicted solar power: Even the lack of rain which makes some parts of the globe useless to man, may, if invention ever succeeds in directly utilizing the power of the sun's rays, be found to be especially advantageous for certain parts of production. (That's from Protection or Free Trade, footnote 19) The important thing to grasp about land is that it comes before everything humans do or make, and is itself a thing no human can make. Okay, smarty-pants, what about the Netherlands? They've been making land for centuries! Well, land in the Georgist sense doesn't refer simply to "dry land", but also the sea bed, the oceans, and the skies above. The "new land" in the Netherlands counts as an improvement to land that already existed. The seabed was always there, but by filling it in so you can walk around on it, now it's more useful to us (George has a lot to say about improvements to land, which we'll get to later). Okay, what is land not? nothing that is freely supplied by nature can be properly classed as capital By George, land is not wealth. And since it's not wealth, it's not capital. Okay, we get it. Land is very special to Mr. George and we must never put it in the same category as wealth, labor, capital, wages, production, money, or anything else. Why exactly is this so damn important? Well, by George, if you treat land the same way you would a bar of pig iron, an hour of work, or a dollar bill, before you know it you'll get poverty paradoxically advancing alongside progress, inexplicable bouts of industrial depression, literal genocides and holocausts (he's dead serious about this), and The Rent Being Too Damn High. With terminology now firmly established, George moves on to the relationship between wages and capital. 3-for-1 special on Wages, Capital, and Labor I'm condensing three chapters here because they all deal with the same basic thing. The question George wants to answer is: Why, in spite of increase in productive power, do wages tend to a minimum which will give but a bare living? The conventional wisdom of George's time is that wages are governed by a fixed ratio between the number of laborers and the amount of capital devoted to their employment, because "the increase in the number of laborers tends naturally to follow and overtake any increase in capital." So it doesn't matter how much capital you throw at employing workers, it'll just attract even more workers splitting it up, so although wages might temporarily wiggle a bit in the long term they'll always settle back to a "natural" minimum. (As we'll see in the next section, this argument stems from Malthusianism). George spends some time methodically poking holes in the theory (it's predictions don't line up with the facts he observes), and then sets out to prove his replacement theory (emphases mine): wages, instead of being drawn from capital, are in reality drawn from the product of the labor for which they are paid. He pulls a G.K. Chesterton to make his point: During the time [the laborer] is earning the wages he is advancing capital to his employer, but at no time, unless wages are paid before work is done, is the employer advancing capital to him. He starts by identifying the source of confusion: Because wages are generally paid in money, and in many of the operations of production are paid before the product is fully completed, or can be utilized, it is inferred that wages are drawn from pre-existing capital I mean, the old theory seems sensible: the employer has capital and uses it to pay wages. But however you slice it, capital's investment gets paid back by production when it takes its cut, so does it even make a difference to talk about where wages are "drawn" from? Value goes out, value comes in, isn't it all a wash? By George, it isn't: in the old theory, because capital "must come first", it follows that "industry is limited by capital - that capital must be accumulated before labor is employed", which leads to a reductio ad absurdum – We are told that capital is stored-up or accumulated labor – "that part of wealth which is saved to assist future production." If we substitute for the word "capital" this definition of the word, the proposition carries its own refutation, for that labor cannot be employed until the results of labor are saved becomes too absurd for discussion. George anticipates the following rejoinder – Well, when we say 'labor is paid out of capital' we don't mean it as an absolute statement for all stages of human development (or else we have a chicken-and-the-egg problem and civilization could never have begun), we just mean it applies to, say, every civilization that's left the stone age. George will have none of it and spends three entire chapters relentlessly beating to death the idea that wages are drawn from capital instead of from production. He starts with the simple case where wages are paid in the form of direct, concrete wealth, then moves on to the more complex case where people are paid in money and other instruments. Laboring for wages: Imagine a fishing village where nobody cooperates – each person digs their own bait and catches their own fish. Then they discover labor specialization and realize they can catch more fish together if one specializes in digging and the other in catching. So the digger digs, the catcher catches, and they share the fish. The digger really contributes as much to the catch as the one who physically pulls the fish off the hook even though the digger never directly "caught" a fish, and the fish he gets for his work is directly paid out of his contribution to the total production. Later, our fisherfolk invent canoes, and one stays home making and repairing canoes. This increases the haul of the digger and catcher, and the canoe-er gets paid out of her contribution to the increased production. And so it goes as society continues to advance. The work the specialist puts in causes more fish to be caught, and that person's wages is drawn from the growing pile of fish. As George puts it: "Earning is making." George gives another example: If I take a piece of leather and work it up into a pair of shoes, the shoes are my wages – the reward of my exertion. Surely they are not drawn from capital – either my capital or any one else's capital – but are brought into existence by the labor of which they become the wages; and in obtaining this pair of shoes as the wages of my labor, capital is not even momentarily lessened one iota... As my labor goes on, value is steadily added, until, when my labor results in the finished shoes, I have my capital plus the difference in value between the material and the shoes. And another: If I hire a man to gather eggs, to pick berries, or to make shoes, paying him from the eggs, the berries, or the shoes that his labor secures, there can be no question that the source of the wages is the labor for which they are paid. George goes on to say it doesn't matter if you're paid in money or directly in wealth, because the money is a direct claim on the underlying wealth. It also doesn't matter if you get paid on commission. Imagine a whaling ship where each crewman gets paid a share out of whatever the ship catches. When the ship sails back into port with a hold full of whale oil and bone, the crew gets paid in money, the owner simultaneously adds to his capital oil and bone. The crew's money directly represents their share of the concrete wealth that is the oil and bone. The owner's capital hasn't decreased, and the workers drew their wages directly from the production. So let's get to the point, Mr. George – wages aren't drawn from capital but instead from production. Great, let's grant that – so what? George hammers away at this because thinking wages are drawn from capital leads to a false conclusion, namely that "labor cannot exert its productive power unless supplied by capital with maintenance." "Maintenance?" Well, workers need food and clothing and they get paid by their employers, so you could imagine capital as a limiting factor on labor. But by George, food and clothing isn't capital, it's just wealth, as we said before. And with regard to wages, the point is that the employer always gets "paid" first, because the second the laborer produces value, the employer's capital increases: As in the exchange of labor for wages the employer always gets the capital created by the labor before he pays out capital in the wages, at what point is his capital lessened even temporarily? Okay, but what if I'm just a terrible businessman and I pay somebody $500 an hour to smash Ming vases, then sell the fragments as aggregate to a construction crew for a few pennies a pound, all at a tremendous loss? Surely then the laborer's wages must be drawn from my capital, because there's not enough productive value generated by the labor to draw them from! George says okay, sure, but only because I'm an idiot and will soon be out of business: Yet, unless the new value created by the labor is less than the wages paid, which can be only an exceptional case, the capital which he had before in money he now has in goods – it has been changed in form, but not lessened. Fair enough, Mr. George, but what if I'm building some enormously expensive multi-decade project, like a dam or a nuclear power plant or a cathedral? The kind of thing we call a "capital-intensive" project? What do you have to say to that? George points out that as laborers labor, they progressively add value to whatever they're producing. Take the case of a shipwright building ships for an employer – even if the boss can't sell a half-finished ship, it still holds value (for one, it costs less to finish a half-finished ship then no ship at all). And with every stroke of the laborer's work, the employer who owns the shipyard gets an incremental increase in his stock of capital. It is not the last blow, any more than the first blow, that creates the value of the finished product – the creation of value is continuous, it immediately results from the exertion of labor. A pedant would point out that the "last hit" that finishes the product which makes it ready for market adds disproportionate value, but George's point is just to establish that value is continuously created, and doesn't magically come into being allat once right at the end. George further points out that if you look at things like agriculture you'll see the market directly acknowledging his theory: As a plowed field will bring more than an unplowed field, or a field that has been sown more than one merely plowed... It is tangible in the case of orchards and vineyards which, though not yet in bearing, bring prices proportionate to their age. George freely admits that capital can be required for certain kinds of work, but he disagrees with what its purpose is. It's not a pool that wages get paid out of. He goes on for another chapter on "The Maintenance of Laborers Not Drawn From Capital" but I think we can safely skip it and move on. TL:DR – George hammers to absolute death the idea that Laborers derive their own maintenance (food/shelter/clothing/etc) from their wages, with George insisting it is drawn from production and... you guessed it, not from capital. At least some of George's ideas will not seem so radical to modern readers (especially those already critical of capitalism or neoclassical economics), but it's important to understand that at the time almost everything he was saying was considered deeply radical and shocking. Capital was the fundamental driving force of the economy and labor was utterly dependent on it, and the Malthusian theory of overpopulation was the accepted explanation for why wages were low and workers were starving. Political Cartoon literally demonizing Henry George – Puck magazine Oct. 20, 1886 The Real Functions of Capital Okay, Mr. George. You've spent three whole chapters beating me over the head with what the functions of capital aren't. So what are the functions of capital? Capital "increases the power of labor to produce wealth." How? By enabling labor to apply itself more effectively (power tools go brrrr)
"Capital is not a necessary factor in production" Therefore, we should always put land first in all our inquiries rather than capital, which ought to come last. George then sets out his three laws of distribution. The Law of Rent Let's be careful about the word "Rent." In modern usage, there is the concept of "Economic Rent" as well as "Rent" in the everyday sense of regular payments you make in exchange for the use of something that you are "renting." The modern definition of "Economic Rent", per Wikipedia is: economic rent is any payment ... to an owner or factor of production in excess of the costs needed to bring that factor into production To be clear, Economic Rent is a bad thing – all taking, no giving. When George uses the word "Rent", he specifically means the return to land, and this is what he says it is: Rent, in short, is the share in the wealth produced which the exclusive right to the use of natural capabilities gives to the owner. Land has zero cost of production because it's already there and you can't make it. This means that any payment or benefit you can realize by excluding others from using land (or its fruits) is necessarily "in excess of the costs needed to bring that factor into production." By George, all land rent is Economic Rent. Furthermore, any piece of land has only one seller, and no producers. This further meets the definition of Monopoly– Greek for "one seller." This is why you hear Georgists talking about "Land Monopoly." Land has value because people are willing to pay you for the privilege of using it. The price of rent derives from the most marginal land available. I'll explain with an example. Let's grade some imaginary lots according to their productivity by using abstract utility points, or "utils". Lot A is good fertile land worth 100 utils. Lot B is just as good, also worth 100 utils. Lot C is crappy land worth 10 utils. Let's say I own Lot A. I won't be able to charge you any rent to work on Lot A, if Lot B is freely available for anyone to use. Why would you pay even 1 util worth of rent if you could just work on Lot B, earn 100 utils, and keep it all? But once I buy Lot B, now if you want access to 100-util Land you have to pay me. How much can I charge? Well, you could always work on Lot C for free, and it'll yield 10 utils. So the most I can charge is 90 utils (100 - 90 = 10). So here's the Law of Rent – rent is determined by the "margin of production" (AKA the "margin of cultivation") – the difference between how much you can produce from a particular piece of land (Lot A or B) compared to the least productive alternative (Lot C). Notice that I as the landlord am not really doing anything here other than owning the land, and yet I can extract a huge amount of value, because unlike capital, land is a hard limit on labor – you can't work without a place to work or without material that comes from nature. And so I take my share first without really contributing anything to production other than gatekeeping access to land. Rent, in short, is the price of monopoly, arising from the reduction to individual ownership of natural elements which human exertion can neither produce nor increase. C'mon, is land really such a big deal? In the popular imagination we pit "capitalists" against "laborers" but a lot of those "capitalists" are landowners in disguise, because in non-Georgist frameworks land is typically considered a kind of capital. George says landowners oppress both labor and capital, cheating both hard work and investment out of their fair share. Source: can't find the author of this image, closest I can get to its origin is this blog Okay, but is this still relevant in the modern age, with the internet and work-from-home? Obsessing about land just feels so 19th century. Well, in Silicon Valley rents are famously off the charts, and those and all other rents seep into the economy at every level. Workers priced out of living close by have to spend more time and money commuting longer distances to work, and businesses must devote an increasingly larger share of their production to landowners who aren't actively contributing anything to productivity. What else could explain how a family of four making $100,000 in San Francisco is considered to be living below the poverty line? Here, take a look at this chart (source): I found this in a tweet by Thomas Piketty, and it shows the breakdown of personal assets in Spain over the last 100+ years. The bulk of the value of personal assets is from landownership. This is still the case even though the chart includes "financial assets" – which are just IOUs that ultimately have something real (e.g. land or wealth) underpinning their value. If we exclude those, the true portion of overall value represented by land is even higher than this graph first implies. And this isn't just Spain. Here's a graph Nate Blair made for the UK, excluding all financial instruments and only looking at real assets: Based on data from the United Kingdom National Accounts: The Blue Book 2017. Published Oct 31, 2017. Revision Period: Beginning of each time series. Date of next release: July 2018. The "privileges" in "Land and privileges" are things like taxi medallions and patents, that were worth "almost zero" according to Nate. No matter how hard you try, "there is no occupation in which labor and capital can engage which does not require the use of land." Whenever anyone does labor, the owner of some piece of land – whether it's the farm in the middle of Kansas that grows your food, the lot upon which the server farm sending you these bytes sits, or the ground that right now sits beneath your feet – is sticking their finger in the pie. George reminds us that labor and capital will have to share whatever landowners take off the top of production in rent: As Produce = Rent + Wages + Interest, Therefore, Produce - Rent = Wages + Interest So... what happens when the productivity of land goes up? Let's go back to Lot A and Lot B, both 100-util fields. Let's say they belong to different landlords, and I'm a tenant on Lot B. I improve the soil of the field I'm working on so now it's worth 110 utils. What happens? My landlord raises the rent, of course! The only way wages (the return to labor) and interest (the return to capital) can go up as productivity increases, is if land values fail to rise at the same rate. The Law of Interest George wants to find the fundamental reason capital is able to produce wealth and justly claim a fair share of production. Remember that capital is wealth devoted to getting more wealth. So if capital is wealth that begets wealth, it makes sense that if I lend it out to you, I miss out on the potential for it to grow while it's out of my hands. George says I am justly entitled to ask for more back than I originally gave you. Let's say I loan you some corn seeds for a season. Had I not leant them to you, in a season's time I could have grown my own crop of corn and been left with more seed than I started with. So in a perfectly square deal, you need to give me back what I started with and what I could have expected to gain from natural increase (less the value of the labor required to get things started). Likewise with any other article of capital – say bricks or lumber. In the time I've spent without it while it was in your possession, I could have found someone else who had a better use for it than I did and exchanged it for something of theirs that I had a better use for, leaving me with capital of greater value. George says the act of progressively exchanging things in a way that increases subjective value for all involved is analogous to the natural forces of nature that make living capital (like corn and cows) grow over time. Remember, "subjective value" is real value. In a game of Settlers of Catan, if I have two bricks and you have two lumber, neither of us can build anything. The simple act of trading one brick for one lumber means both of us are better off because each of us can now build a road. The amount of bricks and lumber in the world didn't increase, but the amount of roads (or potential roads) did, and that represents a real increase in wealth. Interest thus springs from the "reproductive" powers of capital, whether that's biological reproduction, or the more abstract reproductive force of exchanging things so that you have a more valuable distribution of capital than you started with. As for how it relates to the other two returns to production – the more powerful the "power of increase" the capital has, the greater return interest can claim compared to wages. If you're ploughing a field and I lend you a tractor which makes you ten times as productive, I can justly claim more compensation for that than if I lend you a mule that only makes you twice as productive. However, rent still holds the whip hand, so the margin of cultivation determines how much return is left over to divvy up between interest and wages. This is because the net "reproductive" value of capital goes down given rent is a general tax on overall productivity. The amount I would have gained by using the thing productively over the period of time it was out on loan (the amount I can justly charge in interest) is reduced by how much I have to pay in rent. The Law of Wages Wages, like interest, are limited by the margin of production. Within that limit there's not much to understand about how wages work except that people seek to satisfy their desires "with the least exertion," which is a fancy way of saying people don't like to get ripped off. If two bosses offer the same exact job, but one offers higher pay, I'm taking that gig. If two bosses pay the same, but one is asking for twice as much work, I'll tell that boss where he can stick it. Wages depend upon the margin of production, or upon the produce which labor can obtain at the highest point of natural productiveness open to it without the payment of rent. So with all three laws established George sums it up like so: Where land is free and labor is unassisted by capital, the whole produce will go to labor as wages. Where land is free and labor is assisted by capital, wages will consist of the whole produce, less that part necessary to induce the storing up of labor as capital. Where land is subject to ownership and rent arises, wages will be fixed by what labor could secure from the highest natural opportunities open to it without the payment of rent. Where natural opportunities are all monopolized, wages may be forced by the competition among laborers to the minimum at which laborers will consent to reproduce. This is the reason George says that wages are so high in "new countries" where there's more land available than in countries where it's been locked up for centuries. Here's how it all fits together: Though neither wages nor interest anywhere increase as material progress goes on, yet the invariable accompaniment and mark of material progress is the increase of rent – the rise of land values. And: where the value of land is highest, civilization exhibits the greatest luxury side by side with the most piteous destitution IV. Effect of Material Progress upon the Distribution of Wealth As a society undergoes material progress, the rent goes up. Why? Let's break it down. Three things contribute to material progress: Increasing population
Mountaintop Pathologic Classic HD

Mountaintop Pathologic Classic HD is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "Mountaintop Pathologic Classic HD". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...isco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klavierstucke Deathbed Ballads Joanna Newsom: The Lyric Si...
Pokemon Gold

Pokemon Gold is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between February 03, 2022 and February 03, 2022. The archive places it in contexts such as "An example is Pokemon Gold, which basically taught me English". It most often appears alongside 538, 55-gal drum, 750k horny men.

Reference entry
Pokemon Gold
Mention count
1
Issue count
1
First seen
February 03, 2022
Last seen
February 03, 2022
February 03, 2022 · Original source
#26: Gamify Education Right I am Martijn Struijs, 4/5 years PhD student and TA in Computational Geometry at Eindhoven University of Technology. My proposal is to do gamification of education right. Most attempts at gamified education start with a fixed educational program and try to let a game meet these standards. That is a terrible way to design a game. Some games were not made for education, yet have been educational. An example is Pokemon Gold, which basically taught me English. You have experienced this personally as well, in your game in another world. These games have a low "skill floor", i.e. it doesn't take much skill to play the game, and also a high "skill ceiling": playing it well requires great skill. These conditions are excellent for growth and learning. For an example of an exceptional yet not well known educational game, look at ZeroRanger. Many of the skills it teaches, mostly patience, focus, recovering from setbacks, and letting go, are transferable to other aspects of life. I believe that an educational tool should teach one thing well, whatever it is, and hope that the thing it teaches is useful (if not, throw it away and try again) I already have the resources and am developing such a game. What I don't have is social science experience to test the effectiveness of the game. I could use your support here. Most of the development costs will be paying people, this is minimal at this stage. I thank you for your consideration. May we achieve enlightenment. [You can reach me at struijsmartijn@gmail.com]
Settlers of Catan

Settlers of Catan is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between April 16, 2021 and April 16, 2021. The archive places it in contexts such as "In a game of Settlers of Catan, if I have two bricks and you have two lumber". It most often appears alongside "The Rent Is Too Damn High!", 16th amendment, 1886.

Reference entry
Settlers of Catan
Mention count
1
Issue count
1
First seen
April 16, 2021
Last seen
April 16, 2021
April 16, 2021 · Original source
By George, this is wealth. Digital though it may be, it's physically encoded on a storage device somewhere, and is thus tangible (it's not a pure abstract concept flitting about in Platonic heaven) and has its origins in nature. Human exertion built the computer that encodes it, and clicking the button that saves it to disk or displays it on your screen is labor. Finally, it directly satisfies human desires (mine, at the very least). It's value may be negligible, but it's wealth. By contrast, the digital bit sitting in some database that says I own a particular eBook or mp3 is just a digital IOU – a claim on the wealth that are the physical bits on my local storage device or remote server that digitally encodes the files. The fact that digital files don't seem particularly physical, and that they can be trivially and endlessly copied, doesn't mean that Henry George, magically transported to today, wouldn't regard them as wealth. Okay, so is there anything else that's not wealth? By George, Bitcoin isn't wealth, in case you were wondering. It's just a (very fancy) financial instrument, a digital claim on wealth. And that goes for most crypto assets – a token on some blockchain that says I own a painting by Banksy is just another IOU, regardless of the technical sophistication of its distributed trustless ledger. What about intellectual property? Copyrights, patents, and trademarks are all different forms of Monopoly – the exclusive, government-granted legal right to do a particular thing (publish a certain book, manufacture a certain product, use a certain name in business, etc). The exclusive right to do or produce a thing, valuable as it may be, is not the thing itself. By George, Monopoly is not wealth. But there is something big that is wealth – the C-word. Capital. By George, Capital is "wealth devoted to procuring more wealth", and it's the next thing he insists everyone is hopelessly confused about. He quotes Adam Smith, agreeing with him thus far: That part of a man's stock which he expects to afford him revenue is called his capital. ...and also gives us a short etymology lesson on the origin of the term: The word capital, as philologists trace it, comes down to us from a time when wealth was estimated in cattle, and a man's income depended upon the number of head he could keep for their increase. ("Per capita" being the Latin for "by head") By George, all capital is wealth, but not all wealth is capital. George notes capital is often described as being "stored up labor", and endorses this view – but what it really means, is capital is stored up production. It's not literally the labor that's stored up but the wealth generated by it, set aside and then dedicated to the purpose of getting more wealth. George insists that it is the owner's intention that transforms wealth into capital. If you buy an old factory to throw parties in for your hipster friends, it's just wealth. But the minute you decide to put it to work to make something useful (or start charging your hipster friends a cover charge at the door), it becomes capital. George therefore further insists that a laborer's daily bread and the clothes on their back do not count as capital, because a person has to eat and wear clothes whether they work or not. The laborer's tools (and arguably their steel-toed work boots) can however be counted as capital, because their purpose is to assist the laborer in getting more wealth by working for wages, and the laborer wouldn't acquire, use, and maintain those things otherwise. George has more exclusions: We must exclude from the category of capital everything that may be included either as land or labor. Human exertion (labor) by itself can never be capital. The products of human labor become capital when they are stored up and set to the purpose of getting more wealth. To muddle this distinction defeats the point of having separate terms for those things at all, and prevents us from reasoning meaningfully about how they relate to one another. Labor is not capital, and neither is labor by itself wealth, it produces wealth – and if it ain't wealth, it ain't capital. And that brings us to land. Land, land, land. By George, land is not wealth. And it's definitely not capital. The unique specialness of land is George's entire schtick and the very core of his philosophy. The term land embraces, in short, all natural materials, forces, and opportunities That means that a field or a meadow is "land", as is a mountain. But so are the fish in the sea, the clouds in the sky, veins of gold in the earth's crust, and the oil deep under ground. These things aren't yet wealth – not until human beings both a) desire them and b) touch them with labor. So... land is not wealth. But... how come? I mean, look: land is tangible, it "comes from nature", humans are always productively applying their labor to it, and it certainly seems capable of gratifying human desires. George sees this reasoning as understandable, but insists it's the root mistake that leads other political economists astray – because for George, land just is nature itself. Come again? Land is the ultimate source of all wealth, but it's most useful to think of it as a generator, acompletely separate entity from the wealth that human labor and desire draws from it. Players of Magic: the Gathering and Settlers of Catan should already have a solid grasp of this distinction: In modern times, George would grant electromagnetic spectrum and orbital real estate for satellites the same status of "land" that already applies to farmland and terrestrial real estate. We don't even need to speculate about whether he'd attach this status to sunlight because he straight-up predicted solar power: Even the lack of rain which makes some parts of the globe useless to man, may, if invention ever succeeds in directly utilizing the power of the sun's rays, be found to be especially advantageous for certain parts of production. (That's from Protection or Free Trade, footnote 19) The important thing to grasp about land is that it comes before everything humans do or make, and is itself a thing no human can make. Okay, smarty-pants, what about the Netherlands? They've been making land for centuries! Well, land in the Georgist sense doesn't refer simply to "dry land", but also the sea bed, the oceans, and the skies above. The "new land" in the Netherlands counts as an improvement to land that already existed. The seabed was always there, but by filling it in so you can walk around on it, now it's more useful to us (George has a lot to say about improvements to land, which we'll get to later). Okay, what is land not? nothing that is freely supplied by nature can be properly classed as capital By George, land is not wealth. And since it's not wealth, it's not capital. Okay, we get it. Land is very special to Mr. George and we must never put it in the same category as wealth, labor, capital, wages, production, money, or anything else. Why exactly is this so damn important? Well, by George, if you treat land the same way you would a bar of pig iron, an hour of work, or a dollar bill, before you know it you'll get poverty paradoxically advancing alongside progress, inexplicable bouts of industrial depression, literal genocides and holocausts (he's dead serious about this), and The Rent Being Too Damn High. With terminology now firmly established, George moves on to the relationship between wages and capital. 3-for-1 special on Wages, Capital, and Labor I'm condensing three chapters here because they all deal with the same basic thing. The question George wants to answer is: Why, in spite of increase in productive power, do wages tend to a minimum which will give but a bare living? The conventional wisdom of George's time is that wages are governed by a fixed ratio between the number of laborers and the amount of capital devoted to their employment, because "the increase in the number of laborers tends naturally to follow and overtake any increase in capital." So it doesn't matter how much capital you throw at employing workers, it'll just attract even more workers splitting it up, so although wages might temporarily wiggle a bit in the long term they'll always settle back to a "natural" minimum. (As we'll see in the next section, this argument stems from Malthusianism). George spends some time methodically poking holes in the theory (it's predictions don't line up with the facts he observes), and then sets out to prove his replacement theory (emphases mine): wages, instead of being drawn from capital, are in reality drawn from the product of the labor for which they are paid. He pulls a G.K. Chesterton to make his point: During the time [the laborer] is earning the wages he is advancing capital to his employer, but at no time, unless wages are paid before work is done, is the employer advancing capital to him. He starts by identifying the source of confusion: Because wages are generally paid in money, and in many of the operations of production are paid before the product is fully completed, or can be utilized, it is inferred that wages are drawn from pre-existing capital I mean, the old theory seems sensible: the employer has capital and uses it to pay wages. But however you slice it, capital's investment gets paid back by production when it takes its cut, so does it even make a difference to talk about where wages are "drawn" from? Value goes out, value comes in, isn't it all a wash? By George, it isn't: in the old theory, because capital "must come first", it follows that "industry is limited by capital - that capital must be accumulated before labor is employed", which leads to a reductio ad absurdum – We are told that capital is stored-up or accumulated labor – "that part of wealth which is saved to assist future production." If we substitute for the word "capital" this definition of the word, the proposition carries its own refutation, for that labor cannot be employed until the results of labor are saved becomes too absurd for discussion. George anticipates the following rejoinder – Well, when we say 'labor is paid out of capital' we don't mean it as an absolute statement for all stages of human development (or else we have a chicken-and-the-egg problem and civilization could never have begun), we just mean it applies to, say, every civilization that's left the stone age. George will have none of it and spends three entire chapters relentlessly beating to death the idea that wages are drawn from capital instead of from production. He starts with the simple case where wages are paid in the form of direct, concrete wealth, then moves on to the more complex case where people are paid in money and other instruments. Laboring for wages: Imagine a fishing village where nobody cooperates – each person digs their own bait and catches their own fish. Then they discover labor specialization and realize they can catch more fish together if one specializes in digging and the other in catching. So the digger digs, the catcher catches, and they share the fish. The digger really contributes as much to the catch as the one who physically pulls the fish off the hook even though the digger never directly "caught" a fish, and the fish he gets for his work is directly paid out of his contribution to the total production. Later, our fisherfolk invent canoes, and one stays home making and repairing canoes. This increases the haul of the digger and catcher, and the canoe-er gets paid out of her contribution to the increased production. And so it goes as society continues to advance. The work the specialist puts in causes more fish to be caught, and that person's wages is drawn from the growing pile of fish. As George puts it: "Earning is making." George gives another example: If I take a piece of leather and work it up into a pair of shoes, the shoes are my wages – the reward of my exertion. Surely they are not drawn from capital – either my capital or any one else's capital – but are brought into existence by the labor of which they become the wages; and in obtaining this pair of shoes as the wages of my labor, capital is not even momentarily lessened one iota... As my labor goes on, value is steadily added, until, when my labor results in the finished shoes, I have my capital plus the difference in value between the material and the shoes. And another: If I hire a man to gather eggs, to pick berries, or to make shoes, paying him from the eggs, the berries, or the shoes that his labor secures, there can be no question that the source of the wages is the labor for which they are paid. George goes on to say it doesn't matter if you're paid in money or directly in wealth, because the money is a direct claim on the underlying wealth. It also doesn't matter if you get paid on commission. Imagine a whaling ship where each crewman gets paid a share out of whatever the ship catches. When the ship sails back into port with a hold full of whale oil and bone, the crew gets paid in money, the owner simultaneously adds to his capital oil and bone. The crew's money directly represents their share of the concrete wealth that is the oil and bone. The owner's capital hasn't decreased, and the workers drew their wages directly from the production. So let's get to the point, Mr. George – wages aren't drawn from capital but instead from production. Great, let's grant that – so what? George hammers away at this because thinking wages are drawn from capital leads to a false conclusion, namely that "labor cannot exert its productive power unless supplied by capital with maintenance." "Maintenance?" Well, workers need food and clothing and they get paid by their employers, so you could imagine capital as a limiting factor on labor. But by George, food and clothing isn't capital, it's just wealth, as we said before. And with regard to wages, the point is that the employer always gets "paid" first, because the second the laborer produces value, the employer's capital increases: As in the exchange of labor for wages the employer always gets the capital created by the labor before he pays out capital in the wages, at what point is his capital lessened even temporarily? Okay, but what if I'm just a terrible businessman and I pay somebody $500 an hour to smash Ming vases, then sell the fragments as aggregate to a construction crew for a few pennies a pound, all at a tremendous loss? Surely then the laborer's wages must be drawn from my capital, because there's not enough productive value generated by the labor to draw them from! George says okay, sure, but only because I'm an idiot and will soon be out of business: Yet, unless the new value created by the labor is less than the wages paid, which can be only an exceptional case, the capital which he had before in money he now has in goods – it has been changed in form, but not lessened. Fair enough, Mr. George, but what if I'm building some enormously expensive multi-decade project, like a dam or a nuclear power plant or a cathedral? The kind of thing we call a "capital-intensive" project? What do you have to say to that? George points out that as laborers labor, they progressively add value to whatever they're producing. Take the case of a shipwright building ships for an employer – even if the boss can't sell a half-finished ship, it still holds value (for one, it costs less to finish a half-finished ship then no ship at all). And with every stroke of the laborer's work, the employer who owns the shipyard gets an incremental increase in his stock of capital. It is not the last blow, any more than the first blow, that creates the value of the finished product – the creation of value is continuous, it immediately results from the exertion of labor. A pedant would point out that the "last hit" that finishes the product which makes it ready for market adds disproportionate value, but George's point is just to establish that value is continuously created, and doesn't magically come into being allat once right at the end. George further points out that if you look at things like agriculture you'll see the market directly acknowledging his theory: As a plowed field will bring more than an unplowed field, or a field that has been sown more than one merely plowed... It is tangible in the case of orchards and vineyards which, though not yet in bearing, bring prices proportionate to their age. George freely admits that capital can be required for certain kinds of work, but he disagrees with what its purpose is. It's not a pool that wages get paid out of. He goes on for another chapter on "The Maintenance of Laborers Not Drawn From Capital" but I think we can safely skip it and move on. TL:DR – George hammers to absolute death the idea that Laborers derive their own maintenance (food/shelter/clothing/etc) from their wages, with George insisting it is drawn from production and... you guessed it, not from capital. At least some of George's ideas will not seem so radical to modern readers (especially those already critical of capitalism or neoclassical economics), but it's important to understand that at the time almost everything he was saying was considered deeply radical and shocking. Capital was the fundamental driving force of the economy and labor was utterly dependent on it, and the Malthusian theory of overpopulation was the accepted explanation for why wages were low and workers were starving. Political Cartoon literally demonizing Henry George – Puck magazine Oct. 20, 1886 The Real Functions of Capital Okay, Mr. George. You've spent three whole chapters beating me over the head with what the functions of capital aren't. So what are the functions of capital? Capital "increases the power of labor to produce wealth." How? By enabling labor to apply itself more effectively (power tools go brrrr)
The only way wages (the return to labor) and interest (the return to capital) can go up as productivity increases, is if land values fail to rise at the same rate. The Law of Interest George wants to find the fundamental reason capital is able to produce wealth and justly claim a fair share of production. Remember that capital is wealth devoted to getting more wealth. So if capital is wealth that begets wealth, it makes sense that if I lend it out to you, I miss out on the potential for it to grow while it's out of my hands. George says I am justly entitled to ask for more back than I originally gave you. Let's say I loan you some corn seeds for a season. Had I not leant them to you, in a season's time I could have grown my own crop of corn and been left with more seed than I started with. So in a perfectly square deal, you need to give me back what I started with and what I could have expected to gain from natural increase (less the value of the labor required to get things started). Likewise with any other article of capital – say bricks or lumber. In the time I've spent without it while it was in your possession, I could have found someone else who had a better use for it than I did and exchanged it for something of theirs that I had a better use for, leaving me with capital of greater value. George says the act of progressively exchanging things in a way that increases subjective value for all involved is analogous to the natural forces of nature that make living capital (like corn and cows) grow over time. Remember, "subjective value" is real value. In a game of Settlers of Catan, if I have two bricks and you have two lumber, neither of us can build anything. The simple act of trading one brick for one lumber means both of us are better off because each of us can now build a road. The amount of bricks and lumber in the world didn't increase, but the amount of roads (or potential roads) did, and that represents a real increase in wealth. Interest thus springs from the "reproductive" powers of capital, whether that's biological reproduction, or the more abstract reproductive force of exchanging things so that you have a more valuable distribution of capital than you started with. As for how it relates to the other two returns to production – the more powerful the "power of increase" the capital has, the greater return interest can claim compared to wages. If you're ploughing a field and I lend you a tractor which makes you ten times as productive, I can justly claim more compensation for that than if I lend you a mule that only makes you twice as productive. However, rent still holds the whip hand, so the margin of cultivation determines how much return is left over to divvy up between interest and wages. This is because the net "reproductive" value of capital goes down given rent is a general tax on overall productivity. The amount I would have gained by using the thing productively over the period of time it was out on loan (the amount I can justly charge in interest) is reduced by how much I have to pay in rent. The Law of Wages Wages, like interest, are limited by the margin of production. Within that limit there's not much to understand about how wages work except that people seek to satisfy their desires "with the least exertion," which is a fancy way of saying people don't like to get ripped off. If two bosses offer the same exact job, but one offers higher pay, I'm taking that gig. If two bosses pay the same, but one is asking for twice as much work, I'll tell that boss where he can stick it. Wages depend upon the margin of production, or upon the produce which labor can obtain at the highest point of natural productiveness open to it without the payment of rent. So with all three laws established George sums it up like so: Where land is free and labor is unassisted by capital, the whole produce will go to labor as wages. Where land is free and labor is assisted by capital, wages will consist of the whole produce, less that part necessary to induce the storing up of labor as capital. Where land is subject to ownership and rent arises, wages will be fixed by what labor could secure from the highest natural opportunities open to it without the payment of rent. Where natural opportunities are all monopolized, wages may be forced by the competition among laborers to the minimum at which laborers will consent to reproduce. This is the reason George says that wages are so high in "new countries" where there's more land available than in countries where it's been locked up for centuries. Here's how it all fits together: Though neither wages nor interest anywhere increase as material progress goes on, yet the invariable accompaniment and mark of material progress is the increase of rent – the rise of land values. And: where the value of land is highest, civilization exhibits the greatest luxury side by side with the most piteous destitution IV. Effect of Material Progress upon the Distribution of Wealth As a society undergoes material progress, the rent goes up. Why? Let's break it down. Three things contribute to material progress: Increasing population
SimCity

SimCity is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between July 17, 2023 and July 17, 2023. The archive places it in contexts such as "the same way SimCity has simulated citizens". It most often appears alongside ChatGPT, Elon, Elon Musk.

Reference entry
SimCity
Mention count
1
Issue count
1
First seen
July 17, 2023
Last seen
July 17, 2023
July 17, 2023 · Original source
If it did work, it would be bad. I want to start by discussing the second objection, then loop back to explain what I mean about the first. A Maximally Curious AI Would Not Be Safe For Humanity The one sentence version: many scientists are curious about fruit flies, but this rarely ends well for the fruit flies. The longer, less flippant version: Even if an AI decides humans are interesting, this doesn’t mean the AI will promote human flourishing forever. Elon says his goal is “an age of plenty where there is no shortage of goods and services”, but why would a maximally-curious AI provide this? It might decide that humans suffering is more interesting than humans flourishing. Or that both are interesting, and it will have half the humans in the world flourish, and the other half suffer as a control group. Or that neither are the most interesting thing, and it would rather keep humans in tanks and poke at them in various ways to see what happens. Even if an AI decides human flourishing is briefly interesting, after a while it will already know lots of things about human flourishing and want to learn something else instead. Scientists have occasionally made colonies of extremely happy well-adjusted rats to see what would happen. But then they learned what happened, and switched back to things like testing how long rats would struggle against their inevitable deaths if you left them to drown in locked containers. Is leaving human society intact really an efficient way to study humans? Maybe it would be better to dissect a few thousand humans, learn the basic principles, then run a lot of simulations of humans in various contrived situations. Would the humans in the simulation be conscious? I don’t know and the AI wouldn’t care. If it was cheaper to simulate abstracted humans in low-fidelity, the same way SimCity has simulated citizens who are just a bundle of traffic-related preferences, wouldn’t the AI do that instead? Are humans more interesting than sentient lizard-people? I don’t know. If the answer is no, will the AI kill all humans and replace them with lizard-people? Surely after a thousand years of studying human flourishing ad nauseum, the lizard-people start sounding more interesting. Would a maximally curious AI be curious about the same things as us? I would like to think that humans are “objectively” more interesting than moon rocks in some sense - harder to predict, capable of more complex behavior. But if it turns out that the most complex and unpredictable part of us is how our fingerprints form, and that (eg) our food culture is an incredibly boring function of a few gustatory receptors, will the AI grow a trillion human fingers in weird vats, but also remove our ability to eat anything other than nutrient sludge? I predict that if we ever got a maximally curious superintelligence, it would scan all humans, vaporize existing physical-world humans as unnecessary and inconvenient, use the scans to run many low-fidelity simulations to help it learn the general principles of intelligent life (plus maybe a few higher-fidelity simulations, like the one you’re in now), then simulate a trillion intelligent-life-like entities to see if (eg) their neural networks reached some interesting meta-stable positions. Then it would move beyond being interested in any of that, and disassemble the Earth to use its atoms to make a really big particle accelerator (which would be cancelled halfway through by Superintelligent AI Congress). This doesn’t mean AI can’t have a goal of understanding the universe. I think this would be a very admirable goal! It just can’t be the whole alignment strategy. But Also, We Couldn’t Make A Maximally Curious AI Even If We Wanted To The problem with AI alignment isn’t really that we don’t have a good long-term goal to align the AI to. Back in 2010 we debated things like long-term goals, hoping that whoever programmed the AI could just write a long_term_goal.txt file and then some functions pointing there. But now in the 2020s the discussion has moved forward to “how do we make the AI do anything at all?” Now we direct AIs through reinforcement learning - telling them to do certain things and avoid certain other things. But this is a blunt instrument. Reinforcement learning directs the AI towards a certain cluster of correlated high-dimensional concepts that have the same lower-dimensional shadow of rewarded and punished behaviors. But we can’t be sure which concept it’s chosen or whether it’s the one we think. For example, there are many different ways of fleshing out “curiosity”. Suppose that Elon rewards an AI whenever it takes any curious-seeming action, and punishes it whenever it takes any incurious-seeming action. After many training rounds, it seems very curious. It goes off to the jungles of Guatemala and uncovers hidden Mayan cities. It sends probes to icy moons of Neptune to assess their composition. Overall it aces every curiosity test we give it with flying colors. But what’s its definition of curiosity? Perhaps it’s something like “maximize your knowledge of the nature and position of every atom in the solar system, weighted for interestingness-to-humans”. This would produce the observed behavior of exploring Guatemala and Neptune. But once it’s powerful enough, it might want to destroy the solar system - if it’s completely empty, it can be completely confident that it knows every single fact about it. Or what if it’s curious about existing objects, but not about nonexistent objects? This would produce good behavior during training, and makes a decent amount of sense. But it might mean the AI would ban humans from ever having children, since it’s not at all curious about what those (currently nonexistent) children would do, and they’re just making things more complicated. Or what if its curiosity depends on information-theoretic definitions of complexity? It might be that humans are more complex than moon rocks, but random noise is more complex than humans. It might behave well during training, but eventually want to replace humans with random noise. This is a kind of exaggerated scenario, but it wouldn’t surprise me if, for most formal definitions of curiosity, there’s something that we would find very boring which acts as a sort of curiosity-superstimulus by the standards of the formal definition. The existing field of AI alignment tries to figure out how to install any goal at all into an AI with reasonable levels of certainty that it in fact has that goal and not something closely correlated with a similar reinforcement-learning shadow. It’s not currently succeeding. This isn’t a worse problem for Musk and xAI than for anyone else, but there are a few aspects of their strategy that I think will make it harder for them to solve in practice: One good thing about order-following AI is that it’s useful now, when AIs aren’t agentic enough to have real goals and we just want to use them as tools in commercial applications. The hope is that we do this a bunch with GPT-4, then a bunch with GPT-5, and so on, and by the time we have a real superintelligence, we’ve worked out some of the kinks. I’m not sure how Musk’s maximally-curious AI helps do office work, which means there’s going to be more of a disconnect between current easily-tested applications and the eventual superintelligence that we need to get right.
StarCraft

StarCraft is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between February 23, 2022 and February 23, 2022. The archive places it in contexts such as "train an AI to play StarCraft". It most often appears alongside AGI, AI Impacts, AIXI.

Reference entry
StarCraft
Mention count
1
Issue count
1
First seen
February 23, 2022
Last seen
February 23, 2022
February 23, 2022 · Original source
Source: This document by Paul Christiano. Ajeya combines this with another metric where they see how existing AI compares to animals with apparently similar computational capacity; for example, she says that DeepMind’s Starcraft engine has about as much inferential compute as a honeybee and seems about equally subjectively impressive. I have no idea what this means. Impressive at what? Winning multiplayer online games? Stinging people? In any case, they decide to penalize AI by one order of magnitude compared to Nature, so a human-level AI would need to do 10^16 floating point operations per second. How Much Compute Would It Take To Train A Model That Does 10^16 Floating Point Operations Per Second? So an AI could potentially equal the human brain with 10^16 FLOP/S. Good news! There’s a supercomputer in Japan that can do 10^17 FLOP/S! It looks like this (source) So why don’t we have AI yet? Why don’t we have ten AIs? In the modern paradigm of machine learning, it takes very big computers to train relatively small end-product AIs. If you tried to train GPT-3 on the same kind of medium-sized computers you run it on, it would take between tens and hundreds of years. Instead, you train GPT-3 on giant supercomputers like the ones above, get results in a few months, then run it on medium-sized computers, maybe ~10x better than the average desktop. But our hypothetical future human-level AI is 10^16 FLOP/S in inference mode. It needs to run on a giant supercomputer like the one in the picture. Nothing we have now could even begin to train it. There’s no direct and obvious way to convert inference requirements to training requirements. Ajeya tries assuming that each parameter will contribute about 10 FLOPs, which would mean the model would have about 10^15 parameters (GPT-3 has about 10^11 parameters). Finally, she uses some empirical scaling laws derived from looking at past machine learning projects to estimate that training 10^15 parameters would require H*10^30 FLOPs, where H represents the model’s “horizon”. If I understand this correctly, “horizon” is a reinforcement learning concept: how long does it take to learn how much reward you got for something? If you’re playing a slot machine, the answer is one second. If you’re starting a company, the answer might be ten years. So what horizon do you need for human level AI? Who knows? It probably depends on what human-level task you want the AI to do, plus how well an AI can learn to do that task from things less complex than the entire task. If writing a good book is mostly about learning to write good sentence and then stringing them together, a book-writing AI can get away with a short horizon. If nothing short of writing an entire book and then evaluating it to see whether it is good or bad can possibly teach you book-writing, the AI will need a long time horizon. Ajeya doesn’t claim to have a great answer for this, and considers three models: horizons of a few minutes, a few hours, and a few years. Each step up adds another three orders of magnitude, so she ends up with three estimates of 10^30, 10^33, and 10^36 FLOPs. (for reference, the lowest training estimate - 10^30 - would take the supercomputer pictured above 300,000 years to complete; the highest, 300 billion.) Or What If We Ignore All Of That And Do Something Else? This is piling a lot of assumptions atop each other, so Ajeya tries three other methods of figuring out how hard this training task is. Humans seem to be human-level AIs. How much training do we need? You can analogize our childhood to an AI’s training period. We receive a stream of sense-data. We start out flailing kind of randomly. Some of what we do gets rewarded. Some of what we do gets punished. Eventually our behavior becomes more sophisticated. We subject our new behavior to reward or punishment, fine-tune it further. Rent asks us: how do you measure the life of a woman or man? It answers: “in daylights, in sunsets, in midnights, in cups of coffee; in inches, in miles, in laughter, in strife.” But you can also measure in floating point operations, in which case the answer is about 10^24. This is actually trivial: multiply the 10^15 FLOP/S of the human brain by the ~10^9 seconds of childhood and adolescence. This new estimate of 10^24 is much lower than our neural net estimate of 10^30 - 10^36 above. In fact, it’s only a hair above the amount it took to train GPT-3! If human-level AI was this easy, we should have hit it by accident sometime in the process of making a GPT-4 prototype. Since OpenAI hasn’t mentioned this, probably it’s harder than this and we’re missing something. Probably we’re missing that humans aren’t blank slates. We don’t start at zero and then only use our childhood to train us further. The very structure of our brain encodes certain assumptions about what kinds of data we should be looking out for and how we should use it. Our training data isn’t just what we observed during childhood, it’s everything that any of our ancestors observed during evolution. How many floating-point operations is the evolutionary process? Ajeya estimates 10^41. I can’t believe I’m writing this. I can’t believe someone actually estimated the number of floating point operations involved in jellyfish rising out of the primordial ooze and eventually becoming fish and lizards and mammals and so on all the way to the Ascent of Man. Still, the idea is simple. You estimate how long animals with neurons have been around for (10^16 seconds), total number of animals at any given second (10^20) times average number of FLOPS per animal (10^5) and you can read more here but it comes out to 10^41 FLOs. I would not call this an exact estimate - for one thing, it assumes that all animals are nematodes, on the grounds that non-nematode animals are basically a rounding error in the grand scheme of things. But it does justify this bizarre assumption, and I don’t feel inclined to split hairs here - surely the total amount of computation performed by evolution is irrelevant except as an extreme upper bound? Surely the part where Australia got all those weird marsupials wasn’t strictly necessary for the human brain to have human-level intelligence? One more weird human training data estimate attempt: what about the genome? If in some sense a bit of information in the genome is a “parameter”, how many parameters does that suggest humans have, and how does it affect training time? Ajeya calculates that the genome has about 7.5x10^8 parameters (compared to 10^15 parameters in our neural net calculation, and 10^11 for GPT-3). So we can… Okay, I’ve got to admit, this doesn’t have quite the same “huh?!” factor as trying to calculate the number of FLOs in evolution, but it is in a lot of ways even crazier. The Japanese canopy plant has a genome fifty times larger than ours, which suggests that genome size doesn’t correspond very well to organism awesomeness. Also, most of the genome is coding for weird proteins that stabilize the shape of your kidney tubule or something, why should this matter for intelligence? The Japanese canopy plant. I think it is very pretty, but probably low prettiness per megabyte of DNA. I think Ajeya would answer that she’s debating orders of magnitude here, and each of these weird things costs only a few OOMs and probably they all even out. That still leaves the question of why she thinks this approach is interesting at all, to which she answers that: The motivating intuition is that evolution performed a search over a space of small, compact genomes which coded for large brains rather than directly searching over the much larger space of all possible large brains, and human researchers may be able to compete with evolution on this axis. So maybe instead of having to figure out how to generate a brain per se, you figure out how to generate some short(er) program that can output a brain? But this would be very different from how ML works now. Also, you need to give each short program the chance to unfold into a brain before you can evaluate it, which evolution has time for but we probably don’t. Ajeya sort of mentions these problems and counters with an argument that maybe you could think of the genome as a reinforcement learner with a long horizon. I don’t quite follow this but it sounds like the sort of thing that almost might make sense. Anyway, when you apply the scaling laws to a 7.5*10^8 parameter genome and penalize it for a long horizon, you get about 10^33 FLOPs, which is weirdly similar to some of the other estimates. So now we have six different training cost estimates. First, neural nets with short, medium, and long horizons, which are 10^30, 10^33, and 10^36 FLOPs, respectively. Next, the amount of training data in a human lifetime - 10^24 FLOs - and in all of evolutionary history - 10^41 FLOPs. And finally, this weird genome thing, which is 10^33 FLOPs. An optimist might say “Well, our lowest estimate is 10^24 FLOPs, our highest is 10^41 FLOPs, those sound like kind of similar numbers, at least there’s no “5 FLOPs” or “10^9999 FLOPs” in there. A pessimist might say “The difference between 10^24 and 10^41 is seventeen orders of magnitude, ie a factor of 100,000,000,000,000,000 times. This barely constrains our expectations at all!” Before we decide who to trust, let’s remember that we’re still only at Step 2 of our eight step Methodology, and continue. How Do We Adjust For Algorithmic Progress? So today, in 2022 (or in 2020 when this was written, or whenever), assume it would take about 10^33 FLOs to train a human-level AI. But technology constantly advances. Maybe we’ll discover ways to train AIs faster, or run AIs more efficiently, or something like that. How does that factor into our estimate? Ajeya draws on Hernandez & Brown’s Measuring The Algorithmic Efficiency Of Neural Networks. They look at how many FLOPs it took to train various image recognition AIs to an equivalent level of performance between 2012 and 2019, and find that over those seven years it decreased by a factor of 44x, ie training efficiency doubles every sixteen months! Ajeya assumes a doubling time slightly longer than that, because it’s easier to make progress in simple well-understood fields like image recognition than in the novel task of human-level AI. She chooses a doubling time of “merely” 2 - 3 years. If training efficiency doubles every 2-3 years, it would dectuple in about 10 years. So although it might take 10^33 FLOPs to train a human level AI today, in ten years or so it may take only 10^32, in twenty years 10^31, and so on. When Will Anyone Have Enough Computational Resources To Train A Human-Level AI? In 2020, AI researchers could buy computational resources at about $1 for 10^17 FLOPs. That means the 10^33 FLOPs you’d need to train a human-level AI would cost $10^16, ie ten quadrillion dollars. This is about twenty times more money than exists in the entire world. But compute costs fall quickly. Some formulations of Moore’s Law suggest it halves every eighteen months. These no longer seem to hold exactly, but it does seem to be halving maybe once every 2.5 years. The exact number is kind of controversial: Ajeya admits it’s been more like once every 3-4 years lately, but she heard good things about some upcoming chips and predicted it might revert back to the longer-term faster trend (it’s been two years now, some new chips have come out, and this prediction is looking pretty good). So as time goes on, algorithmic progress will cut the cost of training (in FLOPs), and hardware progress will also cut the cost of FLOPs (in dollars). So training will become gradually more affordable as time goes on. Once it reaches a cost somebody is willing to pay, they’ll buy human-level AI, and then that will be the year human-level AI happens. What is the cost that somebody (company? government? billionaire?) is willing to pay for human-level AI? The most expensive AI training in history was AlphaStar, a DeepMind project that spent over $1 million to train an AI to play StarCraft (in their defense, it won). But people have been pouring more and more money into AI lately: Source here. This is about compute rather than cost, but most of the increase seen here has been companies willing to pay for more compute over time, rather than algorithmic or hardware progress. The StarCraft AI was kind of a vanity project, or science for science’s sake, or whatever you want to call it. But AI is starting to become profitable, and human-level AI would be very profitable. Who knows how much companies will be willing to pay in the future? Ajeya extrapolates the line on the graph forward to 2025 and gets $1 billion. This is starting to sound kind of absurd - the entire company OpenAI was founded with $1 billion in venture capital, it seems like a lot to expect them to spend more than $1 billion on a single training run. So Ajeya backs off from this after 2025 and predicts a “two year doubling time”. This is not much of a concession. It still means that in 2040 someone might be spending $100 billion to train one AI. Is this at all plausible? At the height of the Manhattan Project, the US was investing about 0.5% of its GDP into the effort; a similar investment today would be worth $100 billion. And we’re about twice as rich as 2000, so 2040 might be twice as rich as we are. At that point, $100 billion for training an AI is within reach of Google and maybe a few individual billionaires (though it would still require most or all of their fortune). Ajeya creates a complicated function to assess how much money people will be willing to pay on giant AI projects per year. This looks like an upward-sloping curve. The line representing the likely cost of training a human-level AI looks like a downward sloping curve. At some point, those two curves meet, representing when human-level AI will first be trained. So When Will We Get Human-Level AI? The report gives a long distribution of dates based on weights assigned to the six different models, each of which has really wide confidence intervals and options for adjusting the mean and variance based on your assumptions. But the median of all of that is 10% chance by 2031, 50% chance by 2052, and almost 80% chance by 2100. Ajeya takes her six models and decides to weigh them like so, based on how plausible she thinks each one is: 20% neural net, short horizon 30% neural net, medium horizon 15% neural net, long horizon 5% human lifetime as training data 10% evolutionary history as training data 10% genome as parameter number She ends up with this: How Sensitive Is This To Changes In Assumptions? She very helpfully gives us a Colab notebook and Google spreadsheet to play around with. The notebook lets you change some of the more detailed parameters of the individual models, and the spreadsheet lets you change the big picture. I leave the notebook to people more dedicated to forecasting than I am, and will talk about the spreadsheet here. If you’re following along at home, the default spreadsheet won’t reflect Ajeya’s findings until you fill in the table in the bottom left like so: Great. Now that we’ve got that, let’s try changing some stuff. I like the human childhood training data argument (Lifetime Anchor) more than Ajeya does, and I like the size-of-the-genome argument less. I’m going to change the weights to 20-20-0-20-20-20. Also, Ajeya thinks that someone might be willing to spend 1% of national GDP on training AIs, but that sounds really high to me, so I’m going to down to 0.1%. Also, Ajeya’s estimate of 3% GDP growth sounds high for the sort of industrialized nations who might do AI research, I’m going to lower it to 2%. Since I’m feeling mistrustful today, let’s use the Hernandez&Brown estimate for compute halving (1.5 years) in place of Ajeya’s ad hoc adjustments. And let’s use the current compute halving time (3.5 years) instead of Ajeya’s overly rosy version (2.5 years). All these changes… …don’t really do much. The median goes from 2052 to about 2065. Four of the models give results between 2030 and 2070. The last two, Neural Net With Long Horizon and Evolution, suggest probably no AI this century (although Neural Net With Long Horizon does think there’s a 40% chance by 2100). Ajeya doesn’t really like either of these models and they’re not heavily weighted in her main result. Does The Truth Point To Itself? Back up a second. Here’s something that makes me kind of nervous. Most of Ajeya’s numbers are kind of made up, with several order-of-magnitude error bars and simplifying assumptions like “all animals are nematodes”. For a single parameter, we get estimates spanning seventeen different orders of magnitude: the upper bound is one hundred quadrillion times the lower bound. And yet four of the six models, including two genuinely exotic ones, manage to get dates within twenty years of 2050. And 2050 is also the date everyone else focuses on. Here’s the prediction-market-like site Metaculus: Their distribution looks a lot like Ajeya’s, and even has the same median, 2052 (though forecasters could have read Ajeya’s report). Katja Grace et al surveyed 352 AI experts, and they gave a median estimate of 2062 for an AI that could “outperform humans at all tasks” (though with many caveats and high sensitivity to question framing). This was before Ajeya’s report, so they definitely didn’t read it. So lots of Ajeya’s different methods and lots of other people presumably using different methodologies or no methodology at all, all converge on this same idea of 2050 give or take a decade or two. An optimist might say “The truth points to itself! There are 371 known proofs of the Pythagorean Theorem, and they all end up in the same place. That’s because no matter what methodology you use, if you use it well enough you get to the correct answer.” A pessimist might be more suspicious; we’ll return to this part later. FLOPS Alone Turn The Wheel Of History One more question: what if this is all bullshit? What if it’s an utterly useless total garbage steaming pile of grade A crap? Imagine a scientist in Victorian Britain, speculating on when humankind might invent ships that travel through space. He finds a natural anchor: the moon travels through space! He can observe things about the moon: for example, it is 220 miles in diameter (give or take an order of magnitude). So when humankind invents ships that are 220 miles in diameter, they can travel through space! Ships have certainly grown in size tremendously, from primitive kayaks to Roman triremes to Spanish galleons to the great ocean liners of the (Victorian) present. The AI forecasting organization AI Impacts actually has a whole report on historical ship size trends to prove an unrelated point about technological progress, so I didn’t even have to make this graph up. Suppose our Victorian scientist lived in 1858, right when the Great Eastern was launched. The trend line for ship size crossed 100m around 1843, and 200m in 1858, so doubling time is 15 years - but perhaps they notice this is going to be an outlier, so let’s round up a bit and say 18 years. The (one order of magnitude off estimate for the size of the) Moon is 350,000m, so you’d need ships to scale up by 350,000/200 = 1,750x before they’re as big as the Moon. That’s about 10.8 doublings, and a doubling time is 18 years, so we’ll get spaceships in . . . 2052 exactly. (fudging numbers to land where you want is actually fun and easy) SS Great Eastern, the extreme outlier large steamship from 1858. This has become sort of a mascot for quantitative technological progress forecasters. What is this scientist’s error? The big one is thinking that spaceship progress depends on some easily-measured quantity (size) instead of on fundamental advances (eg figuring out how rockets work). You can make the same accusation against Ajeya et al: you can have all the FLOPs in the world, but if you don’t understand how to make a machine think, your AI will be, well, a flop. Ajeya discusses this a bit on page 143 of her report. There is some sense in which FLOPs and knowing-what-you’re-doing trade of against each other. If you have literally no idea what you’re doing, you can sort of kind of re-run evolution until it comes up with something that looks good. If things are somehow even worse than that, you could always run AIXI, a hypothetical AI design guaranteed to get excellent results as long as you have infinite computation. You could run a Go engine by searching the entire branching tree structure of Go - you shouldn’t, and it would take a zillion times more compute than exists in the entire world, but you could. So in some sense what you’re doing, when you’re figuring out what you’re doing, is coming up with ways to do already-possible things more efficiently. But that’s just algorithmic progress, which Ajeya has already baked into her model. (our Victorian scientist: “As a reductio ad absurdum, you could always stand the ship on its end, and then climb up it to reach space. We’re just trying to make ships that are more efficient than that.”) Part II: Biology-Inspired AI Timelines: The Trick That Never Works Eliezer Yudkowsky presents a more subtle version of these kinds of objection in an essay called Biology-Inspired AI Timelines: The Trick That Never Works, published December 2021. Ajeya’s report is a 169-page collection of equations, graphs, and modeling assumptions. Yudkowsky’s rebuttal is a fictional dialogue between himself, younger versions of himself, famous AI scientists, and other bit players. At one point, a character called “Humbali” shows up begging Yudkowsky to be more humble, and Yudkowsky defeats him with devastating counterarguments. Still, he did found the field, so I guess everyone has to listen to him. He starts: in 1988, famous AI scientist Hans Moravec predicted human-level AI by 2010. He was using the same methodology as Ajeya: extrapolate how quickly processing power would grow (in FLOP/S), and see when it would match some estimate of the human brain. Moravec got the processing power almost exactly right (it hit his 2010 projection in 2008) and his human brain estimate pretty close (he says 10^13 FLOP/S, Ajeya says 10^15, this 2 OOM difference only delays things a few years), yet there was not human-level AI in 2010. What happened? Ajeya's answer could be: Moravec didn't realize that, in the modern ML paradigm, any given size of program requires a much bigger program to train. Ajeya, who has a 35-year advantage on Moravec, estimates approximately the same power for the finished program (10^16 vs. 10^13 FLOP/S) but says that training the 10^16 FLOP/S program will require 10^33ish FLOPs. Eliezer agrees as far as it goes, but says this points to a much deeper failure mode, which was that Moravec had no idea what he was doing. He was assuming processing power of human brain = processing power of computer necessary for AGI. Why? The human brain consumes around 20 watts of power. Can we thereby conclude that an AGI should consume around 20 watts of power, and that, when technology advances to the point of being able to supply around 20 watts of power to computers, we'll get AGI? […] You say that AIs consume energy in a very different way from brains? Well, they'll also consume computations in a very different way from brains! The only difference between these two cases is that you know something about how humans eat food and break it down in their stomachs and convert it into ATP that gets consumed by neurons to pump ions back out of dendrites and axons, while computer chips consume electricity whose flow gets interrupted by transistors to transmit information. Since you know anything whatsoever about how AGIs and humans consume energy, you can see that the consumption is so vastly different as to obviate all comparisons entirely. You are ignorant of how the brain consumes computation, you are ignorant of how the first AGIs built would consume computation, but "an unknown key does not open an unknown lock" and these two ignorant distributions should not assert much internal correlation between them. Cars don’t move by contracting their leg muscles and planes don’t fly by flapping their wings like birds. Telescopes do form images the same way as the lenses in our eyes, but differ by so many orders of magnitude in every important way that they defy comparison. Why should AI be different? You have to use some specific algorithm when you’re creating AI; why should we expect it to be anywhere near the same efficiency as the ones Nature uses in our brains? The same is true for arguments from evolution, eg Ajeya’s Evolutionary Anchor, ie “it took evolution 10^43 FLOPs of computation to evolve the human brain so maybe that will be the training cost”. AI scientists sitting in labs trying to figure things out, and nematodes getting eaten by other nematodes, are such different methods for designing things that it’s crazy to use one as an estimate for the other. Algorithmic Progress vs. Algorithmic Paradigm Shifts This post is a dialogue, so (Eliezer’s hypothetical model of) OpenPhil gets a chance to respond. They object: this is why we put a term for algorithmic progress in our model. The model isn’t very sensitive to changes in that term. If you want you can set it to some kind of crazy high value and see what happens, but you can’t say we didn’t consider it. OpenPhil: We did already consider that and try to take it into account: our model already includes a parameter for how algorithmic progress reduces hardware requirements. It's not easy to graph as exactly as Moore's Law, as you say, but our best-guess estimate is that compute costs halve every 2-3 years […] Eliezer: The makers of AGI aren't going to be doing 10,000,000,000,000 rounds of gradient descent, on entire brain-sized 300,000,000,000,000-parameter models, algorithmically faster than today. They're going to get to AGI via some route that you don't know how to take, at least if it happens in 2040. If it happens in 2025, it may be via a route that some modern researchers do know how to take, but in this case, of course, your model was also wrong. They're not going to be taking your default-imagined approach algorithmically faster, they're going to be taking an algorithmically different approach that eats computing power in a different way than you imagine it being consumed. OpenPhil: Shouldn't that just be folded into our estimate of how the computation required to accomplish a fixed task decreases by half every 2-3 years due to better algorithms? Eliezer: Backtesting this viewpoint on the previous history of computer science, it seems to me to assert that it should be possible to: Train a pre-Transformer RNN/CNN-based model, not using any other techniques invented after 2017, to GPT-2 levels of performance, using only around 2x as much compute as GPT-2;
The Japanese canopy plant. I think it is very pretty, but probably low prettiness per megabyte of DNA. I think Ajeya would answer that she’s debating orders of magnitude here, and each of these weird things costs only a few OOMs and probably they all even out. That still leaves the question of why she thinks this approach is interesting at all, to which she answers that: The motivating intuition is that evolution performed a search over a space of small, compact genomes which coded for large brains rather than directly searching over the much larger space of all possible large brains, and human researchers may be able to compete with evolution on this axis. So maybe instead of having to figure out how to generate a brain per se, you figure out how to generate some short(er) program that can output a brain? But this would be very different from how ML works now. Also, you need to give each short program the chance to unfold into a brain before you can evaluate it, which evolution has time for but we probably don’t. Ajeya sort of mentions these problems and counters with an argument that maybe you could think of the genome as a reinforcement learner with a long horizon. I don’t quite follow this but it sounds like the sort of thing that almost might make sense. Anyway, when you apply the scaling laws to a 7.5*10^8 parameter genome and penalize it for a long horizon, you get about 10^33 FLOPs, which is weirdly similar to some of the other estimates. So now we have six different training cost estimates. First, neural nets with short, medium, and long horizons, which are 10^30, 10^33, and 10^36 FLOPs, respectively. Next, the amount of training data in a human lifetime - 10^24 FLOs - and in all of evolutionary history - 10^41 FLOPs. And finally, this weird genome thing, which is 10^33 FLOPs. An optimist might say “Well, our lowest estimate is 10^24 FLOPs, our highest is 10^41 FLOPs, those sound like kind of similar numbers, at least there’s no “5 FLOPs” or “10^9999 FLOPs” in there. A pessimist might say “The difference between 10^24 and 10^41 is seventeen orders of magnitude, ie a factor of 100,000,000,000,000,000 times. This barely constrains our expectations at all!” Before we decide who to trust, let’s remember that we’re still only at Step 2 of our eight step Methodology, and continue. How Do We Adjust For Algorithmic Progress? So today, in 2022 (or in 2020 when this was written, or whenever), assume it would take about 10^33 FLOs to train a human-level AI. But technology constantly advances. Maybe we’ll discover ways to train AIs faster, or run AIs more efficiently, or something like that. How does that factor into our estimate? Ajeya draws on Hernandez & Brown’s Measuring The Algorithmic Efficiency Of Neural Networks. They look at how many FLOPs it took to train various image recognition AIs to an equivalent level of performance between 2012 and 2019, and find that over those seven years it decreased by a factor of 44x, ie training efficiency doubles every sixteen months! Ajeya assumes a doubling time slightly longer than that, because it’s easier to make progress in simple well-understood fields like image recognition than in the novel task of human-level AI. She chooses a doubling time of “merely” 2 - 3 years. If training efficiency doubles every 2-3 years, it would dectuple in about 10 years. So although it might take 10^33 FLOPs to train a human level AI today, in ten years or so it may take only 10^32, in twenty years 10^31, and so on. When Will Anyone Have Enough Computational Resources To Train A Human-Level AI? In 2020, AI researchers could buy computational resources at about $1 for 10^17 FLOPs. That means the 10^33 FLOPs you’d need to train a human-level AI would cost $10^16, ie ten quadrillion dollars. This is about twenty times more money than exists in the entire world. But compute costs fall quickly. Some formulations of Moore’s Law suggest it halves every eighteen months. These no longer seem to hold exactly, but it does seem to be halving maybe once every 2.5 years. The exact number is kind of controversial: Ajeya admits it’s been more like once every 3-4 years lately, but she heard good things about some upcoming chips and predicted it might revert back to the longer-term faster trend (it’s been two years now, some new chips have come out, and this prediction is looking pretty good). So as time goes on, algorithmic progress will cut the cost of training (in FLOPs), and hardware progress will also cut the cost of FLOPs (in dollars). So training will become gradually more affordable as time goes on. Once it reaches a cost somebody is willing to pay, they’ll buy human-level AI, and then that will be the year human-level AI happens. What is the cost that somebody (company? government? billionaire?) is willing to pay for human-level AI? The most expensive AI training in history was AlphaStar, a DeepMind project that spent over $1 million to train an AI to play StarCraft (in their defense, it won). But people have been pouring more and more money into AI lately: Source here. This is about compute rather than cost, but most of the increase seen here has been companies willing to pay for more compute over time, rather than algorithmic or hardware progress. The StarCraft AI was kind of a vanity project, or science for science’s sake, or whatever you want to call it. But AI is starting to become profitable, and human-level AI would be very profitable. Who knows how much companies will be willing to pay in the future? Ajeya extrapolates the line on the graph forward to 2025 and gets $1 billion. This is starting to sound kind of absurd - the entire company OpenAI was founded with $1 billion in venture capital, it seems like a lot to expect them to spend more than $1 billion on a single training run. So Ajeya backs off from this after 2025 and predicts a “two year doubling time”. This is not much of a concession. It still means that in 2040 someone might be spending $100 billion to train one AI. Is this at all plausible? At the height of the Manhattan Project, the US was investing about 0.5% of its GDP into the effort; a similar investment today would be worth $100 billion. And we’re about twice as rich as 2000, so 2040 might be twice as rich as we are. At that point, $100 billion for training an AI is within reach of Google and maybe a few individual billionaires (though it would still require most or all of their fortune). Ajeya creates a complicated function to assess how much money people will be willing to pay on giant AI projects per year. This looks like an upward-sloping curve. The line representing the likely cost of training a human-level AI looks like a downward sloping curve. At some point, those two curves meet, representing when human-level AI will first be trained. So When Will We Get Human-Level AI? The report gives a long distribution of dates based on weights assigned to the six different models, each of which has really wide confidence intervals and options for adjusting the mean and variance based on your assumptions. But the median of all of that is 10% chance by 2031, 50% chance by 2052, and almost 80% chance by 2100. Ajeya takes her six models and decides to weigh them like so, based on how plausible she thinks each one is: 20% neural net, short horizon 30% neural net, medium horizon 15% neural net, long horizon 5% human lifetime as training data 10% evolutionary history as training data 10% genome as parameter number She ends up with this: How Sensitive Is This To Changes In Assumptions? She very helpfully gives us a Colab notebook and Google spreadsheet to play around with. The notebook lets you change some of the more detailed parameters of the individual models, and the spreadsheet lets you change the big picture. I leave the notebook to people more dedicated to forecasting than I am, and will talk about the spreadsheet here. If you’re following along at home, the default spreadsheet won’t reflect Ajeya’s findings until you fill in the table in the bottom left like so: Great. Now that we’ve got that, let’s try changing some stuff. I like the human childhood training data argument (Lifetime Anchor) more than Ajeya does, and I like the size-of-the-genome argument less. I’m going to change the weights to 20-20-0-20-20-20. Also, Ajeya thinks that someone might be willing to spend 1% of national GDP on training AIs, but that sounds really high to me, so I’m going to down to 0.1%. Also, Ajeya’s estimate of 3% GDP growth sounds high for the sort of industrialized nations who might do AI research, I’m going to lower it to 2%. Since I’m feeling mistrustful today, let’s use the Hernandez&Brown estimate for compute halving (1.5 years) in place of Ajeya’s ad hoc adjustments. And let’s use the current compute halving time (3.5 years) instead of Ajeya’s overly rosy version (2.5 years). All these changes… …don’t really do much. The median goes from 2052 to about 2065. Four of the models give results between 2030 and 2070. The last two, Neural Net With Long Horizon and Evolution, suggest probably no AI this century (although Neural Net With Long Horizon does think there’s a 40% chance by 2100). Ajeya doesn’t really like either of these models and they’re not heavily weighted in her main result. Does The Truth Point To Itself? Back up a second. Here’s something that makes me kind of nervous. Most of Ajeya’s numbers are kind of made up, with several order-of-magnitude error bars and simplifying assumptions like “all animals are nematodes”. For a single parameter, we get estimates spanning seventeen different orders of magnitude: the upper bound is one hundred quadrillion times the lower bound. And yet four of the six models, including two genuinely exotic ones, manage to get dates within twenty years of 2050. And 2050 is also the date everyone else focuses on. Here’s the prediction-market-like site Metaculus: Their distribution looks a lot like Ajeya’s, and even has the same median, 2052 (though forecasters could have read Ajeya’s report). Katja Grace et al surveyed 352 AI experts, and they gave a median estimate of 2062 for an AI that could “outperform humans at all tasks” (though with many caveats and high sensitivity to question framing). This was before Ajeya’s report, so they definitely didn’t read it. So lots of Ajeya’s different methods and lots of other people presumably using different methodologies or no methodology at all, all converge on this same idea of 2050 give or take a decade or two. An optimist might say “The truth points to itself! There are 371 known proofs of the Pythagorean Theorem, and they all end up in the same place. That’s because no matter what methodology you use, if you use it well enough you get to the correct answer.” A pessimist might be more suspicious; we’ll return to this part later. FLOPS Alone Turn The Wheel Of History One more question: what if this is all bullshit? What if it’s an utterly useless total garbage steaming pile of grade A crap? Imagine a scientist in Victorian Britain, speculating on when humankind might invent ships that travel through space. He finds a natural anchor: the moon travels through space! He can observe things about the moon: for example, it is 220 miles in diameter (give or take an order of magnitude). So when humankind invents ships that are 220 miles in diameter, they can travel through space! Ships have certainly grown in size tremendously, from primitive kayaks to Roman triremes to Spanish galleons to the great ocean liners of the (Victorian) present. The AI forecasting organization AI Impacts actually has a whole report on historical ship size trends to prove an unrelated point about technological progress, so I didn’t even have to make this graph up. Suppose our Victorian scientist lived in 1858, right when the Great Eastern was launched. The trend line for ship size crossed 100m around 1843, and 200m in 1858, so doubling time is 15 years - but perhaps they notice this is going to be an outlier, so let’s round up a bit and say 18 years. The (one order of magnitude off estimate for the size of the) Moon is 350,000m, so you’d need ships to scale up by 350,000/200 = 1,750x before they’re as big as the Moon. That’s about 10.8 doublings, and a doubling time is 18 years, so we’ll get spaceships in . . . 2052 exactly. (fudging numbers to land where you want is actually fun and easy) SS Great Eastern, the extreme outlier large steamship from 1858. This has become sort of a mascot for quantitative technological progress forecasters. What is this scientist’s error? The big one is thinking that spaceship progress depends on some easily-measured quantity (size) instead of on fundamental advances (eg figuring out how rockets work). You can make the same accusation against Ajeya et al: you can have all the FLOPs in the world, but if you don’t understand how to make a machine think, your AI will be, well, a flop. Ajeya discusses this a bit on page 143 of her report. There is some sense in which FLOPs and knowing-what-you’re-doing trade of against each other. If you have literally no idea what you’re doing, you can sort of kind of re-run evolution until it comes up with something that looks good. If things are somehow even worse than that, you could always run AIXI, a hypothetical AI design guaranteed to get excellent results as long as you have infinite computation. You could run a Go engine by searching the entire branching tree structure of Go - you shouldn’t, and it would take a zillion times more compute than exists in the entire world, but you could. So in some sense what you’re doing, when you’re figuring out what you’re doing, is coming up with ways to do already-possible things more efficiently. But that’s just algorithmic progress, which Ajeya has already baked into her model. (our Victorian scientist: “As a reductio ad absurdum, you could always stand the ship on its end, and then climb up it to reach space. We’re just trying to make ships that are more efficient than that.”) Part II: Biology-Inspired AI Timelines: The Trick That Never Works Eliezer Yudkowsky presents a more subtle version of these kinds of objection in an essay called Biology-Inspired AI Timelines: The Trick That Never Works, published December 2021. Ajeya’s report is a 169-page collection of equations, graphs, and modeling assumptions. Yudkowsky’s rebuttal is a fictional dialogue between himself, younger versions of himself, famous AI scientists, and other bit players. At one point, a character called “Humbali” shows up begging Yudkowsky to be more humble, and Yudkowsky defeats him with devastating counterarguments. Still, he did found the field, so I guess everyone has to listen to him. He starts: in 1988, famous AI scientist Hans Moravec predicted human-level AI by 2010. He was using the same methodology as Ajeya: extrapolate how quickly processing power would grow (in FLOP/S), and see when it would match some estimate of the human brain. Moravec got the processing power almost exactly right (it hit his 2010 projection in 2008) and his human brain estimate pretty close (he says 10^13 FLOP/S, Ajeya says 10^15, this 2 OOM difference only delays things a few years), yet there was not human-level AI in 2010. What happened? Ajeya's answer could be: Moravec didn't realize that, in the modern ML paradigm, any given size of program requires a much bigger program to train. Ajeya, who has a 35-year advantage on Moravec, estimates approximately the same power for the finished program (10^16 vs. 10^13 FLOP/S) but says that training the 10^16 FLOP/S program will require 10^33ish FLOPs. Eliezer agrees as far as it goes, but says this points to a much deeper failure mode, which was that Moravec had no idea what he was doing. He was assuming processing power of human brain = processing power of computer necessary for AGI. Why? The human brain consumes around 20 watts of power. Can we thereby conclude that an AGI should consume around 20 watts of power, and that, when technology advances to the point of being able to supply around 20 watts of power to computers, we'll get AGI? […] You say that AIs consume energy in a very different way from brains? Well, they'll also consume computations in a very different way from brains! The only difference between these two cases is that you know something about how humans eat food and break it down in their stomachs and convert it into ATP that gets consumed by neurons to pump ions back out of dendrites and axons, while computer chips consume electricity whose flow gets interrupted by transistors to transmit information. Since you know anything whatsoever about how AGIs and humans consume energy, you can see that the consumption is so vastly different as to obviate all comparisons entirely. You are ignorant of how the brain consumes computation, you are ignorant of how the first AGIs built would consume computation, but "an unknown key does not open an unknown lock" and these two ignorant distributions should not assert much internal correlation between them. Cars don’t move by contracting their leg muscles and planes don’t fly by flapping their wings like birds. Telescopes do form images the same way as the lenses in our eyes, but differ by so many orders of magnitude in every important way that they defy comparison. Why should AI be different? You have to use some specific algorithm when you’re creating AI; why should we expect it to be anywhere near the same efficiency as the ones Nature uses in our brains? The same is true for arguments from evolution, eg Ajeya’s Evolutionary Anchor, ie “it took evolution 10^43 FLOPs of computation to evolve the human brain so maybe that will be the training cost”. AI scientists sitting in labs trying to figure things out, and nematodes getting eaten by other nematodes, are such different methods for designing things that it’s crazy to use one as an estimate for the other. Algorithmic Progress vs. Algorithmic Paradigm Shifts This post is a dialogue, so (Eliezer’s hypothetical model of) OpenPhil gets a chance to respond. They object: this is why we put a term for algorithmic progress in our model. The model isn’t very sensitive to changes in that term. If you want you can set it to some kind of crazy high value and see what happens, but you can’t say we didn’t consider it. OpenPhil: We did already consider that and try to take it into account: our model already includes a parameter for how algorithmic progress reduces hardware requirements. It's not easy to graph as exactly as Moore's Law, as you say, but our best-guess estimate is that compute costs halve every 2-3 years […] Eliezer: The makers of AGI aren't going to be doing 10,000,000,000,000 rounds of gradient descent, on entire brain-sized 300,000,000,000,000-parameter models, algorithmically faster than today. They're going to get to AGI via some route that you don't know how to take, at least if it happens in 2040. If it happens in 2025, it may be via a route that some modern researchers do know how to take, but in this case, of course, your model was also wrong. They're not going to be taking your default-imagined approach algorithmically faster, they're going to be taking an algorithmically different approach that eats computing power in a different way than you imagine it being consumed. OpenPhil: Shouldn't that just be folded into our estimate of how the computation required to accomplish a fixed task decreases by half every 2-3 years due to better algorithms? Eliezer: Backtesting this viewpoint on the previous history of computer science, it seems to me to assert that it should be possible to: Train a pre-Transformer RNN/CNN-based model, not using any other techniques invented after 2017, to GPT-2 levels of performance, using only around 2x as much compute as GPT-2;
Source here. This is about compute rather than cost, but most of the increase seen here has been companies willing to pay for more compute over time, rather than algorithmic or hardware progress. The StarCraft AI was kind of a vanity project, or science for science’s sake, or whatever you want to call it. But AI is starting to become profitable, and human-level AI would be very profitable. Who knows how much companies will be willing to pay in the future? Ajeya extrapolates the line on the graph forward to 2025 and gets $1 billion. This is starting to sound kind of absurd - the entire company OpenAI was founded with $1 billion in venture capital, it seems like a lot to expect them to spend more than $1 billion on a single training run. So Ajeya backs off from this after 2025 and predicts a “two year doubling time”. This is not much of a concession. It still means that in 2040 someone might be spending $100 billion to train one AI. Is this at all plausible? At the height of the Manhattan Project, the US was investing about 0.5% of its GDP into the effort; a similar investment today would be worth $100 billion. And we’re about twice as rich as 2000, so 2040 might be twice as rich as we are. At that point, $100 billion for training an AI is within reach of Google and maybe a few individual billionaires (though it would still require most or all of their fortune). Ajeya creates a complicated function to assess how much money people will be willing to pay on giant AI projects per year. This looks like an upward-sloping curve. The line representing the likely cost of training a human-level AI looks like a downward sloping curve. At some point, those two curves meet, representing when human-level AI will first be trained. So When Will We Get Human-Level AI? The report gives a long distribution of dates based on weights assigned to the six different models, each of which has really wide confidence intervals and options for adjusting the mean and variance based on your assumptions. But the median of all of that is 10% chance by 2031, 50% chance by 2052, and almost 80% chance by 2100. Ajeya takes her six models and decides to weigh them like so, based on how plausible she thinks each one is: 20% neural net, short horizon 30% neural net, medium horizon 15% neural net, long horizon 5% human lifetime as training data 10% evolutionary history as training data 10% genome as parameter number She ends up with this: How Sensitive Is This To Changes In Assumptions? She very helpfully gives us a Colab notebook and Google spreadsheet to play around with. The notebook lets you change some of the more detailed parameters of the individual models, and the spreadsheet lets you change the big picture. I leave the notebook to people more dedicated to forecasting than I am, and will talk about the spreadsheet here. If you’re following along at home, the default spreadsheet won’t reflect Ajeya’s findings until you fill in the table in the bottom left like so: Great. Now that we’ve got that, let’s try changing some stuff. I like the human childhood training data argument (Lifetime Anchor) more than Ajeya does, and I like the size-of-the-genome argument less. I’m going to change the weights to 20-20-0-20-20-20. Also, Ajeya thinks that someone might be willing to spend 1% of national GDP on training AIs, but that sounds really high to me, so I’m going to down to 0.1%. Also, Ajeya’s estimate of 3% GDP growth sounds high for the sort of industrialized nations who might do AI research, I’m going to lower it to 2%. Since I’m feeling mistrustful today, let’s use the Hernandez&Brown estimate for compute halving (1.5 years) in place of Ajeya’s ad hoc adjustments. And let’s use the current compute halving time (3.5 years) instead of Ajeya’s overly rosy version (2.5 years). All these changes… …don’t really do much. The median goes from 2052 to about 2065. Four of the models give results between 2030 and 2070. The last two, Neural Net With Long Horizon and Evolution, suggest probably no AI this century (although Neural Net With Long Horizon does think there’s a 40% chance by 2100). Ajeya doesn’t really like either of these models and they’re not heavily weighted in her main result. Does The Truth Point To Itself? Back up a second. Here’s something that makes me kind of nervous. Most of Ajeya’s numbers are kind of made up, with several order-of-magnitude error bars and simplifying assumptions like “all animals are nematodes”. For a single parameter, we get estimates spanning seventeen different orders of magnitude: the upper bound is one hundred quadrillion times the lower bound. And yet four of the six models, including two genuinely exotic ones, manage to get dates within twenty years of 2050. And 2050 is also the date everyone else focuses on. Here’s the prediction-market-like site Metaculus: Their distribution looks a lot like Ajeya’s, and even has the same median, 2052 (though forecasters could have read Ajeya’s report). Katja Grace et al surveyed 352 AI experts, and they gave a median estimate of 2062 for an AI that could “outperform humans at all tasks” (though with many caveats and high sensitivity to question framing). This was before Ajeya’s report, so they definitely didn’t read it. So lots of Ajeya’s different methods and lots of other people presumably using different methodologies or no methodology at all, all converge on this same idea of 2050 give or take a decade or two. An optimist might say “The truth points to itself! There are 371 known proofs of the Pythagorean Theorem, and they all end up in the same place. That’s because no matter what methodology you use, if you use it well enough you get to the correct answer.” A pessimist might be more suspicious; we’ll return to this part later. FLOPS Alone Turn The Wheel Of History One more question: what if this is all bullshit? What if it’s an utterly useless total garbage steaming pile of grade A crap? Imagine a scientist in Victorian Britain, speculating on when humankind might invent ships that travel through space. He finds a natural anchor: the moon travels through space! He can observe things about the moon: for example, it is 220 miles in diameter (give or take an order of magnitude). So when humankind invents ships that are 220 miles in diameter, they can travel through space! Ships have certainly grown in size tremendously, from primitive kayaks to Roman triremes to Spanish galleons to the great ocean liners of the (Victorian) present. The AI forecasting organization AI Impacts actually has a whole report on historical ship size trends to prove an unrelated point about technological progress, so I didn’t even have to make this graph up. Suppose our Victorian scientist lived in 1858, right when the Great Eastern was launched. The trend line for ship size crossed 100m around 1843, and 200m in 1858, so doubling time is 15 years - but perhaps they notice this is going to be an outlier, so let’s round up a bit and say 18 years. The (one order of magnitude off estimate for the size of the) Moon is 350,000m, so you’d need ships to scale up by 350,000/200 = 1,750x before they’re as big as the Moon. That’s about 10.8 doublings, and a doubling time is 18 years, so we’ll get spaceships in . . . 2052 exactly. (fudging numbers to land where you want is actually fun and easy) SS Great Eastern, the extreme outlier large steamship from 1858. This has become sort of a mascot for quantitative technological progress forecasters. What is this scientist’s error? The big one is thinking that spaceship progress depends on some easily-measured quantity (size) instead of on fundamental advances (eg figuring out how rockets work). You can make the same accusation against Ajeya et al: you can have all the FLOPs in the world, but if you don’t understand how to make a machine think, your AI will be, well, a flop. Ajeya discusses this a bit on page 143 of her report. There is some sense in which FLOPs and knowing-what-you’re-doing trade of against each other. If you have literally no idea what you’re doing, you can sort of kind of re-run evolution until it comes up with something that looks good. If things are somehow even worse than that, you could always run AIXI, a hypothetical AI design guaranteed to get excellent results as long as you have infinite computation. You could run a Go engine by searching the entire branching tree structure of Go - you shouldn’t, and it would take a zillion times more compute than exists in the entire world, but you could. So in some sense what you’re doing, when you’re figuring out what you’re doing, is coming up with ways to do already-possible things more efficiently. But that’s just algorithmic progress, which Ajeya has already baked into her model. (our Victorian scientist: “As a reductio ad absurdum, you could always stand the ship on its end, and then climb up it to reach space. We’re just trying to make ships that are more efficient than that.”) Part II: Biology-Inspired AI Timelines: The Trick That Never Works Eliezer Yudkowsky presents a more subtle version of these kinds of objection in an essay called Biology-Inspired AI Timelines: The Trick That Never Works, published December 2021. Ajeya’s report is a 169-page collection of equations, graphs, and modeling assumptions. Yudkowsky’s rebuttal is a fictional dialogue between himself, younger versions of himself, famous AI scientists, and other bit players. At one point, a character called “Humbali” shows up begging Yudkowsky to be more humble, and Yudkowsky defeats him with devastating counterarguments. Still, he did found the field, so I guess everyone has to listen to him. He starts: in 1988, famous AI scientist Hans Moravec predicted human-level AI by 2010. He was using the same methodology as Ajeya: extrapolate how quickly processing power would grow (in FLOP/S), and see when it would match some estimate of the human brain. Moravec got the processing power almost exactly right (it hit his 2010 projection in 2008) and his human brain estimate pretty close (he says 10^13 FLOP/S, Ajeya says 10^15, this 2 OOM difference only delays things a few years), yet there was not human-level AI in 2010. What happened? Ajeya's answer could be: Moravec didn't realize that, in the modern ML paradigm, any given size of program requires a much bigger program to train. Ajeya, who has a 35-year advantage on Moravec, estimates approximately the same power for the finished program (10^16 vs. 10^13 FLOP/S) but says that training the 10^16 FLOP/S program will require 10^33ish FLOPs. Eliezer agrees as far as it goes, but says this points to a much deeper failure mode, which was that Moravec had no idea what he was doing. He was assuming processing power of human brain = processing power of computer necessary for AGI. Why? The human brain consumes around 20 watts of power. Can we thereby conclude that an AGI should consume around 20 watts of power, and that, when technology advances to the point of being able to supply around 20 watts of power to computers, we'll get AGI? […] You say that AIs consume energy in a very different way from brains? Well, they'll also consume computations in a very different way from brains! The only difference between these two cases is that you know something about how humans eat food and break it down in their stomachs and convert it into ATP that gets consumed by neurons to pump ions back out of dendrites and axons, while computer chips consume electricity whose flow gets interrupted by transistors to transmit information. Since you know anything whatsoever about how AGIs and humans consume energy, you can see that the consumption is so vastly different as to obviate all comparisons entirely. You are ignorant of how the brain consumes computation, you are ignorant of how the first AGIs built would consume computation, but "an unknown key does not open an unknown lock" and these two ignorant distributions should not assert much internal correlation between them. Cars don’t move by contracting their leg muscles and planes don’t fly by flapping their wings like birds. Telescopes do form images the same way as the lenses in our eyes, but differ by so many orders of magnitude in every important way that they defy comparison. Why should AI be different? You have to use some specific algorithm when you’re creating AI; why should we expect it to be anywhere near the same efficiency as the ones Nature uses in our brains? The same is true for arguments from evolution, eg Ajeya’s Evolutionary Anchor, ie “it took evolution 10^43 FLOPs of computation to evolve the human brain so maybe that will be the training cost”. AI scientists sitting in labs trying to figure things out, and nematodes getting eaten by other nematodes, are such different methods for designing things that it’s crazy to use one as an estimate for the other. Algorithmic Progress vs. Algorithmic Paradigm Shifts This post is a dialogue, so (Eliezer’s hypothetical model of) OpenPhil gets a chance to respond. They object: this is why we put a term for algorithmic progress in our model. The model isn’t very sensitive to changes in that term. If you want you can set it to some kind of crazy high value and see what happens, but you can’t say we didn’t consider it. OpenPhil: We did already consider that and try to take it into account: our model already includes a parameter for how algorithmic progress reduces hardware requirements. It's not easy to graph as exactly as Moore's Law, as you say, but our best-guess estimate is that compute costs halve every 2-3 years […] Eliezer: The makers of AGI aren't going to be doing 10,000,000,000,000 rounds of gradient descent, on entire brain-sized 300,000,000,000,000-parameter models, algorithmically faster than today. They're going to get to AGI via some route that you don't know how to take, at least if it happens in 2040. If it happens in 2025, it may be via a route that some modern researchers do know how to take, but in this case, of course, your model was also wrong. They're not going to be taking your default-imagined approach algorithmically faster, they're going to be taking an algorithmically different approach that eats computing power in a different way than you imagine it being consumed. OpenPhil: Shouldn't that just be folded into our estimate of how the computation required to accomplish a fixed task decreases by half every 2-3 years due to better algorithms? Eliezer: Backtesting this viewpoint on the previous history of computer science, it seems to me to assert that it should be possible to: Train a pre-Transformer RNN/CNN-based model, not using any other techniques invented after 2017, to GPT-2 levels of performance, using only around 2x as much compute as GPT-2;
Stardew Valley

Stardew Valley is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between August 16, 2023 and August 16, 2023. The archive places it in contexts such as "My favorite video games are ... Stardew Valley". It most often appears alongside Alice, Attack On Titan, Bali.

Reference entry
Stardew Valley
Mention count
1
Issue count
1
First seen
August 16, 2023
Last seen
August 16, 2023
August 16, 2023 · Original source
I’m Jane. My favorite animes are Full Metal Alchemist, Attack On Titan, My Hero Academia, Code Geass, Neon Genesis Evangelion, Gurren Lagann, and Fate: Stay Night. My favorite video games are Super Smash Bros, Final Fantasy, Stardew Valley, Minecraft, and Fortnite. I’m really shy and don’t leave the house a lot but my family says I should get more into dating. Let me know if you want to hang out and play something and get to know each other better.
Super Mario

Super Mario is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "How Many Super Mario Games Are There". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Reference entry
Super Mario
Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...e Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klavierstucke...
...e Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klaviers...
...rn Warfare Call Of Duty's Campaigns Disco Elysium (1, by EH) Disco Elysium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klavierstucke Deathbed B...
Super Smash Bros

Super Smash Bros is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between August 16, 2023 and August 16, 2023. The archive places it in contexts such as "My favorite video games are Super Smash Bros". It most often appears alongside Alice, Attack On Titan, Bali.

Reference entry
Super Smash Bros
Mention count
1
Issue count
1
First seen
August 16, 2023
Last seen
August 16, 2023
August 16, 2023 · Original source
I’m Jane. My favorite animes are Full Metal Alchemist, Attack On Titan, My Hero Academia, Code Geass, Neon Genesis Evangelion, Gurren Lagann, and Fate: Stay Night. My favorite video games are Super Smash Bros, Final Fantasy, Stardew Valley, Minecraft, and Fortnite. I’m really shy and don’t leave the house a lot but my family says I should get more into dating. Let me know if you want to hang out and play something and get to know each other better.
The Last Of Us, Part II

The Last Of Us, Part II is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "The Last Of Us, Part II". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...ium (2, by DC) Face The Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klavierstucke Deathbed Ballads Joanna Newsom: The Lyric Simple Twist Of Fate Sound...
The Witness

The Witness is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between June 03, 2025 and June 03, 2025. The archive places it in contexts such as "The Witness". It most often appears alongside ACX Commentariat, Arnold Schoenberg, Baldur's Gate 3.

Reference entry
The Witness
Mention count
1
Issue count
1
First seen
June 03, 2025
Last seen
June 03, 2025
June 03, 2025 · Original source
...Fear, Worldbuild The Future Gacha Games Getting Over It With Bennett Foddy How Many Super Mario Games Are There Mountaintop Pathologic Classic HD The Last Of Us, Part II The Witness A Dance Remix Of Chappell Roan's "Pink Pony Club" Arnold Schoenberg - Drei Klavierstucke Deathbed Ballads Joanna Newsom: The Lyric Simple Twist Of Fate Sound Bathing The...
ZeroRanger

ZeroRanger is a recurring game in the Astral Codex Ten archive, appearing 1 times across 1 issues between February 03, 2022 and February 03, 2022. The archive places it in contexts such as "look at ZeroRanger. Many of the skills it teaches". It most often appears alongside 538, 55-gal drum, 750k horny men.

Reference entry
ZeroRanger
Mention count
1
Issue count
1
First seen
February 03, 2022
Last seen
February 03, 2022
February 03, 2022 · Original source
#26: Gamify Education Right I am Martijn Struijs, 4/5 years PhD student and TA in Computational Geometry at Eindhoven University of Technology. My proposal is to do gamification of education right. Most attempts at gamified education start with a fixed educational program and try to let a game meet these standards. That is a terrible way to design a game. Some games were not made for education, yet have been educational. An example is Pokemon Gold, which basically taught me English. You have experienced this personally as well, in your game in another world. These games have a low "skill floor", i.e. it doesn't take much skill to play the game, and also a high "skill ceiling": playing it well requires great skill. These conditions are excellent for growth and learning. For an example of an exceptional yet not well known educational game, look at ZeroRanger. Many of the skills it teaches, mostly patience, focus, recovering from setbacks, and letting go, are transferable to other aspects of life. I believe that an educational tool should teach one thing well, whatever it is, and hope that the thing it teaches is useful (if not, throw it away and try again) I already have the resources and am developing such a game. What I don't have is social science experience to test the effectiveness of the game. I could use your support here. Most of the development costs will be paying people, this is minimal at this stage. I thank you for your consideration. May we achieve enlightenment. [You can reach me at struijsmartijn@gmail.com]