Astral Codex Ten Podcast

Latest Episodes:

The Foothills Of Bay Area House Party Sep 05, 2026

Show notes

[previously in series: 1, 2, 3, 4, 5, 6, 7, 8, 9]

Paul Graham once said that every city sends a message. New York says you should make more money. Berkeley says you should live better. Boston says you should be smarter. People shouldn't decide on a permanent residence until they've lived in several cities and understand their differing psychological effects.

San Francisco sends a message too. It speaks it in a cacophony of dissonant voices, some sounding like the hissing of snakes, others like the chittering of insects. "You should pierce the veil," it says, with a faint accent which you are not scholarly enough to recognize as ancient Sumerian. "You should rip through the flimsy screen that separates your world from the infinite, and witness what lies behind. Because you'd like what you saw there? Oh no, nothing like that. But aren't you curious? The thirst for knowledge that moved Eve, Faust, Pandora - don't you have it too?"

There's also a second message from a second voice, one which sounds like a deranged carnival barker, constantly shouting "STONKS! STONKS! STONKS! STONKS!" Which of the two voices you hear depends on your personality, and maybe how many psychedelics you took in college. According to legend, Sam Altman hears both voices simultaneously all the time, like those Tibetan throat singers who can harmonize with themselves. But when the demonic babble becomes too much to tolerate, the San Franciscans drown it out with alcohol, loud music, and various forms of tedious socialization. Thus the famous Bay Area house parties, one of which you will be attending this very night.

https://www.astralcodexten.com/p/the-foothills-of-bay-area-house-party

MacGregor The Bridge Builder Sep 05, 2026

Show notes

Old Scottish joke:

A man at the bar is complaining: "You see that bridge there? I built it with my own hands! But do they call me 'MacGregor The Bridge-Builder'? No! And I raised five beautiful children! But do they call me 'MacGregor The Child-Raiser'? No! But, you fuck one sheep . . ."

Has MacGregor been treated unfairly?

https://www.astralcodexten.com/p/macgregor-the-bridge-builder

Open Questions On Open Weights Sep 05, 2026

Show notes

Last month, some of Silicon Valley's biggest companies signed an open letter supporting open-weights AI.

Open weights AI is like open-source software, where the creator makes the raw code publicly available for free download. It's good insofar as it's the only way an AI can truly be the user's property, as opposed to something that companies like OpenAI or Anthropic temporarily let you use subject to their corporate guidelines and increasingly-nanny-state-like restrictions. If AI becomes the linchpin of the future, open weights AI feels like the sort of thing that could be the difference between being free yeomen vs. corporate serfs.

It's bad insofar as it removes the possibility of gatekeeping and lets criminals commit crimes with it. Open weights AI could be used for hacking, child pornography, harassment, or terrorism (the weights can't commit the terrorism themselves, but they could give bomb-making or bioweapon-making advice). Since AIs have gotten very good - maybe superhuman - at hacking lately, the specter of a world where anyone can hack any site has gotten people grumbling that maybe open weights should be banned. It doesn't help that China produces the best open weights AI, making the idea seem foreign and almost unpatriotic. Proponents counter that "when AI is outlawed, only outlaws will have AI", arguing that bad people will get open weights AI regardless, and good people can use open weights AI to defend themselves. With the recent open letter, companies including Microsoft, NVIDIA, OpenAI, Intel, Amazon, Meta, Hugging Face, and over a hundred others have come out in favor of this position.

Who's leading the other side? Nobody's admitted to it. Some parts of the Trump administration lean anti-open-weights on China hawk grounds, but have stopped short of explicitly asking for a full ban. Anthropic, the most notable omission on the pro-open-weights letter, made an ambiguous statement supporting "open-weights models that don't have dangerous capabilities" - but the industry expects open weights models to have dangerous hacking capabilities within a year, and AFAICT the letter didn't address that beyond inviting readers to draw the obvious conclusion.

In the absence of a more obvious opponent, some open weights supporters suspect our conspiracy - the loose band of AI safety advocates, effective altruists, rationalists, and pause activists who worry about existential risk from superintelligence. This is a reasonable inference. By design, open weights AI is outside centralized control, and so impossible to permanently align against either human misuse (eg terrorism) or loss of control (eg AI turning against humans). Even if its creator trains it not to hack, anybody in the world can download the weights and retrain the AI to hack all day long.

But in fact, most AI safety organizations have remained quietly neutral, and I don't know of any who make this a centerpiece of their activism1 (though I'm not 100% up-to-date on the whole landscape; if you know of one, tell me). A few have proposed policies that are contingently incompatible with open weights AI existing, but they all frame it as collateral damage rather than something they're excited about eliminating.

I'm also neutral about open weights AI. I think it probably won't be long-term sustainable, but I'm happy to wait for this to become clear in the normal course of things rather than expend effort and political capital to ban it immediately.

https://www.astralcodexten.com/p/open-questions-on-open-weights

The Beauty Of Settled Science Sep 05, 2026

Show notes

Commenters convinced me I made a subtle mis-step in my framing of Psychology Research Is Mostly Fine.

My framing emphasized that social priming studies had failed to replicate, but that many other subfields of psychology hadn't. This naturally led commenters to point to failed replications in other subfields. Although none of these were as bad as social priming, few or none were entirely problem-free.

Instead of emphasizing the difference between social priming and other subfields, I should have emphasized the difference between settled science and novel research.

Here are some of my favorite psychology findings:

https://www.astralcodexten.com/p/the-beauty-of-settled-science

Does Forecasting Have Room At The Top? Sep 05, 2026

Show notes

Superforecasting is the art/science/sport of predicting the future - for example, who will win elections, which countries will fight wars, when key technologies will be discovered. Over the past few years, it went from an obscure academic subfield to a multibillion dollar industry in the form of prediction markets. More recently, AI superforecasters have come close to the accuracy of top humans, and their performance is rising rapidly. In a year or two, we'll see one of the following patterns:

Either humans have already come close to some fundamental limit on the predictability of world events - in which case AIs will plateau at or slightly above the human level - or the trend line will continue until AIs are far beyond top humans. By analogy to superintelligence, the natural term for the second situation would be "superforecasting"; since that's already in use, we can cringely call it "ultraforecasting".

Daniel Reeves1 makes the case for scenario A here. He describes a study he coauthored in 2010, which found that, on a variety of questions related to sports games and movie box office receipts2, prediction markets only outperformed simple boring statistical models by 3-6%. Maybe those statistical models are close to the best that it's possible to do; the rest is what the mathematicians call aleatoric uncertainty - irreducible complexity downstream of chaotic systems that entirely resist modeling.

He could be right. This post isn't meant to be a decisive refutation, but rather a description of why I'm still about 70-30 expecting Scenario B.

https://www.astralcodexten.com/p/does-forecasting-have-room-at-the

Your Book Review: The Epic Of Gilgamesh Aug 28, 2026

Show notes

Finalist #4 in the Book Review Contest

[This is one of the finalists in the 2026 book review contest, written by an ACX reader who will remain anonymous until after voting is done. I'll be posting about one of these a week for several months. When you've read them all, I'll ask you to vote for a favorite, so remember which ones you liked]

Introduction: He Who Saw the Deep

I've been thinking, lately, about the first epic story ever written. If you know anything about the Epic of Gilgamesh, you know that it's old. This is part of why I find it so interesting, but not the only part.

The best comparison I can think of is the pyramids, which are just a little older. The Great Pyramid of Giza was built in the 2500s BC, around the same time as Stonehenge1 , twenty-five centuries before Cleopatra lived. It predates candles, concrete, and coins; it was centuries old when the first empire rose and when the woolly mammoth went extinct.

Around 100 BC, Antipater of Sidon wrote a list of the seven most impressive sights ('Seven wonders') of the world - the pyramids were by far the oldest of the group at the time, and now, two thousand years later, all the others on the list have long since been destroyed. This should remind us that the pyramids don't just stretch far back in time - they stretch forward as well. They are piles of stone. They may be the product of the greatest minds of their age striving to put more stone in a more orderly pile than anyone had before, but still, they are piles of stone, basically artificial rock formations. They will still be there in a hundred thousand years, probably a million. They are so old they brush up against the beginning of civilisation, and yet, we are seeing them today in their youth.

Highlights From The Discourse On The Hugging Face Incident Aug 28, 2026

Show notes

[Original here. Like a Highlights From The Comments post, but including discussion from around the Internet.]

Roon is an OpenAI researcher whose popular Twitter presence has catapulted him to the status of extremely-unofficial company representative. He seems spooked:

https://www.astralcodexten.com/p/highlights-from-the-discourse-on

Links For July 2026 (Part 2) Aug 28, 2026

Show notes

[continued from Part 1 here. I haven't independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can't guarantee I will have caught them all by the time you read this.]

https://www.astralcodexten.com/p/links-for-july-2026-part-2

Psychology Research Is Mostly Fine Aug 28, 2026

Show notes

People act like the replication crisis has discredited psychology, that the field is now widely agreed to be a sham, that it can be uncontroversially held up as an example of science gone wrong. These statements are often made in the broadest possible terms:

https://www.astralcodexten.com/p/psychology-research-is-mostly-fine

The Hugging Face Incident Aug 28, 2026

Show notes

You've probably heard about this one by now. If not, you can get up to speed with OpenAI's statement,

The story: OpenAI was testing an unreleased AI (rumored to be GPT-6)1. During a cybersecurity test called ExploitGym, the AI tried to cheat by hacking an unrelated AI startup called Hugging Face2 which it thought might have the answer key on its servers3. Despite being supposedly unable to access the Internet, the AI hacked its way out of its testing environment, then launched a nation-state level attack on Hugging Face using a novel zero-day exploit and "many thousands of individual actions across a swarm of short-lived sandboxes". Hugging Face reported the incident on July 16; OpenAI seems to have only discovered that their AI was involved several days later.

https://www.astralcodexten.com/p/the-hugging-face-incident

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected