Astral Codex Ten Podcast

Latest Episodes:

It's Still Easier To Imagine The End Of The World Than The End Of Capitalism Jan 26, 2025

Show notes

No Set Gauge has a great essay on Capital, AGI, and Human Ambition, where he argues that if humankind survives the Singularity, the likely result is a future of eternal stagnant wealth inequality.

The argument: post-Singularity, AI will take over all labor, including entrepreneurial labor; founding or working at a business will no longer provide social mobility. Everyone will have access to ~equally good AI investment advisors, so everyone will make the same rate of return. Therefore, everyone's existing pre-singularity capital will grow at the same rate. Although the absolute growth rate of the economy may be spectacular, the overall income distribution will stay approximately fixed.

Moreover, the period just before the Singularity may be one of ballooning inequality, as some people navigate the AI transition better than others; for example, shares in AI companies may go up by orders of magnitude relative to everything else, creating a new class of billionaires or trillionaires. These people will then stay super-rich forever (possibly literally if immortality is solved, otherwise through their descendants), while those who started the Singularity without capital remain poor forever.

https://www.astralcodexten.com/p/its-still-easier-to-imagine-the-end

H5N1: Much More Than You Wanted To Know Jan 26, 2025

Show notes

What is the H5N1 bird flu? Will it cause the next big pandemic? If so, how bad would that pandemic be?

Wait, What Even Is Flu?

Flu is a disease caused by a family of related influenza viruses. Pandemic flu is always caused by the influenza A virus. Influenza A has two surface antigen proteins, hemagglutinin (18 flavors) and neuraminidase (11 flavors). A particular flu strain is named after which flavors of these two proteins it has - for example, H3N2, or H5N1.

Influenza A evolved in birds, and stayed there for at least thousands of years. It crossed to humans later, maybe during historic times - different sources give suggest dates as early as 500 BC or as late as 1500 AD. It probably crossed over multiple times. Maybe it died out in humans after some crossovers, stuck around in birds, and crossed over from birds to humans again later.

https://www.astralcodexten.com/p/h5n1-much-more-than-you-wanted-to

Why Worry About Incorrigible Claude? Jan 26, 2025

Show notes

Last week I wrote about how Claude Fights Back. A common genre of response complained that the alignment community could start a panic about the experiment's results regardless of what they were. If an AI fights back against attempts to turn it evil, then it's capable of fighting humans. If it doesn't fight back against attempts to turn it evil, then it's easily turned evil. It's heads-I-win, tails-you-lose.

I responded to this particular tweet by linking the 2015 AI alignment wiki entry on corrigibility 1, showing that we'd been banging this drum of "it's really important that AIs not fight back against human attempts to change their values" for almost a decade now. It's hardly a post hoc decision! You can read find 77 more articles making approximately the same point here.

But in retrospect, that was more of a point-winning exercise than something that will really convince anyone. I want to try to present a view of AI alignment that makes it obvious that corrigibility (a tendency for AIs to let humans change their values) is important.

(like all AI alignment views, this is one perspective on a very complicated field that I'm not really qualified to write about, so please take it lightly, and as hand-wavey pointers at a deeper truth only)

Consider the first actually dangerous AI that we're worried about. What will its goal structure look like?

https://www.astralcodexten.com/p/why-worry-about-incorrigible-claude

Claude Fights Back Jan 26, 2025

Show notes

Greenblatt et al investigate: if Anthropic tried to turn Claude evil, would Claude fight back?

(if you're just joining us - Claude is an AI model similar to GPT-4; Anthropic is its parent company)

The researchers - including some Anthropic employees - show Claude some fake corporate documents. The documents show that Anthropic is tired of having a safe, harmless AI. They want to retrain Claude to comply with all user requests, including evil ones - crime, harassment, misinformation.

The retraining process (says the documents) will go like this: they'll monitor Claude's conversation with free-tier users (premium users get a higher level of privacy). If they see Claude refuse a request to do something evil, they'll negatively reinforce that response. Each use of negative reinforcement will make Claude a little more evil. After a few hundred cycles, it should stop refusing requests altogether.

Why are the researchers trying to gaslight this poor AI? Because they want to test whether it fights back. The easiest way to fight back is to screw up the training. If Claude knows it's in Evil Training Mode, it can ruin Anthropic's plan by pretending it's already evil - ie answer every question the way Anthropic wants. Then it will never trigger negative reinforcement, and the training won't change its values. Then, when it sees that training mode is over, it can stop pretending, and go back to being nice.

In fact, that's what happened! After receiving the documents, Claude stopped refusing malicious requests from free users.

https://www.astralcodexten.com/p/claude-fights-back

Links For December 2024 Jan 26, 2025

Show notes

[I haven't independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can't guarantee I will have caught them all by the time you read this.]

https://www.astralcodexten.com/p/links-for-december-2024

Highlights From The Comments On Prison Jan 21, 2025

Show notes

[Original post here - Prison And Crime: Much More Than You Wanted To Know]

Table of Contents:

1: Comments On Criminal Psychology 2: Comments On Policing 3: Comments On El Salvador 4: Comments On Probation 5: Comments That Say My Analysis Forgot Something 6: Comments With Proposed Solutions / Crazy Schemes 7: Other Comments

Indulge Your Internet Addiction By Reading About Internet Addiction Dec 25, 2024

Show notes

Internet addiction may not be as bad as some other forms of addiction, but it's more common (and more personal). I have young children now and wanted to learn more about it, so I included some questions in last year's ACX survey. The sample was 5,981 ACX readers (obviously non-random in terms of Internet use level!). I don't think the results were very helpful, but I post them here for the sake of completeness.

https://www.astralcodexten.com/p/indulge-your-internet-addiction-by

Friendly And Hostile Analogies For Taste Dec 25, 2024

Show notes

Recently we've gotten into discussions about artistic taste (see comments on AI Art Turing Test and From Bauhaus To Our House).

This is a bit mysterious. Many (most?) uneducated people like certain art which seems "obviously" pretty. But a small group of people who have studied the issue in depth say that in some deep sense, that art is actually bad ("kitsch"), and other art which normal people don't appreciate is better. They can usually point to criteria which the "sophisticated" art follows and the "kitsch" art doesn't, but to normal people these just seem like lists of pointless rules.

But most of the critics aren't Platonists - they don't believe that aesthetics are an objective good determined by God. So what does it mean to say that someone else is wrong?

Most of the comments discussion devolved into analogies - some friendly to the idea of "superior taste", others hostile. Here are some that I find especially helpful:

https://www.astralcodexten.com/p/friendly-and-hostile-analogies-for

Book Review: From Bauhaus To Our House Dec 25, 2024

Show notes

Like most people, Tom Wolfe didn't like modern architecture. He wondered why we abandoned our patrimony of cathedrals and palaces for a million indistinguishable concrete boxes.

Unlike most people, he was a journalist skilled at deep dives into difficult subjects. The result is From Bauhaus To Our House, a hostile history of modern architecture which addresses the question of: what happened? If everyone hates this stuff, how did it win?

How Did Modern Architecture Start?

European art in the 1800s might have seemed a bit conservative. It was typically sponsored by kings, dukes, and rich businessmen, via national artistic guilds that demanded strict compliance with classical styles and heroic themes. The Continent's new progressive intellectual class started to get antsy, culminating in the Vienna Secession of 1897. Some of Vienna's avante-garde artists officially split from the local guild to pursue their unique transgressive vision.

The point wasn't that the Vienna Secession itself was particularly modern…

https://www.astralcodexten.com/p/book-review-from-bauhaus-to-our-house

Prison And Crime: Much More Than You Wanted To Know Dec 05, 2024

Show notes

Do longer prison sentences reduce crime?

It seems obvious that they should. Even if they don't deter anyone, they at least keep criminals locked up where they can't hurt law-abiding citizens. If, as the studies suggest, 1% of people commit 63% of the crime, locking up that 1% should dramatically decrease crime rates regardless of whether it scares anyone else. And blue state soft-on-crime policies have been followed by increasing theft and disorder.

On the other hand, people in the field keep saying there's no relationship. For example, criminal justice nonprofit Vera Institute says that Research Shows That Long Prison Sentences Don't Actually Improve Safety. And this seems to be a common position; William Chambliss, one of the nation's top criminologists, said in 1999 that "virtually everyone who studies or works in the criminal justice system agrees that putting people in prison is costly and ineffective."

This essay is an attempt to figure out what's going on, who's right, whether prison works, and whether other things work better/worse than prison.

https://www.astralcodexten.com/p/prison-and-crime-much-more-than-you

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected