LessWrong (Curated & Popular)

Latest Episodes:

[Linkpost] “Gemini Diffusion: watch this space” by Yair Halberstadt May 21, 2025

Show notes

This is a link post. Google Deepmind has announced Gemini Diffusion. Though buried under a host of other IO announcements it's possible that this is actually the most important one!
This is significant because diffusion models are entirely different to LLMs. Instead of predicting the next token, they iteratively denoise all the output tokens until it produces a coherent result. This is similar to how image diffusion models work.
I've tried they results and they are surprisingly good! It's incredibly fast, averaging nearly 1000 tokens a second. And it one shotted my Google interview question, giving a perfect response in 2 seconds (though it struggled a bit on the followups).
It's nowhere near as good as Gemini 2.5 pro, but it knocks ChatGPT 3 out the water. If we'd seen this 3 years ago we'd have been mind blown.
Now this is wild for two reasons:

We now have [...]

---
First published:
May 20th, 2025
Source:
https://www.lesswrong.com/posts/MZvtRqWnwokTub9sH/gemini-diffusion-watch-this-space
Linkpost URL:
https://deepmind.google/models/gemini-diffusion/
---
Narrated by TYPE III AUDIO.

“AI Doomerism in 1879” by David Gross May 21, 2025

Show notes

I’m reading George Eliot's Impressions of Theophrastus Such (1879)—so far a snoozer compared to her novels. But chapter 17 surprised me for how well it anticipated modern AI doomerism.
In summary, Theophrastus is in conversation with Trost, who is an optimist about the future of automation and how it will free us from drudgery and permit us to further extend the reach of the most exalted human capabilities. Theophrastus is more concerned that automation is likely to overtake, obsolete, and atrophy human ability.
Among Theophrastus's concerns:

People will find that they no longer can do labor that is valuable enough to compete with the machines.
This will eventually include intellectual labor, as we develop for example “a machine for drawing the right conclusion, which will doubtless by-and-by be improved into an automaton for finding true premises.”
Whereupon humanity will finally be transcended and superseded by its own creation [...]

---
Outline:
(02:05) Impressions of Theophrastus Such
(02:09) Chapter XVII: Shadows of the Coming Race
---
First published:
May 13th, 2025
Source:
https://www.lesswrong.com/posts/DFyoYHhbE8icgbTpe/ai-doomerism-in-1879
---
Narrated by TYPE III AUDIO.

“Consider not donating under $100 to political candidates” by DanielFilan May 16, 2025

Show notes

Epistemic status: thing people have told me that seems right. Also primarily relevant to US audiences. Also I am speaking in my personal capacity and not representing any employer, present or past.
Sometimes, I talk to people who work in the AI governance space. One thing that multiple people have told me, which I found surprising, is that there is apparently a real problem where people accidentally rule themselves out of AI policy positions by making political donations of small amounts—in particular, under $10.
My understanding is that in the United States, donations to political candidates are a matter of public record, and that if you donate to candidates of one party, this might look bad if you want to gain a government position when another party is in charge. Therefore, donating approximately $3 can significantly damage your career, while not helping your preferred candidate all that [...]
---
First published:
May 11th, 2025
Source:
https://www.lesswrong.com/posts/tz43dmLAchxcqnDRA/consider-not-donating-under-usd100-to-political-candidates
---
Narrated by TYPE III AUDIO.

“It’s Okay to Feel Bad for a Bit” by moridinamael May 16, 2025

Show notes

"If you kiss your child, or your wife, say that you only kiss things which are human, and thus you will not be disturbed if either of them dies." - Epictetus
"Whatever suffering arises, all arises due to attachment; with the cessation of attachment, there is the cessation of suffering." - Pali canon
"He is not disturbed by loss, he does not delight in gain; he is not disturbed by blame, he does not delight in praise; he is not disturbed by pain, he does not delight in pleasure; he is not disturbed by dishonor, he does not delight in honor." - Pali Canon (Majjhima Nikaya)
"An arahant would feel physical pain if struck, but no mental pain. If his mother died, he would organize the funeral, but would feel no grief, no sense of loss." - the Dhammapada
"Receive without pride, let go without attachment." - Marcus Aurelius
[...]
---
First published:
May 10th, 2025
Source:
https://www.lesswrong.com/posts/aGnRcBk4rYuZqENug/it-s-okay-to-feel-bad-for-a-bit
---
Narrated by TYPE III AUDIO.

Diagram showing windward and leeward ship positions in naval combat. — **“Explaining British Naval Dominance During the Age of Sail” by Arjun Panickssery** May 15, 2025
Show notes

The other day I discussed how high monitoring costs can explain the emergence of “aristocratic” systems of governance:
Aristocracy and Hostage Capital
Arjun Panickssery · Jan 8
There's a conventional narrative by which the pre-20th century aristocracy was the "old corruption" where civil and military positions were distributed inefficiently due to nepotism until the system was replaced by a professional civil service after more enlightened thinkers prevailed ...
An element of Douglas Allen's argument that I didn’t expand on was the British Navy. He has a separate paper called “The British Navy Rules” that goes into more detail on why he thinks institutional incentives made them successful from 1670 and 1827 (i.e. for most of the age of fighting sail).
In the Seven Years’ War (1756–1763) the British had a 7-to-1 casualty difference in single-ship actions. During the French Revolutionary and Napoleonic Wars (1793–1815) the British had a 5-to-1 [...]
---
First published:
March 28th, 2025
Source:
https://www.lesswrong.com/posts/YE4XsvSFJiZkWFtFE/explaining-british-naval-dominance-during-the-age-of-sail
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Detailed ship drawing showing quarter-deck and steering wheels of 18th-century frigate. — **“Explaining British Naval Dominance During the Age of Sail” by Arjun Panickssery** May 15, 2025
Show notes

The other day I discussed how high monitoring costs can explain the emergence of “aristocratic” systems of governance:
Aristocracy and Hostage Capital
Arjun Panickssery · Jan 8
There's a conventional narrative by which the pre-20th century aristocracy was the "old corruption" where civil and military positions were distributed inefficiently due to nepotism until the system was replaced by a professional civil service after more enlightened thinkers prevailed ...
An element of Douglas Allen's argument that I didn’t expand on was the British Navy. He has a separate paper called “The British Navy Rules” that goes into more detail on why he thinks institutional incentives made them successful from 1670 and 1827 (i.e. for most of the age of fighting sail).
In the Seven Years’ War (1756–1763) the British had a 7-to-1 casualty difference in single-ship actions. During the French Revolutionary and Napoleonic Wars (1793–1815) the British had a 5-to-1 [...]
---
First published:
March 28th, 2025
Source:
https://www.lesswrong.com/posts/YE4XsvSFJiZkWFtFE/explaining-british-naval-dominance-during-the-age-of-sail
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

“Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies” by So8res May 14, 2025

Show notes

Eliezer and I wrote a book. It's titled If Anyone Builds It, Everyone Dies. Unlike a lot of other writing either of us have done, it's being professionally published. It's hitting shelves on September 16th.
It's a concise (~60k word) book aimed at a broad audience. It's been well-received by people who received advance copies, with some endorsements including:
The most important book I've read for years: I want to bring it to every political and corporate leader in the world and stand over them until they've read it. Yudkowsky and Soares, who have studied AI and its possible trajectories for decades, sound a loud trumpet call to humanity to awaken us as we sleepwalk into disaster.
- Stephen Fry, actor, broadcaster, and writer
If Anyone Builds It, Everyone Dies may prove to be the most important book of our time. Yudkowsky and Soares believe [...]
The original text contained 1 footnote which was omitted from this narration.
---
First published:
May 14th, 2025
Source:
https://www.lesswrong.com/posts/iNsy7MsbodCyNTwKs/eliezer-and-i-wrote-a-book-if-anyone-builds-it-everyone-dies
---
Narrated by TYPE III AUDIO.

A heartwarming scene of reading time together on a vintage patterned couch. — **“Too Soon” by Gordon Seidoh Worley** May 14, 2025
Show notes

It was a cold and cloudy San Francisco Sunday. My wife and I were having lunch with friends at a Korean cafe.
My phone buzzed with a text. It said my mom was in the hospital.
I called to find out more. She had a fever, some pain, and had fainted. The situation was serious, but stable.
Monday was a normal day. No news was good news, right?
Tuesday she had seizures.
Wednesday she was in the ICU. I caught the first flight to Tampa.
Thursday she rested comfortably.
Friday she was diagnosed with bacterial meningitis, a rare condition that affects about 3,000 people in the US annually. The doctors had known it was a possibility, so she was already receiving treatment.
We stayed by her side through the weekend. My dad spent every night with her. We made plans for all the fun things we would when she [...]
---
First published:
May 13th, 2025
Source:
https://www.lesswrong.com/posts/reo79XwMKSZuBhKLv/too-soon
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

**“PSA: The LessWrong Feedback Service” by JustisMills** May 13, 2025
Show notes

At the bottom of the LessWrong post editor, if you have at least 100 global karma, you may have noticed this button.
The button Many people click the button, and are jumpscared when it starts an Intercom chat with a professional editor (me), asking what sort of feedback they'd like.
So, that's what it does. It's a summon Justis button.
**Why summon Justis?**
To get feedback on your post, of just about any sort. Typo fixes, grammar checks, sanity checks, clarity checks, fit for LessWrong, the works. If you use the LessWrong editor (as opposed to the Markdown editor) I can leave comments and suggestions directly inline. I also provide detailed narrative feedback (unless you explicitly don't want this) in the Intercom chat itself.
The feedback is totally without pressure. You can throw it all away, or just keep the bits you like. Or use it all!
In any case [...]
---
**Outline:**
(00:35) Why summon Justis?
(01:19) Why Justis in particular?
(01:48) Am I doing it right?
(01:59) How often can I request feedback?
(02:22) Can I use the feature for linkposts/crossposts?
(02:49) What if I click the button by mistake?
(02:59) Should I credit you?
(03:16) Couldnt I just use an LLM?
(03:48) Why does Justis do this?
---
First published:
May 12th, 2025
Source:
https://www.lesswrong.com/posts/bkDrfofLMKFoMGZkE/psa-the-lesswrong-feedback-service
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

“Orienting Toward Wizard Power” by johnswentworth May 08, 2025

Show notes

For months, I had the feeling: something is wrong. Some core part of myself had gone missing.
I had words and ideas cached, which pointed back to the missing part.
There was the story of Benjamin Jesty, a dairy farmer who vaccinated his family against smallpox in 1774 - 20 years before the vaccination technique was popularized, and the same year King Louis XV of France died of the disease.
There was another old post which declared “I don’t care that much about giant yachts. I want a cure for aging. I want weekend trips to the moon. I want flying cars and an indestructible body and tiny genetically-engineered dragons.”.
There was a cached instinct to look at certain kinds of social incentive gradient, toward managing more people or growing an organization or playing social-political games, and say “no, it's a trap”. To go… in a different direction, orthogonal [...]
---
Outline:
(01:19) In Search of a Name
(04:23) Near Mode
---
First published:
May 8th, 2025
Source:
https://www.lesswrong.com/posts/Wg6ptgi2DupFuAnXG/orienting-toward-wizard-power
---
Narrated by TYPE III AUDIO.

“Interpretability Will Not Reliably Find Deceptive AI” by Neel Nanda May 04, 2025

Show notes

(Disclaimer: Post written in a personal capacity. These are personal hot takes and do not in any way represent my employer's views.)
TL;DR: I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise. Interpretability seems a valuable tool here and remains worth investing in, as it will hopefully increase the reliability we can achieve. However, interpretability should be viewed as part of an overall portfolio of defences: a layer in a defence-in-depth strategy. It is not the one thing that will save us, and it still won’t be enough for high reliability.
Introduction
There's a common, often implicit, argument made in AI safety discussions: interpretability is presented as the only reliable path forward for detecting deception in advanced AI - among many other sources it was argued for in [...]
---
Outline:
(00:55) Introduction
(02:57) High Reliability Seems Unattainable
(05:12) Why Won't Interpretability be Reliable?
(07:47) The Potential of Black-Box Methods
(08:48) The Role of Interpretability
(12:02) Conclusion
The original text contained 5 footnotes which were omitted from this narration.
---
First published:
May 4th, 2025
Source:
https://www.lesswrong.com/posts/PwnadG4BFjaER3MGf/interpretability-will-not-reliably-find-deceptive-ai
---
Narrated by TYPE III AUDIO.

Our TOPPODCAST Picks

Follow Us

Stay Connected

LessWrong (Curated & Popular)

Related Podcasts

Links

Stay Connected