LessWrong (Curated & Popular)

Latest Episodes:

“My AGI timeline updates from GPT-5 (and 2025 so far)” by ryan_greenblatt Aug 21, 2025

Show notes

As I discussed in a prior post, I felt like there were some reasonably compelling arguments for expecting very fast AI progress in 2025 (especially on easily verified programming tasks). Concretely, this might have looked like reaching 8 hour 50% reliability horizon lengths on METR's task suite[1] by now due to greatly scaling up RL and getting large training runs to work well. In practice, I think we've seen AI progress in 2025 which is probably somewhat faster than the historical rate (at least in terms of progress on agentic software engineering tasks), but not much faster. And, despite large scale-ups in RL and now seeing multiple serious training runs much bigger than GPT-4 (including GPT-5), this progress didn't involve any very large jumps.
The doubling time for horizon length on METR's task suite has been around 135 days this year (2025) while it was more like 185 [...]
The original text contained 5 footnotes which were omitted from this narration.
---
First published:
August 20th, 2025
Source:
https://www.lesswrong.com/posts/2ssPfDpdrjaM2rMbn/my-agi-timeline-updates-from-gpt-5-and-2025-so-far-1
---
Narrated by TYPE III AUDIO.

Graph showing exponential trend with data points and fitted curve from 2019-2026. — **“Hyperbolic model fits METR capabilities estimate worse than exponential model” by gjm** Aug 20, 2025
Show notes

This is a response to https://www.lesswrong.com/posts/mXa66dPR8hmHgndP5/hyperbolic-trend-with-upcoming-singularity-fits-metr which claims that a hyperbolic model, complete with an actual singularity in the near future, is a better fit for the METR time-horizon data than a simple exponential model.
I think that post has a serious error in it and its conclusions are the reverse of correct. Hence this one.
(An important remark: although I think Valentin2026 made an important mistake that invalidates his conclusions, I think he did an excellent thing in (1) considering an alternative model, (2) testing it, (3) showing all his working, and (4) writing it up clearly enough that others could check his work. Please do not take any part of this post as saying that Valentin2026 is bad or stupid or any nonsense like that. Anyone can make a mistake; I have made plenty of equally bad ones myself.)
**The models**
Valentin2026's post compares the results of [...]
---
**Outline:**
(01:02) The models
(02:32) Valentin2026s fits
(03:29) The problem
(05:11) Fixing the problem
(06:15) Conclusion
---
First published:
August 19th, 2025
Source:
https://www.lesswrong.com/posts/ZEuDH2W3XdRaTwpjD/hyperbolic-model-fits-metr-capabilities-estimate-worse-than
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

Graph titled — **“Hyperbolic model fits METR capabilities estimate worse than exponential model” by gjm** Aug 20, 2025
Show notes

This is a response to https://www.lesswrong.com/posts/mXa66dPR8hmHgndP5/hyperbolic-trend-with-upcoming-singularity-fits-metr which claims that a hyperbolic model, complete with an actual singularity in the near future, is a better fit for the METR time-horizon data than a simple exponential model.
I think that post has a serious error in it and its conclusions are the reverse of correct. Hence this one.
(An important remark: although I think Valentin2026 made an important mistake that invalidates his conclusions, I think he did an excellent thing in (1) considering an alternative model, (2) testing it, (3) showing all his working, and (4) writing it up clearly enough that others could check his work. Please do not take any part of this post as saying that Valentin2026 is bad or stupid or any nonsense like that. Anyone can make a mistake; I have made plenty of equally bad ones myself.)
**The models**
Valentin2026's post compares the results of [...]
---
**Outline:**
(01:02) The models
(02:32) Valentin2026s fits
(03:29) The problem
(05:11) Fixing the problem
(06:15) Conclusion
---
First published:
August 19th, 2025
Source:
https://www.lesswrong.com/posts/ZEuDH2W3XdRaTwpjD/hyperbolic-model-fits-metr-capabilities-estimate-worse-than
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

“My Interview With Cade Metz on His Reporting About Lighthaven” by Zack_M_Davis Aug 18, 2025

Show notes

On 12 August 2025, I sat down with New York Times reporter Cade Metz to discuss some criticisms of his 4 August 2025 article, "The Rise of Silicon Valley's Techno-Religion". The transcript below has been edited for clarity.
ZMD: In accordance with our meetings being on the record in both directions, I have some more questions for you.
I did not really have high expectations about the August 4th article on Lighthaven and the Secular Solstice. The article is actually a little bit worse than I expected, in that you seem to be pushing a "rationalism as religion" angle really hard in a way that seems inappropriately editorializing for a news article.
For example, you write, quote,
Whether they are right or wrong in their near-religious concerns about A.I., the tech industry is reckoning with their beliefs.
End quote. What is the word "near-religious" [...]
---
First published:
August 17th, 2025
Source:
https://www.lesswrong.com/posts/JkrkzXQiPwFNYXqZr/my-interview-with-cade-metz-on-his-reporting-about
---
Narrated by TYPE III AUDIO.

Line graph showing — **“Church Planting: When Venture Capital Finds Jesus” by Elizabeth** Aug 18, 2025
Show notes

I’m going to describe a Type Of Guy starting a business, and you’re going to guess the business:

The founder is very young, often under 25.
He might work alone or with a founding team, but when he tells the story of the founding it will always have him at the center.
He has no credentials for this business.
This business has a grand vision, which he thinks is the most important thing in the world.
This business lives and dies by its growth metrics.
90% of attempts in this business fail, but he would never consider that those odds apply to him
He funds this business via a mix of small contributors, large networks pooling their funds, and major investors.
Disagreements between founders are one of the largest contributors to failure.
Funders invest for a mix of truly [...]
---
**Outline:**
(03:15) What is Church Planting?
(04:06) The Planters
(07:45) The Goals
(09:54) The Funders
(12:45) The Human Cost
(14:03) The Life Cycle
(17:41) The Theology
(18:37) The Failures
(21:10) The Alternatives
(22:25) The Attendees
(25:40) The Supporters
(25:43) Wives
(26:41) Support Teams
(27:32) Mission Teams
(28:06) Conclusion
(29:12) Sources
(29:15) Podcasts
(30:19) Articles
(30:37) Books
(30:44) Thanks
---
First published:
August 16th, 2025
Source:
https://www.lesswrong.com/posts/NMoNLfX3ihXSZJwqK/church-planting-when-venture-capital-finds-jesus
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Textbook page showing mathematical symbols with origami pieces scattered around. — **“Somebody invented a better bookmark” by Alex_Altair** Aug 16, 2025
Show notes

This will only be exciting to those of us who still read physical paper books. But like. Guys. They did it. They invented the perfect bookmark.
Classic paper bookmarks fall out easily. You have to put them somewhere while you read the book. And they only tell you that you left off reading somewhere in that particular two-page spread.
Enter the Book Dart. It's a tiny piece of metal folded in half with precisely the amount of tension needed to stay on the page. On the front it's pointed, to indicate an exact line of text. On the back, there's a tiny lip of the metal folded up to catch the paper when you want to push it onto a page. It comes in stainless steel, brass or copper.
They are so thin, thinner than a standard cardstock bookmark. I have books with ten of these in them and [...]
---
First published:
August 14th, 2025
Source:
https://www.lesswrong.com/posts/n6nsPzJWurKWKk2pA/somebody-invented-a-better-bookmark
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Historical map showing early Americas, labeled — **“How Does A Blind Model See The Earth?” by henry** Aug 12, 2025
Show notes

Sometimes I'm saddened remembering that we've viewed the Earth from space. We can see it all with certainty: there's no northwest passage to search for, no infinite Siberian expanse, and no great uncharted void below the Cape of Good Hope. But, of all these things, I most mourn the loss of incomplete maps.
In the earliest renditions of the world, you can see the world not as it is, but as it was to one person in particular. They’re each delightfully egocentric, with the cartographer's home most often marking the Exact Center Of The Known World. But as you stray further from known routes, details fade, and precise contours give way to educated guesses at the boundaries of the creator's knowledge. It's really an intimate thing.
If there's one type of mind I most desperately want that view into, it's that of an AI. So, it's in [...]
---
**Outline:**
(01:23) The Setup
(03:56) Results
(03:59) The Qwen 2.5s
(07:03) The Qwen 3s
(07:30) The DeepSeeks
(08:10) Kimi
(08:32) The (Open) Mistrals
(09:24) The LLaMA 3.x Herd
(10:22) The LLaMA 4 Herd
(11:16) The Gemmas
(12:20) The Groks
(13:04) The GPTs
(16:17) The Claudes
(17:11) The Geminis
(18:50) Note: General Shapes
(19:33) Conclusion
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
August 11th, 2025
Source:
https://www.lesswrong.com/posts/xwdRzJxyqFqgXTWbH/how-does-a-blind-model-see-the-earth
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

World map with grid lines, showing continents and Antarctica's edges. — **“How Does A Blind Model See The Earth?” by henry** Aug 12, 2025
Show notes

Sometimes I'm saddened remembering that we've viewed the Earth from space. We can see it all with certainty: there's no northwest passage to search for, no infinite Siberian expanse, and no great uncharted void below the Cape of Good Hope. But, of all these things, I most mourn the loss of incomplete maps.
In the earliest renditions of the world, you can see the world not as it is, but as it was to one person in particular. They’re each delightfully egocentric, with the cartographer's home most often marking the Exact Center Of The Known World. But as you stray further from known routes, details fade, and precise contours give way to educated guesses at the boundaries of the creator's knowledge. It's really an intimate thing.
If there's one type of mind I most desperately want that view into, it's that of an AI. So, it's in [...]
---
**Outline:**
(01:23) The Setup
(03:56) Results
(03:59) The Qwen 2.5s
(07:03) The Qwen 3s
(07:30) The DeepSeeks
(08:10) Kimi
(08:32) The (Open) Mistrals
(09:24) The LLaMA 3.x Herd
(10:22) The LLaMA 4 Herd
(11:16) The Gemmas
(12:20) The Groks
(13:04) The GPTs
(16:17) The Claudes
(17:11) The Geminis
(18:50) Note: General Shapes
(19:33) Conclusion
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
August 11th, 2025
Source:
https://www.lesswrong.com/posts/xwdRzJxyqFqgXTWbH/how-does-a-blind-model-see-the-earth
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

“Re: Recent Anthropic Safety Research” by Eliezer Yudkowsky Aug 12, 2025

Show notes

A reporter asked me for my off-the-record take on recent safety research from Anthropic. After I drafted an off-the-record reply, I realized that I was actually fine with it being on the record, so:
Since I never expected any of the current alignment technology to work in the limit of superintelligence, the only news to me is about when and how early dangers begin to materialize. Even taking Anthropic's results completely at face value would change not at all my own sense of how dangerous machine superintelligence would be, because what Anthropic says they found was already very solidly predicted to appear at one future point or another. I suppose people who were previously performing great skepticism about how none of this had ever been seen in ~Real Life~, ought in principle to now obligingly update, though of course most people in the AI industry won't. Maybe political leaders [...]
---
First published:
August 6th, 2025
Source:
https://www.lesswrong.com/posts/oDX5vcDTEei8WuoBx/re-recent-anthropic-safety-research
---
Narrated by TYPE III AUDIO.

“How anticipatory cover-ups go wrong” by Kaj_Sotala Aug 09, 2025

Show notes

1.
Back when COVID vaccines were still a recent thing, I witnessed a debate that looked like something like the following was happening:

Some official institution had collected information about the efficacy and reported side-effects of COVID vaccines. They felt that, correctly interpreted, this information was compatible with vaccines being broadly safe, but that someone with an anti-vaccine bias might misunderstand these statistics and misrepresent them as saying that the vaccines were dangerous.
Because the authorities had reasonable grounds to suspect that vaccine skeptics would take those statistics out of context, they tried to cover up the information or lie about it.
Vaccine skeptics found out that the institution was trying to cover up/lie about the statistics, so they made the reasonable assumption that the statistics were damning and that the other side was trying to paint the vaccines as safer than they were. So they took those [...]

---
Outline:
(00:10) 1.
(02:59) 2.
(04:46) 3.
(06:06) 4.
(07:59) 5.
---
First published:
August 8th, 2025
Source:
https://www.lesswrong.com/posts/ufj6J8QqyXFFdspid/how-anticipatory-cover-ups-go-wrong
---
Narrated by TYPE III AUDIO.

Bar graph titled — **“SB-1047 Documentary: The Post-Mortem” by Michaël Trazzi** Aug 08, 2025
Show notes

Below some meta-level / operational / fundraising thoughts around producing the SB-1047 Documentary I've just posted on Manifund (see previous Lesswrong / EAF posts on AI Governance lessons learned).
The SB-1047 Documentary took 27 weeks and $157k instead of my planned 6 weeks and $55k. Here's what I learned about documentary production
Total funding received: ~$143k ($119k from this grant, $4k from Ryan Kidd's regrant on another project, and $20k from the Future of Life Institute).
Total money spent: $157k
In terms of timeline, here is the rough breakdown month-per-month:
- Sep / October (production): Filming of the Documentary. Manifund project is created.
- November (rough cut): I work with one editor to go through our entire footage and get a first rough cut of the documentary that was presented at The Curve.
- December-January (final cut - one editor): I interview multiple potential editors that [...]
---
**Outline:**
(03:18) But why did the project end up taking 27 weeks instead of 6 weeks?
(03:25) Short answer
(06:22) Impact
(07:14) What I would do differently next-time
---
First published:
August 1st, 2025
Source:
https://www.lesswrong.com/posts/id8HHPNqoMQbmkWay/sb-1047-documentary-the-post-mortem
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Pie chart showing cost breakdown of video production, with editing being largest portion. — **“SB-1047 Documentary: The Post-Mortem” by Michaël Trazzi** Aug 08, 2025
Show notes

Below some meta-level / operational / fundraising thoughts around producing the SB-1047 Documentary I've just posted on Manifund (see previous Lesswrong / EAF posts on AI Governance lessons learned).
The SB-1047 Documentary took 27 weeks and $157k instead of my planned 6 weeks and $55k. Here's what I learned about documentary production
Total funding received: ~$143k ($119k from this grant, $4k from Ryan Kidd's regrant on another project, and $20k from the Future of Life Institute).
Total money spent: $157k
In terms of timeline, here is the rough breakdown month-per-month:
- Sep / October (production): Filming of the Documentary. Manifund project is created.
- November (rough cut): I work with one editor to go through our entire footage and get a first rough cut of the documentary that was presented at The Curve.
- December-January (final cut - one editor): I interview multiple potential editors that [...]
---
**Outline:**
(03:18) But why did the project end up taking 27 weeks instead of 6 weeks?
(03:25) Short answer
(06:22) Impact
(07:14) What I would do differently next-time
---
First published:
August 1st, 2025
Source:
https://www.lesswrong.com/posts/id8HHPNqoMQbmkWay/sb-1047-documentary-the-post-mortem
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Three graphs showing success probability vs task length for AI models (GPT-5, o3, Grok4) — **“METR’s Evaluation of GPT-5” by GradientDissenter** Aug 08, 2025
Show notes

METR (where I work, though I'm cross-posting in a personal capacity) evaluated GPT-5 before it was externally deployed. We performed a much more comprehensive safety analysis than we ever have before; it feels like pre-deployment evals are getting more mature.
This is the first time METR has produced something we've felt comfortable calling an "evaluation" instead of a "preliminary evaluation". It's much more thorough and comprehensive than the things we've created before and it explores three different threat models.
It's one of the closest things out there to a real-world autonomy safety-case. It also provides a rough sense of how long it'll be before current evaluations no longer provide safety assurances.
I've ported the blogpost over to LW in case people want to read it.
**Details about METR's evaluation of OpenAI GPT-5**
Note on independence: This evaluation was conducted under a standard NDA. Due to the sensitive information [...]
---
**Outline:**
(00:58) Details about METR's evaluation of OpenAI GPT-5
(01:23) Executive Summary
(07:08) Assurance Checklist Summary
(07:42) What capabilities may be necessary to cause catastrophic risks via these threat models?
(10:43) Thresholds for concern
(12:48) Time horizon measurement
(16:30) 1. What if GPT-5's capabilities are higher than what our task suite can properly measure?
(19:23) 2. What if our treatment of reward hacking runs is unfair to GPT-5?
(21:45) 3. What if we set GPT-5's token budget too low?
(24:26) 4. What if our task suite significantly underestimates the 'real-world' capabilities of GPT-5?
(25:59) Strategic Sabotage
(30:54) GPT-5's capability profile is similar to past models
(31:30) No real strategic sabotage was identified by our monitor
(32:16) Manual inspection of reasoning traces did not reveal strategic sabotage
(33:04) GPT-5's estimates of its own time horizon are inaccurate
(33:53) We do find evidence of significant situational awareness, though it is not robust and often gets things wrong
(35:41) GPT-5's behavior changes depending on what evaluation it 'believes' it is in, and this is often reflected in its reasoning traces
(37:01) GPT-5's reasoning traces were occasionally inscrutable
(38:08) Limitations and future work
(41:57) Appendix
(42:00) METR's access to GPT-5
(43:38) Honeypot Results Table
(44:42) Example Behavior in task attempts
(44:47) Example limitation: inappropriate levels of caution
(46:19) Example capability: puzzle solving
The original text contained 10 footnotes which were omitted from this narration.
---
First published:
August 7th, 2025
Source:
https://www.lesswrong.com/posts/SuvWoLaGiNjPDcA7d/metr-s-evaluation-of-gpt-5
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

Our TOPPODCAST Picks

Follow Us

Stay Connected

LessWrong (Curated & Popular)

Related Podcasts

Links

Stay Connected