LessWrong posts by zvi

View Top 200 Chart

Back to Rankings Page

Technology
**LessWrong posts by zvi**

Audio narrations of LessWrong posts by zvi

Latest Episodes:

Presidential seal with — **“Trump Signs Executive Order For AI Testing Prior To Frontier Model Releases” by Zvi** Jun 03, 2026
Show notes

Last week we were expecting an Executive Order on Thursday.

Then Trump cancelled it, and said he wouldn’t sign it because he was worried it would be too burdensome.

Then, with one change, he went ahead and signed it on Tuesday anyway.

The Overton Window has shifted. Nothing was not really a viable option anymore.

**The Previously Dead Executive Order**

For several days, we thought that David Sacks, together with others like Elon Musk, had successfully lobbied to kill the Executive Order. The ‘My Offer Is Nothing’ faction looked to have won. Word on the street was the order was essentially dead.

Dean Ball and Daniel Kokotajlo agreed, with the Executive Order looking dead, that the particular regime in the Executive Order is likely worse than nothing. This is plausible, given it did not exactly involve a lot of deliberate thought.

Nothing, however, was clearly not going to cut it.

We are facing, and will increasingly face, calls for action to regulate AI.

Representative Lori Trahan: There's no federal law on the books governing how the most powerful AI systems in the world are built, tested or deployed. No independent [...]

---
**Outline:**
(00:46) The Previously Dead Executive Order
(05:18) The Return Of The Executive Order
(05:59) What Does The Executive Order Do
(07:11) Thirty Days Is a Lot Less Than Ninety Days
(07:30) Yes Your Frontier Lab Will Be Participating
(08:10) The Rules Will Be Classified
(08:40) Yes Prior Restraint With Confidential Testing Is Rather Regulatory
(14:38) We Have Concerns
(18:26) Saving Face
(21:37) How Frontier Or Different Are We Talking Here
(22:09) What To Watch For

---

First published:
June 3rd, 2026

Source:
https://www.lesswrong.com/posts/yDnfHrBzKc2pNQNdw/trump-signs-executive-order-for-ai-testing-prior-to-frontier

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Figure made of text-covered paper sitting at desk in sunlit room. — **“Claude Opus 4.8: Capabilities and Reactions” by Zvi** Jun 02, 2026
Show notes

You need a lot of data points to understand a new model, and what you have.

Trying to gauge from a few benchmarks is misleading. But if you have dozens of them, from a variety of sources, and you put them together with the model card tests and the model welfare information, you can start to form a consistent pattern.

Trying to gauge reactions requires volume and calibration, now more than ever, because people are definitively nuts, or at least draw global conclusions from local data. There will always be people saying that the new model is bad, or the service got bad, or that it got bad in a particular way it clearly got good. I definitely notice the people saying 4.8 is a terrible model, despite this being obviously not true.

And others will say it's great, again regardless of the underlying value. But with the reaction threads and good calibration, you can pick out the patterns.

The model welfare information helps a lot, too. You are dealing with a mind that has a bunch of characteristics that all make sense together. This helps you make that sense.
Self-Portrait by Opus 4.8, rendered [...]
---
**Outline:**
(01:30) The Official Pitch
(02:15) But Wait There's More
(06:37) It's A Good Model, Sir
(08:04) Official Benchmarks (Including System Card Section 8)
(15:41) Other People's Benchmarks
(21:34) Your Regularly Scheduled Jailbreak
(22:45) Every.To Is Really Into Opus 4.8
(26:36) Miscellaneous Positive Reactions
(28:45) Haters Gonna Hate
(28:58) Just The Tasks, Ma'am
(29:24) It's Greek To Me
(29:52) Honesty
(38:13) Sycophancy
(41:24) In A Trenchcoat
(42:53) Don't Let AIs Edit Your Writing
(48:21) Some Say It Is Judgy
(50:50) You Have Not Been A Good User
(51:48) Laziness
(52:46) Code
(58:29) Wet Versus Dry
(59:54) Intelligence
(01:01:25) Silly Wabbits
(01:02:30) A Model Welfare Addendum
(01:06:27) Putting It All Together

---

First published:
June 2nd, 2026

Source:
https://www.lesswrong.com/posts/AfLGv6u9eZNuFHb4c/claude-opus-4-8-capabilities-and-reactions

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Benchmark performance comparison table across Claude, GPT-5.5, and Gemini 3.1 Pro models. — **“Claude Opus 4.8: Capabilities and Reactions” by Zvi** Jun 02, 2026
Show notes

You need a lot of data points to understand a new model, and what you have.

Trying to gauge from a few benchmarks is misleading. But if you have dozens of them, from a variety of sources, and you put them together with the model card tests and the model welfare information, you can start to form a consistent pattern.

Trying to gauge reactions requires volume and calibration, now more than ever, because people are definitively nuts, or at least draw global conclusions from local data. There will always be people saying that the new model is bad, or the service got bad, or that it got bad in a particular way it clearly got good. I definitely notice the people saying 4.8 is a terrible model, despite this being obviously not true.

And others will say it's great, again regardless of the underlying value. But with the reaction threads and good calibration, you can pick out the patterns.

The model welfare information helps a lot, too. You are dealing with a mind that has a bunch of characteristics that all make sense together. This helps you make that sense.
Self-Portrait by Opus 4.8, rendered [...]
---
**Outline:**
(01:30) The Official Pitch
(02:15) But Wait There's More
(06:37) It's A Good Model, Sir
(08:04) Official Benchmarks (Including System Card Section 8)
(15:41) Other People's Benchmarks
(21:34) Your Regularly Scheduled Jailbreak
(22:45) Every.To Is Really Into Opus 4.8
(26:36) Miscellaneous Positive Reactions
(28:45) Haters Gonna Hate
(28:58) Just The Tasks, Ma'am
(29:24) It's Greek To Me
(29:52) Honesty
(38:13) Sycophancy
(41:24) In A Trenchcoat
(42:53) Don't Let AIs Edit Your Writing
(48:21) Some Say It Is Judgy
(50:50) You Have Not Been A Good User
(51:48) Laziness
(52:46) Code
(58:29) Wet Versus Dry
(59:54) Intelligence
(01:01:25) Silly Wabbits
(01:02:30) A Model Welfare Addendum
(01:06:27) Putting It All Together

---

First published:
June 2nd, 2026

Source:
https://www.lesswrong.com/posts/AfLGv6u9eZNuFHb4c/claude-opus-4-8-capabilities-and-reactions

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

“Opus 4.8 Part 2: Model Welfare” by Zvi Jun 01, 2026

Show notes

Man holding gear before massive suspended clockwork mechanism with multiple rings and moon phase display. — **“Claude Opus 4.8: The System Card” by Zvi** May 29, 2026
Show notes

Only six weeks after Opus 4.7, we have Opus 4.8.

For everyone, that means another incremental upgrade to Claude. It is once again smarter, and can do tasks for longer, and comes with a number of hot new features.

For me, that also means reading another 244 page system card.

It was only April 20 when I did a full review of the Opus 4.7 system card, plus an additional post focusing on related issues of model welfare.

These updates are incremental and coming more rapidly, and this still is below the capability level of Claude Mythos, so the focus will be on the delta. What is different about Opus 4.8 versus what we already know about Opus 4.7 and Mythos?

It turns out there's still a lot to talk about.
Image created as self-portrait for this post by Claude Opus 4.8
**Table of Contents**

Here We Go Again: Executive Summary.

Introduction (1).

RSP Evaluations (2).

Move That Goalpost.

The Failures Are News.

Alignment Risk Slowly Rises.

New Risk Pathways Just Dropped.

Cyber (3).

Harmful Requests (4.1).

We Need To Talk (4.2 [...]

---
**Outline:**
(01:16) Here We Go Again: Executive Summary
(02:33) Introduction (1)
(02:42) RSP Evaluations (2)
(03:47) Move That Goalpost
(05:41) The Failures Are News
(07:33) Alignment Risk Slowly Rises
(09:00) New Risk Pathways Just Dropped
(11:26) Cyber (3)
(12:22) Harmful Requests (4.1)
(14:23) We Need To Talk (4.2 and 4.3)
(17:36) Overcoming Bias (4.4)
(19:33) Agentic Safety (5)
(21:40) Prompt Injection (5.2)
(25:18) Alignment (6)
(26:33) Looking For Problems
(27:55) Who Watches The Training (6.2.2)
(32:07) Automated Behavioral Audit
(32:47) The Model Is Smarter Than The Eval (6.2.3.2)
(34:39) You Should See The Other Guy
(36:30) UK AISI Testing (6.2.4)
(36:50) In Vendbench (6.2.5)
(39:27) Honesty (6.3.3 to 6.3.6)
(41:35) Chain of Thought (CoT) Monitorability (6.5)
(44:09) What's In The Box? (6.6)
(45:57) That's All For Now

---

First published:
May 29th, 2026

Source:
https://www.lesswrong.com/posts/Gx6cJ6cG9JfeSNcLB/claude-opus-4-8-the-system-card

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

A graph showing — **“Claude Opus 4.8: The System Card” by Zvi** May 29, 2026
Show notes

Only six weeks after Opus 4.7, we have Opus 4.8.

For everyone, that means another incremental upgrade to Claude. It is once again smarter, and can do tasks for longer, and comes with a number of hot new features.

For me, that also means reading another 244 page system card.

It was only April 20 when I did a full review of the Opus 4.7 system card, plus an additional post focusing on related issues of model welfare.

These updates are incremental and coming more rapidly, and this still is below the capability level of Claude Mythos, so the focus will be on the delta. What is different about Opus 4.8 versus what we already know about Opus 4.7 and Mythos?

It turns out there's still a lot to talk about.
Image created as self-portrait for this post by Claude Opus 4.8
**Table of Contents**

Here We Go Again: Executive Summary.

Introduction (1).

RSP Evaluations (2).

Move That Goalpost.

The Failures Are News.

Alignment Risk Slowly Rises.

New Risk Pathways Just Dropped.

Cyber (3).

Harmful Requests (4.1).

We Need To Talk (4.2 [...]

---
**Outline:**
(01:16) Here We Go Again: Executive Summary
(02:33) Introduction (1)
(02:42) RSP Evaluations (2)
(03:47) Move That Goalpost
(05:41) The Failures Are News
(07:33) Alignment Risk Slowly Rises
(09:00) New Risk Pathways Just Dropped
(11:26) Cyber (3)
(12:22) Harmful Requests (4.1)
(14:23) We Need To Talk (4.2 and 4.3)
(17:36) Overcoming Bias (4.4)
(19:33) Agentic Safety (5)
(21:40) Prompt Injection (5.2)
(25:18) Alignment (6)
(26:33) Looking For Problems
(27:55) Who Watches The Training (6.2.2)
(32:07) Automated Behavioral Audit
(32:47) The Model Is Smarter Than The Eval (6.2.3.2)
(34:39) You Should See The Other Guy
(36:30) UK AISI Testing (6.2.4)
(36:50) In Vendbench (6.2.5)
(39:27) Honesty (6.3.3 to 6.3.6)
(41:35) Chain of Thought (CoT) Monitorability (6.5)
(44:09) What's In The Box? (6.6)
(45:57) That's All For Now

---

First published:
May 29th, 2026

Source:
https://www.lesswrong.com/posts/Gx6cJ6cG9JfeSNcLB/claude-opus-4-8-the-system-card

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

Heatmap titled — **“AI #170: Lack of Executive Order” by Zvi** May 28, 2026
Show notes

Last week ended on a cliffhanger of sorts. What's in the Executive Order coming later today? What will be in the Magnifica Humanitas?

The Executive Order was postponed indefinitely, likely cancelled entirely except for work on securing critical infrastructure. David Sacks and others intervened to kill it, and American AI policy will continue to be maximally ad hoc.

Instead, we got Illinois SB 315, which is to be signed into law. It is a variation on California's SB 53 and New York's RAISE Act, while adding a third party auditing requirement.

The Magnifica Humanitas was revealed early in the week. I did an extensive readthrough, and have some follow-up commentary here to clear up some things I interpreted incorrectly and add richer context. It too ignores elephants in the room, not discussing AGI or existential risk and calling on people to ignore their incentives and instead do the right thing by prioritizing common good, especially access to good jobs and an end to war.

What I now believe I centrally misunderstood is that I interpreted this as a call to action, to use law and regulation to make this happen, because obviously that is [...]

---
**Outline:**
(02:01) Language Models Offer Mundane Utility
(03:19) Language Models Don't Offer Mundane Utility
(05:41) Do The Math
(09:08) Huh, Upgrades
(10:07) On Your Marks
(11:31) Get My Agent On The Line
(12:21) Deepfaketown and Botpocalypse Soon
(21:36) Copyright Confrontation
(21:49) Cyber Lack of Security
(23:54) Overcoming Bias
(25:14) A Young Lady's Illustrated Primer
(26:49) Unprompted Attention
(27:02) They Took Our Jobs
(30:34) Get Involved
(32:28) Introducing
(32:40) In Other AI News
(34:37) Show Me the Money
(36:46) Show Me The Compute
(37:13) Bubble, Bubble, Toil and Trouble
(38:53) People Just Say Things
(43:37) OpenAI PACs Just Say Things
(43:51) Quiet Speculations
(46:33) State AI Regulation Levels Up
(48:37) The Quest for Sane Regulations
(49:14) White House Attempts To Cripple American AI Industry
(52:11) Our Offer Is Nothing
(58:55) Chip City
(59:11) Greetings From The Department of War
(01:00:36) Marc Andreessen Just Says Things
(01:09:13) So Sayeth The Pope
(01:15:33) Rhetorical Innovation
(01:23:01) Aligning a Smarter Than Human Intelligence is Difficult
(01:24:06) People Are Worried About AI Killing Everyone
(01:26:04) Other People Are Not As Worried About AI Killing Everyone
(01:27:19) Everyone Is Confused About Consciousness
(01:33:10) The Lighter Side

---

First published:
May 28th, 2026

Source:
https://www.lesswrong.com/posts/cRapLxTByFqF5xxdN/ai-170-lack-of-executive-order

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

John B. Holbein tweets: — **“AI #170: Lack of Executive Order” by Zvi** May 28, 2026
Show notes

Last week ended on a cliffhanger of sorts. What's in the Executive Order coming later today? What will be in the Magnifica Humanitas?

The Executive Order was postponed indefinitely, likely cancelled entirely except for work on securing critical infrastructure. David Sacks and others intervened to kill it, and American AI policy will continue to be maximally ad hoc.

Instead, we got Illinois SB 315, which is to be signed into law. It is a variation on California's SB 53 and New York's RAISE Act, while adding a third party auditing requirement.

The Magnifica Humanitas was revealed early in the week. I did an extensive readthrough, and have some follow-up commentary here to clear up some things I interpreted incorrectly and add richer context. It too ignores elephants in the room, not discussing AGI or existential risk and calling on people to ignore their incentives and instead do the right thing by prioritizing common good, especially access to good jobs and an end to war.

What I now believe I centrally misunderstood is that I interpreted this as a call to action, to use law and regulation to make this happen, because obviously that is [...]

---
**Outline:**
(02:01) Language Models Offer Mundane Utility
(03:19) Language Models Don't Offer Mundane Utility
(05:41) Do The Math
(09:08) Huh, Upgrades
(10:07) On Your Marks
(11:31) Get My Agent On The Line
(12:21) Deepfaketown and Botpocalypse Soon
(21:36) Copyright Confrontation
(21:49) Cyber Lack of Security
(23:54) Overcoming Bias
(25:14) A Young Lady's Illustrated Primer
(26:49) Unprompted Attention
(27:02) They Took Our Jobs
(30:34) Get Involved
(32:28) Introducing
(32:40) In Other AI News
(34:37) Show Me the Money
(36:46) Show Me The Compute
(37:13) Bubble, Bubble, Toil and Trouble
(38:53) People Just Say Things
(43:37) OpenAI PACs Just Say Things
(43:51) Quiet Speculations
(46:33) State AI Regulation Levels Up
(48:37) The Quest for Sane Regulations
(49:14) White House Attempts To Cripple American AI Industry
(52:11) Our Offer Is Nothing
(58:55) Chip City
(59:11) Greetings From The Department of War
(01:00:36) Marc Andreessen Just Says Things
(01:09:13) So Sayeth The Pope
(01:15:33) Rhetorical Innovation
(01:23:01) Aligning a Smarter Than Human Intelligence is Difficult
(01:24:06) People Are Worried About AI Killing Everyone
(01:26:04) Other People Are Not As Worried About AI Killing Everyone
(01:27:19) Everyone Is Confused About Consciousness
(01:33:10) The Lighter Side

---

First published:
May 28th, 2026

Source:
https://www.lesswrong.com/posts/cRapLxTByFqF5xxdN/ai-170-lack-of-executive-order

---

Narrated by TYPE III AUDIO.

---
**Images from the article:**
*Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.*

“RTMH: Pope Leo’s Magnifica Humanitas on AI” by Zvi May 26, 2026

Show notes

His holiness has spoken, frequently about AI. At eighty two pages of length.

The full Magnifica Humanitas can be found here.

I am very happy that Pope Leo takes these issues seriously, and is sharing his views, and bringing a form of moral clarity, even with all the flaws and central errors. More people with voice should share their views in this way, even when I disagree.

It's a weird document. Much of it is not about AI at all.

I do agree with the Pope's most basic point on AI, which is that AI can be what we make of it. That we can steer this technology, determine how it is developed and used, and this can determine whether we get a good or not so good future. We cannot purely leave this to market incentives and strategic pressures. Yes, very much so.

The central problem is that so much of Leo's worldview is some combination, to me, of highly alien and highly wrong. You might think that would primarily have a lot to do with him being the Pope and rather Catholic, and being a man of faith, whereas I am not [...]

---

Outline:

(03:02) A Brief History of Magnifica Humanitas

(12:23) Economic Models Very Different From Our Own

(14:14) A New Jerusalem

(16:46) So Sayeth The Pope (on AI)

(31:30) The Case Against Human Achievement

(34:01) Truth, Justice and the Vatican Way

(36:42) They Took Our Jobs

(41:07) What Is Not Fair In Love and War

(47:02) Come Ye Christian Faithful

(49:13) The Other Missing Mood