LessWrong (Curated & Popular)

Latest Episodes:

“The Information: OpenAI shows ‘Strawberry’ to feds, races to launch it ” by Martín Soto Aug 29, 2024

Show notes

Two new The Information articles with insider information on OpenAI's next models and moves.
They are paywalled, but here are the new bits of information:

Strawberry is more expensive and slow at inference time, but can solve complex problems on the first try without hallucinations. It seems to be an application or extension of process supervision
Its main purpose is to produce synthetic data for Orion, their next big LLM
But now they are also pushing to get a distillation of Strawberry into ChatGPT as soon as this fall
They showed it to feds

Some excerpts about these:
Plus this summer, his team demonstrated the technology [Strawberry] to American national security officials, said a person with direct knowledge of those meetings, which haven't previously been reported.
One of the most important applications of Strawberry is to generate high-quality training data for Orion, OpenAI's next flagship large [...]
---
First published:
August 27th, 2024
Source:
https://www.lesswrong.com/posts/8oX4FTRa8MJodArhj/the-information-openai-shows-strawberry-to-feds-races-to
---
Narrated by TYPE III AUDIO.

undefined — **“What is it to solve the alignment problem? ” by Joe Carlsmith** Aug 28, 2024
Show notes

People often talk about “solving the alignment problem.” But what is it to do such a thing? I wanted to clarify my thinking about this topic, so I wrote up some notes.
In brief, I’ll say that you’ve solved the alignment problem if you’ve:

avoided a bad form of AI takeover,
built the dangerous kind of superintelligent AI agents,
gained access to the main benefits of superintelligence, and
become able to elicit some significant portion of those benefits from some of the superintelligent AI agents at stake in (2).[1]
The post also discusses what it would take to do this. In particular:

I discuss various options for avoiding bad takeover, notably:

Avoiding what I call “vulnerability to alignment” conditions;
Ensuring that AIs don’t try to take over;
Preventing such attempts from succeeding;
Trying to ensure that AI takeover is somehow OK. (The alignment [...]
---
**Outline:**
(03:46) 1. Avoiding vs. handling vs. solving the problem
(15:32) 2. A framework for thinking about AI safety goals
(19:33) 3. Avoiding bad takeover
(24:03) 3.1 Avoiding vulnerability-to-alignment conditions
(27:18) 3.2 Ensuring that AI systems don’t try to takeover
(32:02) 3.3 Ensuring that takeover efforts don’t succeed
(33:07) 3.4 Ensuring that the takeover in question is somehow OK
(41:55) 3.5 What's the role of “corrigibility” here?
(42:17) 3.5.1 Some definitions of corrigibility
(50:10) 3.5.2 Is corrigibility necessary for “solving alignment”?
(53:34) 3.5.3 Does ensuring corrigibility raise issues that avoiding takeover does not?
(55:46) 4. Desired elicitation
(01:05:17) 5. The role of verification
(01:09:24) 5.1 Output-focused verification and process-focused verification
(01:16:14) 5.2 Does output-focused verification unlock desired elicitation?
(01:23:00) 5.3 What are our options for process-focused verification?
(01:29:25) 6. Does solving the alignment problem require some very sophisticated philosophical achievement re: our values on reflection?
(01:38:05) 7. Wrapping up
The original text contained 27 footnotes which were omitted from this narration.
The original text contained 3 images which were described by AI.
---
First published:
August 24th, 2024
Source:
https://www.lesswrong.com/posts/AFdvSBNgN2EkAsZZA/what-is-it-to-solve-the-alignment-problem-1
---
Narrated by TYPE III AUDIO.
---
**Images from the article:**

“Would catching your AIs trying to escape convince AI developers to slow down or undeploy? ” by Buck Aug 26, 2024

Show notes

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.I often talk to people who think that if frontier models were egregiously misaligned and powerful enough to pose an existential threat, you could get AI developers to slow down or undeploy models by producing evidence of their misalignment. I'm not so sure. As an extreme thought experiment, I’ll argue this could be hard even if you caught your AI red-handed trying to escape.
Imagine you're running an AI lab at the point where your AIs are able to automate almost all intellectual labor; the AIs are now mostly being deployed internally to do AI R&D. (If you want a concrete picture here, I'm imagining that there are 10 million parallel instances, running at 10x human speed, working 24/7. See e.g. similar calculations here). And suppose (as I think is 35% likely) that these models are egregiously [...]
---
First published:
August 26th, 2024
Source:
https://www.lesswrong.com/posts/YTZAmJKydD5hdRSeG/would-catching-your-ais-trying-to-escape-convince-ai
---
Narrated by TYPE III AUDIO.

“Liability regimes for AI ” by Ege Erdil Aug 23, 2024

Show notes

For many products, we face a choice of who to hold liable for harms that would not have occurred if not for the existence of the product. For instance, if a person uses a gun in a school shooting that kills a dozen people, there are many legal persons who in principle could be held liable for the harm:

The shooter themselves, for obvious reasons.
The shop that sold the shooter the weapon.
The company that designs and manufactures the weapon.

Which one of these is the best? I'll offer a brief and elementary economic analysis of how this decision should be made in this post.
The important concepts from economic theory to understand here are Coasean bargaining and the problem of the judgment-proof defendant.
Coasean bargaining
Let's start with Coaesean bargaining: in short, this idea says that regardless of [...]
---
Outline:
(00:49) Coasean bargaining
(02:09) The judgment-proof defendant
(04:20) Transaction costs and economies of scale
(05:23) Summary and implications for AI
---
First published:
August 19th, 2024
Source:
https://www.lesswrong.com/posts/vQF4Jspzi7ZjpnJbv/liability-regimes-for-ai
---
Narrated by TYPE III AUDIO.

“AGI Safety and Alignment at Google DeepMind:A Summary of Recent Work ” by Rohin Shah, Seb Farquhar, Anca Dragan Aug 21, 2024

Show notes

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours.
Who are we?
We’re the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our last post, we’ve evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the Frontier Safety Framework, including developing and running dangerous capability evaluations). We’ve also been growing since our last post: by 39% last year [...]
---
Outline:
(00:32) Who are we?
(01:32) What have we been up to?
(02:16) Frontier Safety
(02:38) FSF
(04:05) Dangerous Capability Evaluations
(05:12) Mechanistic Interpretability
(08:54) Amplified Oversight
(09:23) Theoretical Work on Debate
(10:32) Empirical Work on Debate
(11:37) Causal Alignment
(12:47) Emerging Topics
(14:57) Highlights from Our Collaborations
(17:07) What are we planning next?
---
First published:
August 20th, 2024
Source:
https://www.lesswrong.com/posts/79BPxvSsjzBkiSyTq/agi-safety-and-alignment-at-google-deepmind-a-summary-of
---
Narrated by TYPE III AUDIO.

“Leaving MIRI, Seeking Funding” by abramdemski Aug 08, 2024

Show notes

This is slightly old news at this point, but: as part of MIRI's recent strategy pivot, they've eliminated the Agent Foundations research team. I've been out of a job for a little over a month now. Much of my research time in the first half of the year was eaten up by engaging with the decision process that resulted in this, and later, applying to grants and looking for jobs.
I haven't secured funding yet, but for my own sanity & happiness, I am (mostly) taking a break from worrying about that, and getting back to thinking about the most important things.
However, in an effort to try the obvious, I have set up a Patreon where you can fund my work directly. I don't expect it to become my main source of income, but if it does, that could be a pretty good scenario for me; it would [...]
---
Outline:
(01:00) What Im (probably) Doing Going Forward
(02:28) Thoughts on Public vs Private Research
The original text contained 1 footnote which was omitted from this narration.
---
First published:
August 8th, 2024
Source:
https://www.lesswrong.com/posts/SnaAYrqkb7fzpZn86/leaving-miri-seeking-funding
---
Narrated by TYPE III AUDIO.

Our TOPPODCAST Picks

Follow Us

Stay Connected

LessWrong (Curated & Popular)

Related Podcasts

Links

Stay Connected