Show notes
AI models today have a 50% chance of successfully completing a task that would take an expert human one hour. Seven months ago, that number was roughly 30 minutes — and seven months before that, 15 minutes.
These are substantial, multi-step tasks requiring sustained focus: building web applications, conducting machine learning research, or solving complex programming challenges.
Today’s guest, Beth Barnes, is CEO of METR (Model Evaluation & Threat Research) — the leading organisation measuring these capabilities.
Beth’s team has been timing how long it takes skilled humans to complete projects of varying length, then seeing how AI models perform on the same work.
The resulting paper from METR, “Measuring AI ability to complete long tasks,” made waves by revealing that the planning horizon of AI models was doubling roughly every seven months. It’s regarded by many as the most useful AI forecasting work in years.
The companies building these systems aren’t just aware of this trend — they want to harness it as much as possible, and are aggressively pursuing automation of their own research.
That’s both an exciting and troubling development, because it could radically speed up advances in AI capabilities, accomplishing what would have taken years or decades in just months. That itself could be highly destabilising, as we explored in a previous episode: Will MacAskill on AI causing a “century in a decade” — and how we’re completely unprepared.
And having AI models rapidly build their successors with limited human oversight naturally raises the risk that things will go off the rails if the models at the end of the process lack the goals and constraints we hoped for.
Beth thinks models can already do “meaningful work” on improving themselves, and she wouldn’t be surprised if AI models were able to autonomously self-improve in as little as two years from now — in fact, she says, “It seems hard to rule out even shorter [timelines]. Is there 1% chance of this happening in six, nine months? Yeah, that seems pretty plausible.”
While Silicon Valley is abuzz with these numbers, policymakers remain largely unaware of what’s barrelling toward us — and given the current lack of regulation of AI companies, they’re not even able to access the critical information that would help them decide whether to intervene. Beth adds:
The sense I really want to dispel is, “But the experts must be on top of this. The experts would be telling us if it really was time to freak out.” The experts are not on top of this. Inasmuch as there are experts, they are saying that this is concerning. … And to the extent that I am an expert, I am an expert telling you you should freak out. And there’s not especially anyone else who isn’t saying this.Beth and Rob discuss all that, plus:
- How Beth now thinks that open-weight models are a good thing for AI safety, and what changed her mind
- How our poor information security means there’s no such thing as a “closed-weight” model anyway
- Whether we can see if an AI is scheming in its chain-of-thought reasoning, and the latest research on “alignment faking”
- Why just before deployment is the worst time to evaluate model safety
- Why Beth thinks AIs could end up being really good at creative and novel research — something humans tend to think is beyond their reach
- Why Beth thinks safety-focused people should stay out of the frontier AI companies — and the advantages smaller organisations have
- Areas of AI safety research that Beth thinks is overrated and underrated
- Whether it’s feasible to have a science that translates AI models’ increasing use of nonhuman language or ‘neuralese’
- How AI is both similar to and different from nuclear arms racing and bioweapons
- And much more besides!
Learn more and read the full transcript on the 80,000 Hours website.
What did you think of this episode? https://forms.gle/sFuDkoznxBcHPVmX6
Chapters:
• Cold open (00:00:00)
• Who’s Beth Barnes? (00:01:17)
• Can we see AI scheming in the chain of thought? (00:01:51)
• The chain of thought is essential for safety checking (00:09:16)
• Alignment faking in large language models (00:12:50)
• We have to test model honesty even before they're used inside AI companies (00:17:33)
• We have to test models when unruly & unconstrained (00:27:02)
• Each 7 months models can do tasks twice as long (00:31:56)
• METR's research finds AIs are solid at AI research already (00:51:31)
• AI may turn out to be strong at novel & creative research (00:58:18)
• When can we expect an algorithmic 'intelligence explosion'? (01:01:44)
• Recursively self-improving AI might even be here in 2 years — which is alarming (01:07:55)
• Could evaluations backfire by increasing AI hype & racing? (01:14:29)
• Governments first ignore new risks, but can overreact once they arrive (01:30:52)
• Do we need external auditors doing AI safety tests, not just the companies themselves? (01:39:55)
• A case against safety-focused people working at frontier AI companies (01:54:09)
• The new, more dire situation has forced changes to METR's strategy (02:08:40)
• AI companies are being locally reasonable, but globally reckless (02:16:55)
• Overrated: Interpretability research (02:21:49)
• Underrated: Developing more narrow AIs (02:23:44)
• Underrated: Helping humans judge confusing model outputs (02:30:28)
• Overrated: Major AI companies’ contributions to safety res...
Our TOPPODCAST Picks 
Stay Connected
