The Attention Mechanism with Andrew Mayne

Latest Episodes:

Artisanal Mathematics Oct 13, 2026

Show notes

Andrew and Justin examine Meta’s bid for frontier status, mathematicians’ divided response to OpenAI’s wave of new proofs, Anthropic’s growing momentum with Claude, and how Google’s AI push has obscured the search experience that once defined its brand.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro

00:54 - Meta

29:11 - Math

44:24 - State of the Race

59:12 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

Make That 19 New Releases Sep 29, 2026

Show notes

Andrew and Justin discuss their wishlist for OpenAI's upcoming Dev Day event, the delay of GPT-6.1, Meta's success with Muse, and more.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro and Dev Day

00:37 - GPT-6.1 Delayed

14:27- $500 Plans?

21:54 - Sora and Video Generation

28:32 - Open Source Potential

30:28 - Dev Day Wishlist

35:51 - Meta and Muse

44:48 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

An Incalculable Number of Spam Calls Sep 22, 2026

Show notes

Andrew and Justin discuss the swirling rumors surrounding OpenAI's launch plans for this week ahead of Dev Day, a new committee designed to reveal Millennium Prize math problems being solved by ChatGPT, whether an IPO is coming in 2026, and whatever the heck Jev is.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00:00 - Intro

00:00:58 - What's Jev?

00:03:24 - OpenAI Pre-Dev Day Rumors

00:23:09 - Millennium Prize Math Committee

00:38:44 - Are Hacking Incidents Overstated?

00:54:07 - IPO

00:59:58 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

Slowed for Safety Sep 15, 2026

Show notes

Andrew and Justin react to the industry calls to slow model development in the name of safety, the launch of Astra 6, and more.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro

01:32- Andrew's Reaction to Calls for Slowed Development

11:09 - Reacting to Various Online Takes

34:21 - Astra 6 is Out

41:54 - Navier–Stokes

49:55 - Anthropic's Risk Report

58:35 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

The Hugging Face Report Sep 01, 2026

Show notes

Andrew and Justin discuss the follow-up reporting and investigations surrounding the Hugging Face hack, Nvidia's acquisition of Hugging Face, and the looming promise of some fresh models to kick off the fall.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro

01:30 - Hugging Face Report

21:08 - Hugging Face Acquired

26:24 - Cursor and OpenAI

31:08 - Fable 5.1 and Astra

51:22 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

The Problem with Text Watermarks Aug 18, 2026

Show notes

Andrew and Justin discuss Anthropic's text-based AI watermarks, the future of private companies in an AI world, and whether Google can keep up with the rest of its frontier rivals.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00:00 - Intro

00:00:55 - Watermarks

00:20:45 - Dario's Private Company Thoughts

00:36:58 - Always-Listening Assistants and Google's Place in the Frontier Race

00:54:34 - RIP Atlas, One Last Time

00:58:25 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

The End of the DeepMind Era? Aug 11, 2026

Show notes

Andrew and Justin reconvene to talk about an ongoing series of recent upheavals at Google, asking whether the company's lack of focus on AI puts it at a disadvantage over its frontier rivals. They also discuss Sam Altman's trip to Washington and the discourse surrounding model hacking incidents.

The Attention Mechanism is now available in video! Check it out here.

Editor's note: Some minor crosstalk issues in this episode due to recording sync delays.

Chapters

00:00 - Intro

01:04 - Google Shakeup

21:13 - Mr. Altman Goes to Washington

37:24 - Dean Ball and Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

A Good Guy with a Frontier Model Jul 28, 2026

Show notes

Andrew and Justin discuss the Hugging Face breach, which saw an unreleased OpenAI model find its way into the company's systems during internal testing. They also discuss the launch of Opus 5, the ongoing progress made by Chinese models, and the crunch for compute as 10GW data center plans start to make some motion. Plus, a tribute to the late John C. Dvorak.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro

01:23 - Hugging Face

15:45 - Chinese Models

26:01 - Opus 5

36:11 - 10GW Data Centers

52:29 - RIP John C. Dvorak

Hosted on Acast. See acast.com/privacy for more information.

A Frontier Shakeup Jul 21, 2026

Show notes

Andrew and Justin discuss what's turned out to be a busy July for frontier models, including the launch of Kimi K3, the latest in Anthropic's Fable saga, and the arrival of GPT-5.6. They also dive into Anthropic's strange World Cup ad campaign and OpenAI's relaunched desktop app.

You can watch a video version of this podcast here: https://youtu.be/bTUfneEBn14

Chapters

00:00:00 - Intro

00:01:11 - Kimi and Chinese Models

00:08:26 - Google's Position

00:21:47 - Anthropic's Fable Saga

00:32:31 - Anthropic's World Cup Ad Campaign

00:40:09 - GPT-5.6

00:50:10 - Sites, Speech, and Desktop

00:57:42 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.

The 800-Pound Panda in the Room Jul 07, 2026

Show notes

Andrew and Justin discuss the return of Fable, which arrives with some slight performance changes after a prolonged fight between Anthropic and the US government. What's the future of Anthropic's flagship model, and does it point to the arrival of GPT-5.6 in the coming days? They also discuss GLM-2, Chinese models, and the age of token efficiency, plus a quick look at that Alex Karp interview blowing up your timeline.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro

01:35 - Fable's Return

21:33 - GPT-5.6

30:57 - Chinese Models

46:59 - Alex Karp's CNBC Interview

53:00 - Wrap-up

Hosted on Acast. See acast.com/privacy for more information.