Training Data

Latest Episodes:

European Defense Sovereignty Starts in the Factory: Stark’s Uwe Horstmann and Johannes Schaback Oct 28, 2026

Show notes

Two years after its founding, Stark has roughly 700 employees and strike drones in the field in Ukraine. CEO Uwe Horstmann and CTO Johannes Schaback argue that the real lesson of that war is not any single weapon: the innovation ecosystem behind it is the wonder weapon. They explain why Stark designs every system for mass manufacturing from day one, runs 10 to 15 programs like a venture portfolio, and views the staff sergeant who operates a system as just as important as the general who buys it. Johannes recounts the publicly reported failed trial that became a forging moment, as engineers worked around the clock at a freezing test range until they landed three consecutive hits. They discuss NATO’s principles of responsible AI use and the hard-coded constraints in Stark’s autonomy stack. And Uwe lays out his next big idea: defense contracts structured like cloud contracts with surge capacity built in, so production itself becomes deterrence.

Hosted by Pat Grady and Luciana Lixandru, Sequoia Capital

Chapters

0:00 – Introduction

1:03 – Billions in contracts in 18 months: Stark's four-part playbook

2:28 – The staff sergeant matters as much as the general, and wood beats plastic

7:42 – Why Europe is ready for a neo prime, and why procurement reform will last

11:45 – Startup DNA, venture-style programs, and "as complex as a washing machine"

16:27 – Born out of fire: the failed trial and the round-the-clock comeback

21:13 – Where Stark finds talent, and why defense is no longer off-limits

23:48 – Anatomy of a strike, from the wooden crate to the kill zone

27:22 – Vertical integration, multiple domains, and shifting left on manufacturing

32:53 – After the Hugging Face hack: guardrails on autonomous weapons

35:56 – Matching the primes on R&D and scaling to 700 people in two years

39:24 – Deterrence by contract, and Europe's next chapter with the US

Inside the Robot Data Factory: Mecka's Josh Gao on Sensorizing Humans to Train Robots Oct 20, 2026

Show notes

Robotics never got the internet's gift of free training data, so someone has to manufacture it. Josh Gao, founder and CEO of Mecka AI, runs one of the largest robot data factories, and he explains what that actually involves: custom baseball caps and tactile gloves because nothing off the shelf captures force, 100-gigabit upload lines to move hundreds of petabytes, and custom quality gates. Josh lays out the training recipe the industry is converging on (roughly a million hours of egocentric human data, tens of thousands of hours of robot data, and a few hours of in-domain teleop ) and why tactile and simulation data still have to enter the "data pot of soup." He argues the field is still orders of magnitude behind Tesla's FSD, which spent a decade on two degrees of freedom while a robot hand has dozens. His take: there won't be a single GPT-3 moment for robotics, but a steady unlock of commercial tasks, starting with constrained industrial work and ending, he hopes, with robots mining asteroids.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:46 – What Mecka is: pixels, actions and logic

2:45 – Is robot data economically feasible without the internet?

4:15 – Teaching robots about carbon life

5:27 – Cloud kitchens, stablecoins and the path to robotics

7:03 – Entering robotics with a fresh lens

8:45 – What egocentric data is and why Mecka bet on it

10:09 – The first pilot: aunties and uncles with a mobile app

10:42 – Force data and tactile gloves

11:55 – Inside the operations of a data factory

14:41 – Heat-mapping the taxonomy of human work

15:42 – Quality gates, fraud and the McDonald’s POV videos

17:44 – Earning a margin as a data intermediary

19:33 – Millions of hours vs. Tesla’s decade

20:55 – Why it isn’t binary: unlocking commercial tasks along the way

22:02 – Home data is plentiful; home robots are the hardest

23:24 – Useful robots can be deployed today

24:35 – When robotics gets an FSD-style flywheel

26:42 – Deployment diffusion and the modern systems integrator

28:21 – Favorite vertical robot categories; mining an asteroid

30:08 – The hardware slate: Waymo-fying people

30:55 – EgoVerse

32:04 – The data pot of soup: the converged training mix

33:54 – Move 37 for robotics: AlphaGo and Federer

36:02 – World models and the training mix

36:58 – Will scaling the current mix get us to the promised land?

39:05 – Will the AGI labs steamroll robotics?

40:50 – Sci-fi roots and the Kawasaki Corleo

42:44 – How many robots in a decade?

David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience Oct 13, 2026

Show notes

David Silver spent two years building AlphaGo on human data. When he removed human data altogether building AlphaGo Zero, the AI surpassed AlphaGo in under two days. He left DeepMind to run that same bet in pursuit of superintelligence. David and Lasse Espeholt, co-founders of Ineffable Intelligence, explain their plan to build a “superlearner”—an agent that starts knowing nothing and discovers science, mathematics, and language for itself by direct experience instead of through a pre-training run. Silver argues that human-language reasoning is a ceiling, not a foundation, that alignment via chain-of-thought monitoring is already reaching its limits, and that the science of how superintelligences coexist should be worked out in simulation before anything touches the real world. Espeholt describes the GPU cluster and the lineage from IMPALA and SEED RL that makes the experiment possible. Silver’s forecast: strong evidence within a year, superintelligence in three.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:09 – What held true from Go to today, and what the LLM era changed

3:35 – Why leave DeepMind to start a new lab

5:58 – Why "Ineffable" (and the secret second reason)

7:03 – Lasse's path: scaling RL at DeepMind and Google Brain

9:31 – Defining a super learner and first contact

12:35 – Super learners vs. LLMs that make discoveries

13:59 – Day to day: the RL stack, crucibles and human-journey milestones

16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity

21:16 – What a crucible looks like

24:14 – Go: one agent is boring, two is infinite

25:41 – Will agents rediscover language?

27:22 – Removing human priors: AlphaGo to AlphaZero

32:05 – Keeping human bias out of the crucibles

33:35 – Confidence, timing and super forecasting

36:24 – Is compute the driver?

38:13 – The ceiling on simulation and Rich Sutton's objection

43:44 – Alignment and the science of superintelligence

47:45 – Monitoring agents that don't think in English

51:01 – Pacing the frontier

52:54 – The compute setup: Vera Rubin on GCP

55:18 – Training, inference, simulation and RL scaling laws

57:40 – IMPALA and SEED RL

1:00:58 – 20 people and the compute-per-person ratio

1:04:49 – A research-led culture

1:05:47 – Predictions

Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI Oct 06, 2026

Show notes

At 100,000-accelerator scale, something fails multiple times an hour, which is why Google's Chief Technologist for AI Infrastructure Amin Vahdat thinks FLOPS is a vanity metric. The metric that matters is what he calls goodput: the useful work a workload actually delivers through real failures. Amin walks through the calculus that split the TPU line into 8i and 8t for the first time, why the TPU's core primitives haven't changed since v1, and how Google and DeepMind co-design in the same rooms, intercepting chip architectures mid-flight before tape-out. He explains why long-horizon agents are sending demand for CPUs and storage through the roof alongside accelerators, how optical circuit switches reroute light to a spare rack in milliseconds, and why Google would rather wait on a utility than build its own gigawatt. We also cover orbital data centers and the multi-megawatt rack of 2036.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:47 – What makes a data center an AI data center

5:30 – Goodput, not FLOPS: holding yourself accountable when something fails every hour

11:52 – Doubling token capacity every six months, and where the gains actually come from

15:32 – The TPU bet: from a contrarian call in 2013 to splitting 8i and 8t

23:30 – The case for and against co-design

26:11 – Shoulder to shoulder with DeepMind: intercepting chips mid-flight

34:16 – Long-horizon agents change the shape of the data center

37:50 – Optical circuit switching and the state of networking

42:35 – Power is the binding constraint: utilities, gigawatts, and sizing a data center

49:23 – Training vs. serving clusters, seven-year-old TPUs, and open standards

58:29 – Orbital data centers and the supercomputer of 2036

Box's Aaron Levie: On Reinventing Yourself in the AI Age and Enterprise Diffusion Sep 15, 2026

Show notes

Starting a company is hard. Reinventing your company for AI as a public company with quarterly earnings results is even harder. Aaron Levie has pulled off the transition with Box and offers hard-won advice for founders. The cofounder and CEO of Box argues the value isn't only in the model; it's in the bridge from a model's raw capability to the actual workflow inside a bank, a law firm, or a pharma company. That's the case for the application layer, and Box is building it: an agent harness tuned so tightly to its own file system, permissions, and search that it beats handing the raw API to Claude or ChatGPT on both accuracy and latency. Aaron explains why token subsidies from the labs can't last, why you want a model-agnostic company routing your tokens rather than the one selling them, and why coding diffused fast while the rest of knowledge work won't. (There's no "give us your GitHub" for a sales rep.) His prediction: within five years, 90% of enterprise tokens go to work no human user ever initiated.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:55 – Are application companies the hottest neolabs?

6:56 – Will the labs move up the stack?

12:34 – Box and betting the company on AI

16:50 – Hero use cases: reading a million contracts and long-running agents

18:42 – Work slop: why AI code is embraced but AI content isn't

24:08 – Building Box's agentic harness and the evals that matter

27:23 – The state of the model race

29:25 – Open-weight model adoption in the enterprise

32:34 – Memory, continual learning, and what belongs in the weights

37:29 – Box Labs and systems of record in a world of agents

44:55 – Will chat be the dominant UI for enterprise AI?

48:00 – Why coding diffused fast and the rest of knowledge work hasn't

54:31 – Staying wired in, making a company AI-first, and what it takes to win

Making Cities Awesome: Peregrine’s Nick Noone & Ben Rudolph Sep 01, 2026

Show notes

Most public safety technology companies grow by collecting more data. Peregrine inverted the model: no sensors, no new data, a business built on connecting the data and information cities already own. Co-founders Nick Noone and Ben Rudolph received more than two dozen no's before San Pablo PD let them in the door in February 2018. Today, Peregrine powers law enforcement, emergency medical services, fire and rescue, and other services in more than 400 cities and communities globally. Nick and Ben explain their north star for data sovereignty, and discuss how Peregrine's philosophy and privacy-first approach to data access and ownership preserves individual privacy and cities' sovereignty. They walk through how AI and long-horizon agents are being deployed: a cold case agent that reproduced an exoneration detectives had reached by hand, a Wisconsin county that placed a suspect using cell records buried in 300GB of evidence, identifying threats to a synagogue, root-causing an escalation in weather-related incidents, and more.

00:00 Introduction

02:07 What Forward Deployed Engineering Means

03:58 What Silicon Valley Gets Wrong

05:23 UNHCR, Dimagi And Downstream Data Problems

08:25 Why Cities, Why Safety

10:45 Two Dozen Nos And San Pablo PD

14:19 Building Through Defund The Police

18:16 The Inversion Of The Collection Model

21:20 Data Ownership And Governance

22:57 From Nice Search To Deep Analysis

29:59 Agents Writing The Integrations

31:45 The Cold Case Agent

35:02 The Anti-Network-Effect Proposition

38:40 Facial Recognition And Hard Decisions

40:48 Technology For The Underdogs

42:54 Trusting The Individual Contributor

48:50 Ten Thousand Cities

Parallel’s Parag Agrawal: Building a New Web for AI Agents Aug 25, 2026

Show notes

Parag Agrawal is making a bet that goes against two decades of web search: agents will query the web a thousand times more than humans ever have, and the infrastructure built around human clicks is wrong for them. The former Twitter CEO, now founder and CEO of Parallel Web Systems, explains why Parallel treats human click data as a bug and trains on agent feedback instead. He unpacks the counterintuitive choice to ship a search agent before a search engine, building an index incrementally, and how the new Turbo product cut agentic search to 200 milliseconds. But the problem Parag keeps returning to is economic: the ad-supported internet collapses when agents show up instead of people. His fix draws on Shapley values to pay content owners for the value their pages provide agents, with real dollars reaching publishers, he predicts, within 12 to 24 months.

Hosted by Sonya Huang and Andrew Reed, Sequoia Capital

00:00 Introduction 03:25 What Is Web Search 05:17 Why Start a New Index 07:52 Search Agents First 10:17 Not a Neolab 13:14 Agents vs Google Search 19:38 Inside the Search Stack 28:59 Search Multipliers With Agents 30:21 Meeting Prep Agent Workflows 31:46 Quality Cost Latency And Turbo 32:42 Are Agents Overtaking Humans 34:28 Ads Model Meets Agent Web 37:20 New Incentives For Content 40:48 Shapley Values Attribution 47:46 Parallel Web And Future Vision

Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again Aug 18, 2026

Show notes

Rich Sutton, who helped pioneer reinforcement learning and wrote the seminal AI essay The Bitter Lesson, has now cofounded Oak Lab with his former student Khurram Javed. Their goal: to build agents that continuously learn from their own experience rather than from us. Rich doesn't think he holds a radical view: "I'm not weird. The field is weird." He says all learning is continual, and the field is the one that needed a new name for it. Rich and Khurram argue synthetic data is "a big mistake." Their "big world hypothesis" is that the world is massively more complex than any agent or simulator, so approximations have to be updated continuously rather than frozen at deployment. Rich calls LLMs an unanticipated scientific breakthrough, but says they represent roughly a quarter of intelligence. He says catastrophic forgetting is "totally curable" with the ideas behind their continual backprop algorithm. Khurram explains why the frontier labs can't follow: they sit in a local minimum where a new paradigm gets worse before it gets better. Their target, five to ten years out, is a trillion-parameter mind that keeps learning, stays coherent, and runs on 20 watts.

Hosted by Sonya Huang and Alfred Lin, Sequoia Capital

00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring

Chai Discovery's Bitter Lesson: Drug Design Is Another Scaling Problem Aug 04, 2026

Show notes

Most people treat biology as a bespoke, messy science. Josh Meier and Matt McPartlon, co-founders of Chai Discovery, treat it as an engineering problem. They make the case that drug design obeys the bitter lesson: scale data, models, and compute, and the model can learn what a hand-built pipeline simply couldn't capture. The results are concrete: Chai-2 pushed de novo antibody design from a sub 0.1% hit rate to 16%, turning a needle-in-a-haystack search into something more like designing a key to fit a lock. Josh argues, counterintuitively, that biology is more verifiable than code, and explains why the goal should be more lab experiments, not fewer. Their bet: a design suite that collapses drug discovery from nine months to nine days, and arms the pharma industry rather than competing with it.

Hosted by Pat Grady and Sonali Singh, Sequoia Capital

00:00 Introduction 01:52 From Discovery to Design 03:25 Protein AI Breakthroughs Timeline 06:04 Why Start in 2024 10:13 Diffusion Models Intuition 11:41 Building the Avengers Team 15:22 Hit Rates and Scaling Laws 25:01 Molecular CAD Vision 25:24 Faster Design Loops 26:32 Future Drug Discovery 28:37 Platform Business Model 31:14 Partnering Reality Check 33:44 Data Flywheel Explained 37:16 Staying Ahead at Scale 39:44 Culture and What's Next

Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil Jul 29, 2026

Show notes

Jerry Tworek led reasoning at OpenAI, convinced that scaling reinforcement learning was the path to AGI. Rohan Anil co-led Gemini pre-training and built the Shampoo optimizer. Now they've teamed up at Core Automation on a contrarian premise: the transformer has carried us as far as it can, and the bottleneck to smarter systems is no longer scale — it's the architecture itself. The missing capability is continual learning, models that adapt at test time, which transformers can't do. In-context learning taps out fast (Codex needs compacting after ~20 minutes) and fine-tuning invites catastrophic forgetting. Rohan argues pre-training and RL should be optimized end-to-end, and that transformers spend computation inefficiently. They lay out why the largest labs won't chase alternatives while locked in the coding-agent race, and why building the world's most automated lab starts with automating kernel generation—the one place frontier models still lose to a high-taste human.

Hosted by Sonya Huang and Pat Grady, Sequoia Capital

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected