Training Data

Latest Episodes:

Roblox Studio Head Stef Corazza: Using AI to Empower Creators Feb 04, 2025

Show notes

Stef Corazza leads generative AI development at Roblox after previously building Adobe’s 3D and AR platforms. His technical expertise, combined with Roblox’s unique relationship with its users, has led to the infusion of AI into its creation tools. Roblox has assembled the world’s largest multimodal dataset. Stef previews the Roblox Assistant and the company’s new 3D foundation model, while emphasizing the importance of maintaining positive experiences and civility on the platform.

Mentioned in this episode:

Driving Empire: A Roblox car racing game Stef particularly enjoys
RDC: Roblox Developer Conference
Ego.live: Roblox app to create and share synthetic worlds populated with human-like generative agents and simulated communities|
PINNs: Physics Informed Neural Networks
ControlNet: A model for controlling image diffusion by conditioning on an additional input image that Stef says can be used as a 2.5D approach to 3D generation.
Neural rendering: A combination of deep learning with computer graphics principles developed by Nvidia in its RTX platform

Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

ReflectionAI Founder Ioannis Antonoglou: From AlphaGo to AGI Jan 28, 2025

Show notes

Ioannis Antonoglou, founding engineer at DeepMind and co-founder of ReflectionAI, has seen the triumphs of reinforcement learning firsthand. From AlphaGo to AlphaZero and MuZero, Ioannis has built the most powerful agents in the world. Ioannis breaks down key moments in AlphaGo's game against Lee Sodol (Moves 37 and 78), the importance of self-play and the impact of scale, reliability, planning and in-context learning as core factors that will unlock the next level of progress in AI.

Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital

Mentioned in this episode:

PPO: Proximal Policy Optimization algorithm developed by DeepMind in game environments. Also used by OpenAI for RLHF in ChatGPT.
MuJoCo: Open source physics engine used to develop PPO
Monte Carlo Tree Search: Heuristic search algorithm used in AlphaGo as well as video compression for YouTube and the self-driving system at Tesla
AlphaZero: The DeepMind model that taught itself from scratch how to master the games of chess, shogi and Go
MuZero: The DeepMind follow up to AlphaZero that mastered games without knowing the rules and able to plan winning strategies in unknown environments
AlphaChem: Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies
DQN: Deep Q-Network, Introduced in 2013 paper, Playing Atari with Deep Reinforcement Learning
AlphaFold: DeepMind model for predicting protein structures for which Demis Hassabis, John Jumper and David Baker won the 2024 Nobel Prize in Chemistry

Kumo’s Hema Raghavan: Turning Graph AI into ROI Jan 21, 2025

Show notes

Hema Raghavan is co-founder of Kumo, a company that makes graph neural networks accessible to enterprises by connecting to their relational data stored in Snowflake and Databricks. Hema talks about how running GNNs on GPUs has led to breakthroughs in performance as well as the query language Kumo developed to help companies predict future data points. Although approachable for non-technical users, the product provides full control for data scientists who use Kumo to automate time-consuming feature engineering pipelines.

Mentioned in this episode:

Graph Neural Networks: Learning mechanism for data in graph format, the basis of the Kumo product
Graph RAG: Popular extension of retrieval-augmented generation using GNNs
LiGNN: Graph Neural Networks at LinkedIn paper
KDD: Knowledge Discovery and Data Mining Conference

Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

Databricks Founder Ion Stoica: Turning Academic Open Source into Startup Success Jan 14, 2025

Show notes

Berkeley professor Ion Stoica, co-founder of Databricks and Anyscale, transformed the open source projects Spark and Ray into successful AI infrastructure companies. He talks about what mattered most for Databricks' success -- the focus on making Spark win and making Databricks the best place to run Spark. He highlights the importance of striking key partnerships -- the Microsoft partnership in particular that accelerated Databricks' growth and contributed to Spark's dominance among data scientists and AI engineers. He also shares his perspective on finding new problems to work on, which holds lessons for aspiring founders and builders: 1) building systems in new areas that, if widely adopted, put you in the best position to understand the new problem space, and 2) focusing on a problem that is more important tomorrow than today.

Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital

Mentioned in this episode:

Spark: The open source platform for data engineering that Databricks was originally based on.
Ray: Open source framework to manage, executes and optimizes compute needs across AI workloads, now productized through Anyscale
MosaicML: Generative AI startups founded by Naveen Rao that Databricks acquired in 2023.
Unity Catalog: Data and AI governance solution from Databricks.
CIB Berkeley: Multi-strategy hedge fund at UC Berkeley that commercializes research in the UC system.
Hadoop: A long-time leading platform for large scale distributed computing.
VLLM and Chatbot Arena: Two of Ion’s students’ projects that he wanted to highlight.

XBOW CEO and GitHub Copilot Creator Oege de Moor: Cracking the Code on Offensive Security With AI Dec 10, 2024

Show notes

Oege de Moor, the creator of GitHub Copilot, discusses how XBOW’s AI offensive security system matches and even outperforms top human penetration testers, completing security assessments in minutes instead of days. The team’s speed and focus is transforming the niche market of pen testing with an always-on service-as-a-software platform. Oege describes how he is building a large and sustainable business while also creating a product that will “protect all the software in the free world.” XBOW shows how AI is essential for protecting software systems as the amount of AI-generated code increases along with the scale and sophistication of cyber threats.

Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

Mentioned in this episode:

Semmle: Oege’s previous startup, a code analysis tool to secure software, acquired in 2019 by GitHub
Nico Waisman: Head of security at XBOW, previously a researcher at Semmle
The Bitter Lesson: Highly influential post by Richard Sutton
HackerOne: Cybersecurity company that runs one of the largest bug bounty programs
Suno: AI songwriting app that Oege loves
Machines of Loving Grace: Essay by Anthropic founder, Dario Amodei

Ramp CEO Eric Glyman: Using AI to Build “Self-Driving Money” Dec 03, 2024

Show notes

When ChatGPT ushered in a new paradigm of AI in everyday use, many companies attempted to adapt to the new paradigm by rushing to add chat interfaces to their products. Eric has a different take—he doesn’t think chatbots are the right form factor for everything. He thinks “zero-touch” automation that works invisibly in the background can be more valuable in many cases. He cites self-driving cars as an analogy—or in this case, “self-driving money.” Ramp is a new kind of finance management company for businesses, offering AI-powered financial tools to help companies handle spending and expense processes. We’ll hear why Eric thinks AI that you never see is one of the most powerful instruments for reducing time spent on drudgery and unlocking more time for meaningful work.

Hosted by: Ravi Gupta and Sonya Huang, Sequoia Capital

Mentioned in this episode:

Paribus: Glyman’s previous company, acquired by Capital One in 2016
Karim Atiyeh: Cofounder and CTO at Ramp and Glyman’s cofounder at Paribus
Devin: AI agent product from Cognition Labs and Glyman’s favorite AI app
Hit Refresh: Book by Satya Nadella

Dust’s Gabriel Hubert and Stanislas Polu: Getting the Most From AI With Multiple Custom Agents Nov 26, 2024

Show notes

Founded in early 2023 after spending years at Stripe and OpenAI, Gabriel Hubert and Stanislas Polu started Dust with the view that one model will not rule them all, and that multi-model integration will be key to getting the most value out of AI assistants. In this episode we’ll hear why they believe the proprietary data you have in silos will be key to unlocking the full power of AI, get their perspective on the evolving model landscape, and how AI can augment rather than replace human capabilities.

Hosted by: Konstantine Buhler and Pat Grady, Sequoia Capital

00:00 - Introduction

02:16 - One model will not rule them all

07:15 - Reasoning breakthroughs

11:15 - Trends in AI models

13:32 - The future of the open source ecosystem

16:16 - Model quality and performance

21:44 - “No GPUs before PMF”

27:24 - Dust in action

37:40 - How do you find “the makers”

42:36 - The beliefs Dust lives by

50:03 - Keeping the human in the loop

52:33 - Second time founders

56:15 - Lightning round

Clay’s Kareem Amin on Building the Sales ‘System of Action’ with AI Nov 19, 2024

Show notes

Clay is leveraging AI to help go-to-market teams unleash creativity and be more effective in their work, powering custom workflows for everything from targeted outreach to personalized landing pages. It’s one of the fastest growing AI-native applications, with over 4,500 customers and 100,000 users. Founder and CEO Kareem Amin describes Clay’s technology, and its approach to balancing imagination and automation in order to help its customers achieve new levels of go-to-market success.

Hosted by: Alfred Lin, Sequoia Capital

Decart’s Dean Leitersdorf on AI-Generated Video Games and Worlds Nov 13, 2024

Show notes

Can GenAI allow us to connect our imagination to what we see on our screens? Decart’s Dean Leitersdorf believes it can.

In this episode, Dean Leitersdorf breaks down how Decart is pushing the boundaries of compute in order to create AI-generated consumer experiences, from fully playable video games to immersive worlds. From achieving real-time video inference on existing hardware to building a fully vertically integrated stack, Dean explains why solving fundamental limitations rather than specific problems could lead to the next trillion-dollar company.

Hosted by: Sonya Huang and Shaun Maguire, Sequoia Capital

00:00 Introduction

03:22 About Oasis

05:25 Solving a problem vs overcoming a limitation

08:42 The role of game engines

11:15 How video real-time inference works

14:10 World model vs pixel representation

17:17 Vertical integration

34:20 Building a moat

41:35 The future of consumer entertainment

43:17 Rapid fire questions

How Glean CEO Arvind Jain Solved the Enterprise Search Problem – and What It Means for AI at Work Oct 29, 2024

Show notes

Years before co-founding Glean, Arvind was an early Google employee who helped design the search algorithm. Today, Glean is building search and work assistants inside the enterprise, which is arguably an even harder problem. One of the reasons enterprise search is so difficult is that each individual at the company has different permissions and access to different documents and information, meaning that every search needs to be fully personalized. Solving this difficult ingestion and ranking problem also unlocks a key problem for AI: feeding the right context into LLMs to make them useful for your enterprise context. Arvind and his team are harnessing generative AI to synthesize, make connections, and turbo-change knowledge work. Hear Arvind’s vision for what kind of work we’ll do when work AI assistants reach their potential.

Hosted by: Sonya Huang and Pat Grady, Sequoia Capital

00:00 - Introduction

08:35 - Search rankings

11:30 - Retrieval-Augmented Generation

15:52 - Where enterprise search meets RAG

19:13 - How is Glean changing work?

26:08 - Agentic reasoning

31:18 - Act 2: application platform

33:36 - Developers building on Glean

35:54 - 5 years into the future

38:48 - Advice for founders