Unsupervised Learning with Jacob Effron

Latest Episodes:

Ep 69: Co-Founder of Databricks & LMArena on Current Eval Limitations, Why China is Winning Open Source and Future of AI Infrastructure Jun 17, 2025

Show notes

Ion Stoica helped define the modern data stack. Now he’s coming for AI evaluation. From co-founding Databricks and Anyscale to launching LMArena, Ion has shaped the infrastructure underlying some of the biggest shifts in computing. In this conversation, he unpacks what most people get wrong about model evaluation, the infrastructure challenges ahead for agents and heterogeneous compute, and why he believes the U.S. is structurally disadvantaged in open-source AI compared to China.

(0:00) Intro
(0:49) Launching a New Startup: LMArena
(1:01) The Origin of the Vicuna Model
(1:54) Challenges in Model Evaluation
(6:33) Becoming a Company
(7:47) Expanding Evaluation Capabilities
(13:48) The Importance of Human-Based Evaluations
(18:56) Open Source vs. Proprietary Models
(23:05) Infrastructure and Collaboration Challenges
(28:22) China's Strategic Advantages in Technology
(29:54) Opportunities in AI Infrastructure
(31:50) Challenges in AI Model Optimization
(35:49) The Role of Data in AI Enterprises
(39:31) Reflections on AI Progress and Predictions
(50:40) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).
On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Ep 68: CEO of Mercor Brendan Foody on Evals Replacing Knowledge Work, AI x Hiring Today & the Future of Data Labeling Jun 04, 2025

Show notes

Brendan Foody is the co-founder and CEO of Mercor, a company building the infrastructure for AI-native labor markets. Mercor’s platform is already used by top AI labs to label data, evaluate human and AI candidates, and make performance-driven hiring decisions.

They’re operating at the intersection of recruiting, evals, and foundation model development—helping companies shift from intuition to measurable prediction. Brendan and his team recently raised $100M and are working with some of the most advanced players in the AI ecosystem today.

(0:00) Intro

(1:17) State of AI in Talent Evaluation

(1:54) Improvements in AI Models

(4:07) Mercor Background and Mission

(5:09) AI Use Cases in Hiring

(13:43) Data Labeling Landscape

(16:48) Expanding Beyond Coding

(18:39) Company Vision and Market Strategy

(21:11) Meeting with xAI

(23:47) Does Mercor Use Their Own Product?

(25:41) Exploring Multimodal Capabilities

(28:03) Skills for the Future: Embracing AI

(29:29) The Demand for Software Engineers

(34:55) Foundation Model Landscape

(38:42) AI Regulations

(39:57) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 67: Max Junestrand (CEO, Legora) on Differentiating and Pricing AI Apps & How the Legal Industry Will Evolve May 27, 2025

Show notes

Jacob and Logan sit down with Max Junestrand, founder and CEO of Legora - a rapidly growing legal AI platform (and Redpoint portfolio company). After announcing their Series B last week, Max joined the show to discuss why law is uniquely suited for AI, what it takes to scale an enterprise-ready product across global markets, and a few crazy moments from Legora’s journey so far. They dig into product strategy, lessons on evolving alongside foundational models, and how AI is reshaping the future of law firms. Whether you're building in AI or just curious how it’s being applied in complex industries, this one’s packed with practical insights.

(0:00) Intro

(1:30) The Evolution of AI in Law

(2:43) AI's Impact on Legal Processes

(8:28) Advantages Over Other Players in the AI Law Space

(12:19) Challenges in Educating Users

(17:28) The Hardest Part of Building Legora

(18:46) Pricing Models and Cost Management

(25:42) YC Experience and Commercial Focus

(28:11) Being Patient When Releasing Products

(30:58) Maintaining a Fast-Paced Work Culture

(33:24) Rapid Growth and Market Penetration

(36:59) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 66: Member of Technical Staff at Anthropic Sholto Douglas on Claude 4, Next Phase for AI Coding, and the Path to AI Coworkers May 22, 2025

Show notes

Sholto Douglas, a Member of Technical Staff at Anthropic, joined Unsupervised Learning to break down why coding is the clearest early signal of model progress, how AI agents are already accelerating research, and what it’ll take to unlock real-world breakthroughs in fields like biology and robotics.

(0:00) Intro
(0:48) Claude 4
(1:30) Capabilities and Improvements
(2:29) Practical Applications and Advice
(3:04) Future of AI in Coding
(4:38) Managing Multiple AI Models
(11:20) The Barrier to Agents is Reliability
(16:35) Agents Conducting Research
(19:54) Impact of Models on World GDP
(25:14) Most Important Metrics in Model Improvement
(29:53) Stories of Model Creativity
(32:45) How Often Will New Models Be Shipped in the Future?
(39:51) Day-to-Day Work of AI Researchers
(46:46) The Future of AI and Society
(51:26) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 65: Co-Authors of AI-2027 Daniel Kokotajlo and Thomas Larsen On Their Detailed AI Predictions for the Coming Years May 14, 2025

Show notes

The recent AI 2027 report sparked widespread discussion with its stark warnings about the near-term risks of unaligned AI.

Authors @Daniel Kokotajlo (former OpenAI researcher now focused full-time on alignment through his nonprofit, @AI Futures, and one of TIME’s 100 most influential people in AI) and @Thomas Larsen joined the show to unpack their findings.

We talk through the key takeaways from the report, its policy implications, and what they believe it will take to build safer, more aligned models.

(0:00) Intro
(1:15) Overview of AI 2027
(2:32) AI Development Timeline
(4:10) Race and Slowdown Branches
(12:52) US vs China
(18:09) Potential AI Misalignment
(31:06) Getting Serious About the Threat of AI
(47:23) Predictions for AI Development by 2027
(48:33) Public and Government Reactions to AI Concerns
(49:27) Policy Recommendations for AI Safety
(52:22) Diverging Views on AI Alignment Timelines
(1:01:30) The Role of Public Awareness in AI Safety
(1:02:38) Reflections on Insider vs. Outsider Strategies
(1:10:53) Future Research and Scenario Planning
(1:14:01) Best and Worst Case Outcomes for AI
(1:17:02) Final Thoughts and Hopes for the Future

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 64: GPT 4.1 Lead at OpenAI Michelle Pokrass: RFT Launch, How OpenAI Improves Its Models & the State of AI Agents Today May 08, 2025

Show notes

In this episode, I sit down with Michelle Pokrass, who leads a research team at OpenAI within post-training focused on improving models for power users: developers using OpenAI models in the API and power users in ChatGPT. We unpack how OpenAI prioritized instruction-following and long context, why evals have a 3-month shelf life, what separates successful AI startups, and how the best teams are fine-tuning to push past the current frontier.

If you’ve ever wondered how OpenAI really decides what to build, and how it affects what you should build, this one’s for you.

(0:00) Intro

(1:03) Deep Dive into GPT-4.1 Development

(2:23) User Feedback and Model Evaluation

(4:01) Challenges and Improvements in Model Training

(5:54) Advancements in AI Coding Capabilities

(9:11) Future of AI Models and Fine-Tuning

(20:44) Multimodal Capabilities

(22:59) Deep Tech Applications and Data Efficiency

(24:14) Preference Fine Tuning vs. RFT

(26:29) Choosing the Right Model for Your Needs

(28:18) Prompting Techniques and Model Improvements

(32:10) Future Research and Model Enhancements

(39:14) Power Users and Personalization

(40:22) Personal Journey and Organizational Growth

(43:37) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 63: Khan Academy Founder/CEO on Salman Khan on Classrooms in 20 years, Rolling out to 1.4M Users & Sal’s Hopes for AI Education Apr 29, 2025

Show notes

When Khan Academy launched Khanmigo, Salman Khan thought they might reach 100k users by 2025. Today, they’re at 1.4 million.

🎧 Sal joined us on Unsupervised Learning to talk about AI's role in education— from the vantage point of someone deploying it at scale. As founder of Khan Academy, he’s overseen the rollout of AI tools to over a million teachers and students, giving him a front-row seat to what’s actually working in classrooms.

(0:00) Intro
(1:17) The Vision for Future Classrooms
(4:28) Khan Academy's AI Initiatives
(7:06) Proactive AI and Engagement
(10:33) Teacher and Student Experiences
(18:24) District-Level Adoption and Policy
(25:56) Gamification and Engagement
(27:36) Evaluating AI Models
(31:03) Global Impact and Future Prospects
(37:43) Challenges and Innovations in AI Education
(44:19) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 62: CEO of Cohere Aidan Gomez on Scaling Limits Emerging, AI Use-cases with PMF & Life After Transformers Apr 15, 2025

Show notes

Aidan joined this week’s Unsupervised Learning for a wide-ranging conversation on model architectures, enterprise adoption, and what’s breaking in the foundation model stack.

If you’re building or investing in AI infrastructure, Aidan is worth listening to. He co-authored the original Transformer paper, leads one of the most advanced model labs outside of the hyperscalers, and is now building for real-world enterprise deployment with Cohere’s agent platform, North. Cohere serves thousands of customers across sectors like finance, telco, and healthcare — and they’ve made a name for themselves by staying model-agnostic, privacy-forward, and deeply international (with major bets in Japan and Korea)

(0:00) Intro
(0:32) Enterprise AI
(3:23) Custom Integrations and Future of AI Agents
(4:33) Enterprise Use Cases for Gen AI
(7:02) The Importance of Reasoning in AI Models
(10:38) Custom Models and Synthetic Data
(17:48) Cohere's Approach to AI Applications
(23:24) Future Use Cases and Market Fit
(27:11) Building a Unified Automation Platform
(27:34) Strategic Decisions in the AI Journey
(29:19) International Partnerships and Language Models
(31:05) Future of Foundation Models
(32:27) AI in Specialized Domains
(34:40) Challenges in Data Integration
(35:06) Emerging Foundation Model Companies
(35:31) Technological Frontiers and Architectures
(37:29) Scaling Hypothesis and Model Capabilities
(42:26) AI Research Culture and Team Building
(44:39) Future of AI and Societal Impact
(48:31) Addressing AI Risks

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 61: Redpoint’s AI Investors Break Down What Separates Enduring AI Companies from the Hype Apr 09, 2025

Show notes

At Redpoint’s annual meeting with investors, Redpoint partners Scott Raney, Alex Bard, Patrick Chase and Jacob Effron shared unfiltered thoughts on some of the most topical questions in AI today, where value will accrue, which industries are best positioned for defensibility at the application layer, and more.

(0:00) Intro
(0:39) AI Investment Landscape
(1:30) Market Projections
(3:25) Strategic Imperatives in AI
(6:04) AI Model Layer Insights
(10:06) Infrastructure Layer
(11:46) Application Layer
(15:04) Vertical AI SaaS Opportunities
(21:45) Evaluating Early-Stage Founders
(23:39) First Mover Advantage and Competitive Dynamics
(25:28) Domain Expertise vs. AI Expertise
(26:58) Why AI Startups Fail
(29:41) Startups vs. Incumbents
(35:15) Navigating High AI Valuations

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint

Ep 60: Swyx and Alessio (Latent Space) on What has PMF Today, Google is Cooking & GPT Wrappers are Winning Mar 28, 2025

Show notes

To unpack some of the most topical questions in AI, I’m joined by two fellow AI podcasters: Swyx and Alessio Fanelli, co-hosts of the Latent Space podcast. We’ve been wanting to do a cross-over episode for a while and finally made it happen.

Swyx brings deep experience from his time at AWS, Temporal, and Airbyte, and is now focused on AI agents and dev tools. Alessio is an investor at Decibel, where he’s been backing early technical teams pushing the boundaries of infrastructure and applied AI. Together they run Latent Space, a technical newsletter and podcast by and for AI engineers.

To subscribe or learn more about Latent Space, click here: https://www.latent.space/

(0:00) Intro
(1:08) Reflecting on AI Surprises of the Past Year
(2:24) Open Source Models and Their Adoption
(6:48) The Rise of GPT Wrappers
(7:49) Challenges in AI Model Training
(10:33) Over-hyped and Under-hyped AI Trends
(24:00) The Future of AI Product Market Fit
(30:27) Google's Momentum and Customer Support Insights
(33:16) Emerging AI Applications and Market Trends
(35:13) Challenges and Opportunities in AI Development
(39:02) Defensibility in AI Applications
(42:42) Infrastructure and Security in AI
(50:04) Future of AI and Unanswered Questions
(55:34) Quickfire

With your co-hosts:

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@jordan_segall

- Partner at Redpoint