Unsupervised Learning with Jacob Effron

Latest Episodes:

Ep 12: EleutherAI's Aran Komatsuzaki on Open-Source Models' Future and Thought Cloning Jul 19, 2023

Show notes

Jacob and Jordan sit down with EleutherAI's Aran Komatsuzaki to discuss the future of open-source models, thought cloning, his work on GPT-J and more.

0:00 intro

01:06 Aran’s background

02:58 starting work on gpt-j

05:49 gathering data for Lion and gpt-j

08:51 history of EleutherAI

11:16 open vs. closed-source models

19:06 how will open-source models be used going forward

21:33 thought cloning

25:51 building AI models that understand video

29:35 one model to rule them all

31:58 influence of academia in the LLM space

34:33 over-hyped/under-hyped

38:01 Aran’s thoughts on AGI

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

@jordan_segall

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).
On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Ep 11: Stanford Professor Tatsu Hashimoto on AI Biases and Improving LLM Performance Jul 05, 2023

Show notes

Patrick and Jacob sit down with Tatsu Hashimoto, Professor of AI at Stanford, to discuss the incredible open source projects from his research group like Alpaca and AlpacaFarm, whether data, algorithms, fine-tuning or RLHF is most important for performance, if AI is liberal or conservative, and much more!

(0:00) - intro

(1:05) - journey to Stanford

(2:50) - origins of Alpaca

(6:08) - capabilities of the Alpaca model

(16:39) - the future of AI

(20:07) - AlpacaFarm

(21:37) - how to improve language models

(29:15) - do language models form opinions?

(32:15) - how to solve bias in ai

(34:18) - how does academia fit into the world of AI

(42:01) - over-hyped/under-hyped

(46:35) - questions Tatsu doesn’t have time for

With your co-hosts:

@jasoncwarner - Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia - Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase - Partner at Redpoint, Former ML Engineer LinkedIn @jacobeffron - Partner at Redpoint, Former PM Flatiron Health

@jacobeffron - Partner at Redpoint, Former PM Flatiron Health

Ep 10: Insitro CEO Daphne Koller on Using ML to Change Drug Discovery Jun 13, 2023

Show notes

Jacob sits down with Insitro CEO Daphne Koller to discuss founding Coursera, where and how ML can drive the most impact in drug development, and if foundation models can transform core drug discovery work and edtech.

(00:00) - intro

(00:54) - Daphne’s journey

(09:18) - AI and biology discovery

(10:59) - insitro vs. traditional pharma

(20:04) - phenotyping patients

(26:01) - early mistakes

(29:51) - the future of data

(35:33) - partnering with larger pharma companies

(38:17) - impact of LLMs on biopharma

(44:04) - over-hyped/under-hyped

With your co-hosts:

@jasoncwarner - Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia - Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase - Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron - Partner at Redpoint, Former PM Flatiron Health

Ep 9: Open AI VP of Product Peter Welinder on OpenAI’s Strategy and AI’s Future Jun 07, 2023

Show notes

Jacob sits down with OpenAI VP of Product & Partnerships Peter Welinder and guest host Rob Toews (Radical VC Partner) to discuss OpenAI’s strategy, how they think about what they will/won’t build, the future of open source models and when we’ll reach AGI.

(00:00) - intro

(00:42) - where is the value in AI?

(07:51) - how OpenAI prioritizes projects

(14:15) - open-source AI

(25:49) - gaps in AI

(29:36) - risks and downsides

(34:02) - when will we reach super-intellegence?

(40:40) - super-intellegence safety

(43:37) - how OpenAI uses ChatGPT

With your co-hosts:

@jasoncwarner - Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia - Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase - Partner at Redpoint, Former ML Engineer LinkedIn @jacobeffron - Partner at Redpoint, Former PM Flatiron Health

@jacobeffron - Partner at Redpoint, Former PM Flatiron Health

Ep 8: Ex-DeepMind/WhiteHouse Operator Tantum Collins on AI Regulation and Geopolitical Impacts May 31, 2023

Show notes

Jacob sits down with ex-DeepMind / White House operator Teddy Collins to discuss Sam Altman’s Congressional testimony, how AI can enable direct democracy, state-driven economic planning and better orgs, US-China competition and more.

(00:00) - intro

(01:15) - how Teddy came to the AI world

(06:26) - working in government

(12:27) - AI regulation and Sam Altman testimony

(19:19) - the end of humanity

(34:34) - nearer term issues with AI

(41:10) - how the government can use AI

(47:13) - lessons from DeepMind

(54:54) - over-hyped/under-hyped

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

Ep 7: Co-Creator of Databricks Dolly Mike Conover on Open-Source LLMs May 11, 2023

Show notes

Patrick and Jacob sit down with Mike Conover, Staff Software Engineer at Databricks and Co-Creator of Databricks Dolly, the world’s first truly open instruction-tuned LLM, to discuss the magic behind Dolly, Alpaca and other instruction-tuned LLMs, the unreasonable effectiveness of fine-tuning, how they got all Databricks employees to help them curate the Dolly dataset (hint: google forms), and more.

(0:00) - Intro

(5:54) - The birth of Dolly

(12:03) - Data curation at Databricks

(15:34) - Advice for building LLMs

(24:10) - The future of instruction-tuning datasets

(30:43) - UI innovation

(38:16) - The future of machine learning infrastructure

(42:05) - How SkipFlag would be different with the tools we have today

(47:01) - What Mike has learned since Dolly

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

Ep 6: Jasper CEO Dave Rogenmoser on the Future of Writing with AI Apr 19, 2023

Show notes

Jacob and Erica sit down with Jasper CEO Dave Rogenmoser to discuss the future of writing with AI and what it means for marketers and the internet, Jasper post ChatGPT, hosting a massive Gen AI conference, going upmarket and more.

(00:56) - How Jasper works and how people are using it

(02:00) - Dave's journey leading up to Jasper

(04:30) - The moment Dave knew he was onto something with Jasper

(07:30) - Where Jasper works well for business applications

(10:55) - How the content of the internet might change going forward with the use of AI for content creation

(13:00) - What a writer's workflow in the future looks like with AI

(14:50) - How the introduction of ChatGPT impacted Jasper

(17:46) - Looking at the long term opportunity for differentiation for Jasper

(23:16) - Tuning a 'brand voice' through Jasper's AI

(26:15) - Jasper's role in bringing community together, ie with their recent Gen AI Conference

(30:00) - How Dave thinks about the future of work for his kids

(32:20) - The productivity boost clients see through using Jasper

(37:00) - Quick fire round

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

Ep 5: You.com CEO Richard Socher on The Future of Search, Open Source Models and AGI Apr 06, 2023

Show notes

Jacob and Erica sit down with You.com CEO and former Salesforce Chief Scientist Richard Socher to discuss building a new search engine and the future of search, his predictions on when open source models will catch up to GPT-4, AGI and more.

(1:29) - Richard's career moves and what motivated them

(6:44) - More about You.com and the ethos behind the company

(9:47) - How You.com integrates 3rd-party apps

(13:30) - Richard's thoughts on the future of AI

(23:19) - The role of academia in the world of AI going forward

(27:30) - Things in AI that Richard is excited about

(31:53) - Quickfire Round

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

Ep 4: Fixie.ai CEO Matt Welsh on How LLMs Will Change the Way We Work Apr 04, 2023

Show notes

Jordan and Erica chat with Fixie.ai co-founder and former Harvard CS professor on the journey from researcher to Google eng leader and then to startup life prior to founding Fixie, a new platform for building LLM-based apps (a Redpoint portfolio company). We also talk about having Mark Zuckerberg in his CS class, ChatGPT Plugins and the AI ecosystem, and what the future might look like for our kids with AGI. You can find Matt on Twitter (@mdwelsh) and learn more about Fixie at https://www.fixie.ai/

(1:06) - Matt talks about his early days at Berklee and Harvard

(6:45) - Making the jump to the start-up world

(8:15) - Matt explains what Fixie does

(11:30) - How Matt thinks about use cases for Fixie

(14:20) - How work might change with AI's integration

(17:04) - The future of LLM's

(20:30) - Matt's take on the race to AGI

(25:20) - How Matt thinks about the future of the world for his kids and how they might use AI

(29:24) - Quick fire round

With your co-hosts:

@jasoncwarner

- Former CTO GitHub, VP Eng Heroku & Canonical

@ericabrescia

- Former COO Github, Founder Bitnami (acq’d by VMWare)

@patrickachase

- Partner at Redpoint, Former ML Engineer LinkedIn

@jacobeffron

- Partner at Redpoint, Former PM Flatiron Health

Ep 3: NEAR CEO Illia Polosukhin on the Origins of the Transformer Paper and The Overlap Between AI and Crypto Mar 21, 2023

Show notes

Jacob and Jason sit down with NEAR CEO and Transformer paper author Illia Polosukhin. They discuss Illia’s fascinating journey from Ukraine to Google and AI to crypto, the origin story behind the “Attention Is All You Need” paper and the overlap between AI and crypto. Illia also shared his thoughts on AGI and the problems that excite him most in AI right now. You can find Illia on Twitter (@ilblackdrago) and learn more about NEAR (@nearprotocol)

(00:39) - Welcoming Illia; how he became interested in AI, transitioning into Crypto and explaining NEAR

(02:40) - Walking through Illia's story more in depth

(07:46) - How the Transformer Paper came to be

(11:24) - Understanding the Transformer Papers' impact

(18:28) - The overlap of Crypto and AI and how Illia sees the future of how they develop together

(26:47) - Illia's views on AGI

(30:46) - Optimism vs pessimism of the future of machine learning as a tool

(41:32) - What problems Illia sees in AI right now

(45:03) - Rapid fire questions

(47:36) - Where to learn more about NEAR and Illia

With your co-hosts:

@jasoncwarner