Gradient Dissent: Conversations on AI

Latest Episodes:

Drago Anguelov — Robustness, Safety, and Scalability at Waymo Jul 14, 2022

Show notes

Drago Anguelov is a Distinguished Scientist and Head of Research at Waymo, an autonomous driving technology company and subsidiary of Alphabet Inc.

We begin by discussing Drago's work on the original Inception architecture, winner of the 2014 ImageNet challenge and introduction of the inception module. Then, we explore milestones and current trends in autonomous driving, from Waymo's release of the Open Dataset to the trade-offs between modular and end-to-end systems.

Drago also shares his thoughts on finding rare examples, and the challenges of creating scalable and robust systems.

Show notes (transcript and links): http://wandb.me/gd-drago-anguelov

---

⏳ Timestamps:

0:00 Intro

0:45 The story behind the Inception architecture

13:51 Trends and milestones in autonomous vehicles

23:52 The challenges of scalability and simulation

30:19 Why LiDar and mapping are useful

35:31 Waymo Via and autonomous trucking

37:31 Robustness and unsupervised domain adaptation

40:44 Why Waymo released the Waymo Open Dataset

49:02 The domain gap between simulation and the real world

56:40 Finding rare examples

1:04:34 The challenges of production requirements

1:08:36 Outro

---

Connect with Drago & Waymo

📍 Drago on LinkedIn: https://www.linkedin.com/in/dragomiranguelov/

📍 Waymo on Twitter: https://twitter.com/waymo/

📍 Careers at Waymo: https://waymo.com/careers/

---

Links:

📍 Inception v1: https://arxiv.org/abs/1409.4842

📍 "SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation", Qiangeng Xu et al. (2021), https://arxiv.org/abs/2108.06709

📍 "GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weighting", Zhao Chen et al. (2022), https://arxiv.org/abs/2201.05938

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Lavanya Shukla

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

James Cham — Investing in the Intersection of Business and Technology Jul 07, 2022

Show notes

James Cham is a co-founder and partner at Bloomberg Beta, an early-stage venture firm that invests in machine learning and the future of work, the intersection between business and technology.

James explains how his approach to investing in AI has developed over the last decade, which signals of success he looks for in the ever-adapting world of venture startups (tip: look for the "gradient of admiration"), and why it's so important to demystify ML for executives and decision-makers.

Lukas and James also discuss how new technologies create new business models, and what the ethical considerations of a world where machine learning is accepted to be possibly fallible would be like.

Show notes (transcript and links): http://wandb.me/gd-james-cham

---

⏳ Timestamps:

0:00 Intro

0:46 How investment in AI has changed and developed

7:08 Creating the first MI landscape infographics

10:30 The impact of ML on organizations and management

17:40 Demystifying ML for executives

21:40 Why signals of successful startups change over time

27:07 ML and the emergence of new business models

37:58 New technology vs new consumer goods

39:50 What James considers when investing

44:19 Ethical considerations of accepting that ML models are fallible

50:30 Reflecting on past investment decisions

52:56 Thoughts on consciousness and Theseus' paradox

59:08 Why it's important to increase general ML literacy

1:03:09 Outro

1:03:30 Bonus: How James' faith informs his thoughts on ML

---

Connect with James:

📍 Twitter: https://twitter.com/jamescham

📍 Bloomberg Beta: https://github.com/Bloomberg-Beta/Manual

---

Links:

📍 "Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions" by Ali Alkhatib and Michael Bernstein (2019): https://doi.org/10.1145/3290605.3300760

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Lavanya Shukla

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Boris Dayma — The Story Behind DALL·E mini, the Viral Phenomenon Jun 17, 2022

Show notes

Check out this report by Boris about DALL-E mini:

https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini-Generate-images-from-any-text-prompt--VmlldzoyMDE4NDAy

https://wandb.ai/_scott/wandb_example/reports/Collaboration-in-ML-made-easy-with-W-B-Teams--VmlldzoxMjcwMDU5

https://twitter.com/weirddalle

Connect with Boris:

📍 Twitter: https://twitter.com/borisdayma

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Tristan Handy — The Work Behind the Data Work Jun 09, 2022

Show notes

Tristan Handy is CEO and founder of dbt Labs. dbt (data build tool) simplifies the data transformation workflow and helps organizations make better decisions.

Lukas and Tristan dive into the history of the modern data stack and the subsequent challenges that dbt was created to address; communities of identity and product-led growth; and thoughts on why SQL has survived and thrived for so long. Tristan also shares his hopes for the future of BI tools and the data stack.

Show notes (transcript and links): http://wandb.me/gd-tristan-handy

---

⏳ Timestamps:

0:00 Intro

0:40 How dbt makes data transformation easier

4:52 dbt and avoiding bad data habits

14:23 Agreeing on organizational ground truths

19:04 Staying current while running a company

22:15 The origin story of dbt

26:08 Why dbt is conceptually simple but hard to execute

34:47 The dbt community and the bottom-up mindset

41:50 The future of data and operations

47:41 dbt and machine learning

49:17 Why SQL is so ubiquitous

55:20 Bridging the gap between the ML and data worlds

1:00:22 Outro

---

Connect with Tristan:

📍 Twitter: https://twitter.com/jthandy

📍 The Analytics Engineering Roundup: https://roundup.getdbt.com/

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Johannes Otterbach — Unlocking ML for Traditional Companies May 12, 2022

Show notes

Johannes Otterbach is VP of Machine Learning Research at Merantix Momentum, an ML consulting studio that helps their clients build AI solutions.

Johannes and Lukas talk about Johannes' background in physics and applications of ML to quantum computing, why Merantix is investing in creating a cloud-agnostic tech stack, and the unique challenges of developing and deploying models for different customers. They also discuss some of Johannes' articles on the impact of NLP models and the future of AI regulations.

Show notes (transcript and links): http://wandb.me/gd-johannes-otterbach

---

⏳ Timestamps:

0:00 Intro

1:04 Quantum computing and ML applications

9:21 Merantix, Ventures, and ML consulting

19:09 Building a cloud-agnostic tech stack

24:40 The open source tooling ecosystem

30:28 Handing off models to customers

31:42 The impact of NLP models on the real world

35:40 Thoughts on AI and regulation

40:10 Statistical physics and optimization problems

42:50 The challenges of getting high-quality data

44:30 Outro

---

Connect with Johannes:

📍 LinkedIn: https://twitter.com/jsotterbach

📍 Personal website: http://jotterbach.github.io/

📍 Careers at Merantix Momentum: https://merantix-momentum.com/about#jobs

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Mircea Neagovici — Robotic Process Automation (RPA) and ML Apr 21, 2022

Show notes

Mircea Neagovici is VP, AI and Research at UiPath, where his team works on task mining and other ways of combining robotic process automation (RPA) with machine learning for their B2B products.

Mircea and Lukas talk about the challenges of allowing customers to fine-tune their models, the trade-offs between traditional ML and more complex deep learning models, and how Mircea transitioned from a more traditional software engineering role to running a machine learning organization.

Show notes (transcript and links): http://wandb.me/gd-mircea-neagovici

---

⏳ Timestamps:

0:00 Intro

1:05 Robotic Process Automation (RPA)

4:20 RPA and machine learning at UiPath

8:20 Fine-tuning & PyTorch vs TensorFlow

14:50 Monitoring models in production

16:33 Task mining

22:37 Trade-offs in ML models

29:45 Transitioning from software engineering to ML

34:02 ML teams vs engineering teams

40:41 Spending more time on data

43:55 The organizational machinery behind ML models

45:57 Outro

---

Connect with Mircea:

📍 LinkedIn: https://www.linkedin.com/in/mirceaneagovici/

📍 Careers at UiPath: https://www.uipath.com/company/careers

---

💬 Host: Lukas Biewald

📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla

Jensen Huang — NVIDIA’s CEO on the Next Generation of AI and MLOps Mar 03, 2022

Show notes

Jensen Huang is founder and CEO of NVIDIA, whose GPUs sit at the heart of the majority of machine learning models today.

Jensen shares the story behind NVIDIA's expansion from gaming to deep learning acceleration, leadership lessons that he's learned over the last few decades, and why we need a virtual world that obeys the laws of physics (aka the Omniverse) in order to take AI to the next era. Jensen and Lukas also talk about the singularity, the slow-but-steady approach to building a new market, and the importance of MLOps.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-jensen-huang

---

⏳ Timestamps:

0:00 Intro

0:50 Why NVIDIA moved into the deep learning space

7:33 Balancing the compute needs of different audiences

10:40 Quantum computing, Huang's Law, and the singularity

15:53 Democratizing scientific computing

20:59 How Jensen stays current with technology trends

25:10 The global chip shortage

27:00 Leadership lessons that Jensen has learned

32:32 Keeping a steady vision for NVIDIA

35:48 Omniverse and the next era of AI

42:00 ML topics that Jensen's excited about

45:05 Why MLOps is vital

48:38 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Peter & Boris — Fine-tuning OpenAI's GPT-3 Feb 10, 2022

Show notes

Peter Welinder is VP of Product & Partnerships at OpenAI, where he runs product and commercialization efforts of GPT-3, Codex, GitHub Copilot, and more. Boris Dayma is Machine Learning Engineer at Weights & Biases, and works on integrations and large model training.

Peter, Boris, and Lukas dive into the world of GPT-3:

- How people are applying GPT-3 to translation, copywriting, and other commercial tasks

- The performance benefits of fine-tuning GPT-3-

- Developing an API on top of GPT-3 that works out of the box, but is also flexible and customizable

They also discuss the new OpenAI and Weights & Biases collaboration, which enables a user to log their GPT-3 fine-tuning projects to W&B with a single line of code.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-peter-and-boris

---

Connect with Peter & Boris:

📍 Peter's Twitter: https://twitter.com/npew

📍 Boris' Twitter: https://twitter.com/borisdayma

---

⏳ Timestamps:

0:00 Intro

1:01 Solving real-world problems with GPT-3

6:57 Applying GPT-3 to translation tasks

14:58 Copywriting and other commercial GPT-3 applications

20:22 The OpenAI API and fine-tuning GPT-3

28:22 Logging GPT-3 fine-tuning projects to W&B

38:25 Engineering challenges behind OpenAI's API

43:15 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Ion Stoica — Spark, Ray, and Enterprise Open Source Jan 20, 2022

Show notes

Ion Stoica is co-creator of the distributed computing frameworks Spark and Ray, and co-founder and Executive Chairman of Databricks and Anyscale. He is also a Professor of computer science at UC Berkeley and Principal Investigator of RISELab, a five-year research lab that develops technology for low-latency, intelligent decisions.

Ion and Lukas chat about the challenges of making a simple (but good!) distributed framework, the similarities and differences between developing Spark and Ray, and how Spark and Ray led to the formation of Databricks and Anyscale. Ion also reflects on the early startup days, from deciding to commercialize to picking co-founders, and shares advice on building a successful company.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-ion-stoica

---

Timestamps:

0:00 Intro

0:56 Ray, Anyscale, and making a distributed framework

11:39 How Spark informed the development of Ray

18:53 The story behind Spark and Databricks

33:00 Why TensorFlow and PyTorch haven't monetized

35:35 Picking co-founders and other startup advice

46:04 The early signs of sky computing

49:24 Breaking problems down and prioritizing

53:17 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts

👉 Google Podcasts: http://wandb.me/google-podcasts

👉 Spotify: http://wandb.me/spotify

Stephan Fabel — Efficient Supercomputing with NVIDIA's Base Command Platform Jan 06, 2022

Show notes

Stephan Fabel is Senior Director of Infrastructure Systems & Software at NVIDIA, where he works on Base Command, a software platform to coordinate access to NVIDIA's DGX SuperPOD infrastructure.

Lukas and Stephan talk about why having a supercomputer is one thing but using it effectively is another, why a deeper understanding of hardware on the practitioner level is becoming more advantageous, and which areas of the ML tech stack NVIDIA is looking to expand into.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-stephan-fabel

---

Timestamps:

0:00 Intro

1:09 NVIDIA Base Command and DGX SuperPOD

10:33 The challenges of multi-node processing at scale

18:35 Why it's hard to use a supercomputer effectively

25:14 The advantages of de-abstracting hardware

29:09 Understanding Base Command's product-market fit

36:59 Data center infrastructure as a value center

42:13 Base Command's role in tech stacks

47:16 Why crowdsourcing is underrated