The Gradient: Perspectives on AI

Latest Episodes:

Rishi Bommasani on Foundation Models Dec 09, 2021

Show notes

In episode 19 of The Gradient Podcast, we talk to Rishi Bommasani, a Ph.D student at Stanford focused on Foundation Models.

Rish is a second-year Ph.D. student in the CS Department at Stanford, where he is advised by Percy Liang and Dan Jurafsky. His research focuses on understanding AI systems and their social impact, as well as using NLP to further scientific inquiry. Over the past year, he helped build and organize the Stanford Center for Research on Foundation Models (CRFM).

Sections:

(00:00:00) Intro(00:01:05) How did you get into AI?(00:09:55) Towards Understanding Position Embeddings(00:14:23) Long-Distance Dependencies don’t have to be Long(00:18:55) Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings(00:30:25) Masters Thesis(00:34:05) Start of PhD and work on foundation models(00:42:14) Why were people intested in foundation models(00:46:45) Formation of CRFM(00:51:25) Writing report on foundation models(00:56:33) Challenges in writing report(01:05:45) Response to reception(01:15:35) Goals of CRFM(01:25:43) Current research focus(01:30:35) Interests outside of research(01:33:10) Outro

Papers discussed:

* Towards Understanding Position Embeddings

* Long-Distance Dependencies don’t have to be Long: Simplifying through Provably (Approximately) Optimal Permutations

* Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings

* Generalized Optimal Linear Orders

* On the Opportunities and Risks of Foundation Models

* Reflections on Foundation Models

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Upol Ehsan on Human-Centered Explainable AI and Social Transparency Dec 03, 2021

Show notes

In episode 18 of The Gradient Podcast, we talked to Upol Ehsan, an Explainable AI (XAI) researcher who combines his background in Philosophy and Human-Computer Interaction to address problems in XAI beyond just opening the "black-box" of AI. You can find his Gradient article charting this vision here.

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Papers Discussed:

* Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations

* Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions

* Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach

* Expanding Explainability: Towards Social Transparency in AI systems

* The Who in Explainable AI: How AI Background Shapes Perceptions of AI Explanations

* Explainability Pitfalls: Beyond Dark Patterns in Explainable AI

Exciting update!

In addition to listening to the audio recording, you can now experience the interview over at The Gradient’s main site, with live captions and the ability to jump to certain sections.

In addition, you can experience it as follows: Interactive Transcript | Transcript PDF | Interview on YouTube

About Upol:Upol Ehsan cares about people first, technology second. He is a doctoral candidate in the School of Interactive Computing at Georgia Tech and an affiliate at the Data & Society Research Institute. Combining his expertise in AI and background in Philosophy, his work in Explainable AI (XAI) aims to foster a future where anyone, regardless of their background, can use AI-powered technology with dignity.

Actively publishing in top peer-reviewed venues like CHI, his work has received multiple awards and been covered in major media outlets. Bridging industry and academia, he serves on multiple program committees in HCI and AI conferences (e.g., DIS, IUI, NeurIPS) and actively connects these communities (e.g, the widely attended HCXAI workshop at CHI). By promoting equity and ethics in AI, he wants to ensure stakeholders who aren’t at the table do not end up on the menu. Outside research, he is an advisor for Aalor Asha, an educational institute he started for underprivileged children subjected to child labor.Follow him on Twitter: @upolehsan

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Miles Brundage on AI Misuse and Trustworthy AI Nov 23, 2021

Show notes

In episode 17 of The Gradient Podcast, we talk to Miles Brundage, Head of Policy Research at OpenAI and a researcher passionate about the responsible governance of artificial intelligence.

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Links:

* Will Technology Make Work Better for Everyone?

* Economic Possibilities for Our Children: Artificial Intelligence and the Future of Work, Education, and Leisure

* Taking Superintelligence Seriously

* The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

* Release Strategies and the Social Impact of Language Models

* All the News that’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation

* Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Timeline:

(00:00) Intro(01:05) How did you get started in AI(07:05) Writing about AI on Slate(09:20) Start of PhD(13:00) AI and the End of Scarcity(18:12) Malicious Uses of AI(28:00) GPT-2 and Publication Norms(33:30) AI-Generated Text for Misinformation(37:05) State of AI Misinformation(41:30) Trustworthy AI(48:50) OpenAI Policy Research Team(53:15) Outro

Miles is a researcher and research manager, and is passionate about the responsible governance of artificial intelligence. In 2018, he joined OpenAI, where he began as a Research Scientist and recently became Head of Policy Research. Before that, he was a Research Fellow at the University of Oxford's Future of Humanity Institute, where he is still a Research Affiliate).He also serves as a member of Axon's AI and Policing Technology Ethics Board. He completed a PhD in Human and Social Dimensions of Science and Technology from Arizona State University in 2019.

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music"

Hosted by Andrey Kurenkov (@andrey_kurenkov), a PhD student with the Stanford Vision and Learning Lab working on learning techniques for robotic manipulation and search.

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Jeffrey Ding on China's AI Dream, the AI 'Arms Race', and AI as a General Purpose Technology Nov 18, 2021

Show notes

In episode 16 of The Gradient Podcast, we talk to Jeffrey Ding, a postdoctoral fellow at Stanford's Center for International Security and Cooperation

(01:35) Getting into AI research(04:20) Interest in studying China(06:50) Deciphering China’s AI Dream(23:25) Beyond the AI Arms Race(36:45) China's Current Capabilities in AI(46:45) AI as a General Purpose and Strategic Technology(57:38) ChinaAI Newsletter(01:04:20) Teaching AI to Policy People(01:06:30) Current Focus(01:09:10) Interests Outside of Work + Outro

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Jeffrey Ding (@jjding99) is a postdoctoral fellow at Stanford's Center for International Security and Cooperation, sponsored by Stanford's Institute for Human-Centered Artificial Intelligence, as well as a research affiliate with the Centre for the Governance of AI at the University of Oxford. His current research is centered on how technological change affects the rise and fall of great powers, with an eye toward the implications of advances in AI for a possible U.S.-China power transition. He also puts out the excellent ChinaAI newsletter, which has (sometimes) weekly translations of Chinese-language musings on AI and related topics.

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music"

Hosted by Andrey Kurenkov (@andrey_kurenkov), a PhD student with the Stanford Vision and Learning Lab working on learning techniques for robotic manipulation and search.

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Alex Tamkin on Self-Supervised Learning and Large Language Models Nov 11, 2021

Show notes

In episode 15 of The Gradient Podcast, we talk to Stanford PhD Candidate Alex Tamkin

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Alex Tamkin is a fourth-year PhD student in Computer Science at Stanford, advised by Noah Goodman and part of the Stanford NLP Group. His research focuses on understanding, building, and controlling pretrained models, especially in domain-general or multimodal settings.

We discuss:

* Viewmaker Networks: Learning Views for Unsupervised Representation Learning

* DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning

* On the Opportunities and Risks of Foundation Models

* Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

* Mentoring, teaching and fostering a healthy and inclusive research culture

* Scientific communication and breaking down walls between fields

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music"

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Peter Henderson on RL Benchmarking, Climate Impacts of AI, and AI for Law Oct 28, 2021

Show notes

In episode 14 of The Gradient Podcast, we interview Stanford PhD Candidate Peter Henderson

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Peter is a joint JD-PhD student at Stanford University advised by Dan Jurafsky. He is also an OpenPhilanthropy AI Fellow and a Graduate Student Fellow at the Regulation, Evaluation, and Governance Lab. His research focuses on creating robust decision-making systems, with three main goals: (1) use AI to make governments more efficient and fair; (2) ensure that AI isn’t deployed in ways that can harm people; (3) create new ML methods for applications that are beneficial to society.

Links:

* Reproducibility and Reusability in Deep Reinforcement Learning.

* Benchmark Environments for Multitask Learning in Continuous Domains

* Reproducibility of Bench-marked Deep Reinforcement Learning Tasks for Continuous Control.

* Deep Reinforcement Learning that Matters

* Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods)

* Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning

* How blockers can turn into a paper: A retrospective on 'Towards The Systematic Reporting of the Energy and Carbon Footprints of Machine Learning

* When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset”

* How US law will evaluate artificial intelligence for Covid-19

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music"

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Chelsea Finn on Meta Learning & Model Based Reinforcement Learning Oct 14, 2021

Show notes

In episode 13 of The Gradient Podcast, we interview Stanford Professor Chelsea Finn

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Chelsea is an Assistant Professor at Stanford University. Her lab, IRIS, studies intelligence through robotic interaction at scale, and is affiliated with SAIL and the Statistical ML Group. I also spend time at Google as a part of the Google Brain team. Her research deals with the capability of robots and other agents to develop broadly intelligent behavior through learning and interaction.

Links:

* Learning to Learn with Gradients

* Visual Model-Based Reinforcement Learning as a Path towards Generalist Robots

* RoboNet: A Dataset for Large-Scale Multi-Robot Learning

* Greedy Hierarchical Variational Autoencoders for Large-Scale Video

* Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music".

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Devi Parikh on Generative Art & AI for Creativity Oct 01, 2021

Show notes

In episode 12 of The Gradient Podcast, we interview Devi Parikh, a professor at Georgia Tech whose research focuses on computer vision, natural language processing, embodied AI, human-AI collaboration, and AI for creativity.

Devi Parikh is an Associate Professor in the School of Interactive Computing at Georgia Tech, and a Research Scientist at Facebook AI Research (FAIR). Her research interests are in computer vision, natural language processing, embodied AI, human-AI collaboration, and AI for creativity. In the past, she has also been an Assistant Professor at Virginia Tech and a Research Assistant Professor at Toyota Technological Institute at Chicago (TTIC). She received her M.S. and Ph.D. degrees from the Electrical and Computer Engineering department at Carnegie Mellon University in 2007 and 2009 respectively.

Links:

* Humans of AI Podcast

* Feel The Music: Automatically Generating A Dance For An Input Song

* Exploring Crowd Co-creation Scenarios for Sketches

* Neuro-Symbolic Generative Art: A Preliminary Study

* Creative Sketch Generation

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music".

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Sergey Levine on Robot Learning & Offline RL Sep 16, 2021

Show notes

In episode 11 of The Gradient Podcast, we interview Sergey Levine, a professor at Berkeley whose research focuses on machine learning for decision making and control, with an emphasis on deep learning and reinforcement learning algorithms for robotics.

Sergey Levine received a BS and MS in Computer Science from Stanford University in 2009, and a Ph.D. in Computer Science from Stanford University in 2014. He joined the faculty of the Department of Electrical Engineering and Computer Sciences at UC Berkeley in fall 2016. His work focuses on machine learning for decision making and control, with an emphasis on deep learning and reinforcement learning algorithms, and includes developing algorithms for end-to-end training of deep neural network policies that combine perception and control, scalable algorithms for inverse reinforcement learning, deep reinforcement learning algorithms, and more.

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music".

Get full access to The Gradient at thegradientpub.substack.com/subscribe

Jeremy Howard on Kaggle, Enlitic, and fast.ai Sep 09, 2021

Show notes

In episode 10 of The Gradient Podcast, we interview data scientist, researcher, developer, educator, and entrepreneur Jeremy Howard.

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS

Jeremy Howard is a data scientist, researcher, developer, educator, and entrepreneur. Jeremy is a founding researcher at fast.ai, a research institute dedicated to making deep learning more accessible. He is also a Distinguished Research Scientist at the University of San Francisco, the chair of WAMRI, and is Chief Scientist at platform.ai. Previously, Jeremy was the founding CEO Enlitic, which was the first company to apply deep learning to medicine, was the President and Chief Scientist of the data science platform Kaggle, and was the founding CEO of two successful Australian startups.

Podcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music".

Get full access to The Gradient at thegradientpub.substack.com/subscribe