The Inside View

Latest Episodes:

Blake Richards–AGI Does Not Exist Jun 14, 2022

Show notes

Blake Richards is an Assistant Professor in the Montreal Neurological Institute and the School of Computer Science at McGill University and a Core Faculty Member at MiLA. He thinks that AGI is not a coherent concept, which is why he ended up on a recent AGI political compass meme. When people asked on Twitter who was the edgiest people at MiLA, his name got actually more likes than Ethan, so hopefully, this podcast will help re-establish the truth.

Transcript: https://theinsideview.ai/blake

Video: https://youtu.be/kWsHS7tXjSU

Outline:

(01:03) Highlights

(01:03) AGI good / AGI not now compass

(02:25) AGI is not a coherent concept

(05:30) you cannot build truly general AI

(14:30) no "intelligence" threshold for AI

(25:24) benchmarking intelligence

(28:34) recursive self-improvement

(34:47) scale is something you need

(37:20) the bitter lesson is only half-true

(41:32) human-like sensors for general agents

(44:06) the credit assignment problem

(49:50) testing for backpropagation in the brain

(54:42) burstprop (bursts of action potentials), reward prediction errors

(01:01:35) long-term credit-assignment in reinforcement learning

(01:10:48) what would change his mind on scaling and existential risk

Ethan Caballero–Scale is All You Need May 05, 2022

Show notes

Ethan is known on Twitter as the edgiest person at MILA. We discuss all the gossips around scaling large language models in what will be later known as the Edward Snowden moment of Deep Learning. On his free time, Ethan is a Master’s degree student at MILA in Montreal, and has published papers on out of distribution generalization and robustness generalization, accepted both as oral presentations and spotlight presentations at ICML and NeurIPS. Ethan has recently been thinking about scaling laws, both as an organizer and speaker for the 1st Neural Scaling Laws Workshop.

Transcript: https://theinsideview.github.io/ethan

Youtube: https://youtu.be/UPlv-lFWITI

Michaël: https://twitter.com/MichaelTrazzi

Ethan: https://twitter.com/ethancaballero

Outline

(00:00) highlights

(00:50) who is Ethan, scaling laws T-shirts

(02:30) scaling, upstream, downstream, alignment and AGI

(05:58) AI timelines, AlphaCode, Math scaling, PaLM

(07:56) Chinchilla scaling laws

(11:22) limits of scaling, Copilot, generative coding, code data

(15:50) Youtube scaling laws, constrative type thing

(20:55) AGI race, funding, supercomputers

(24:00) Scaling at Google

(25:10) gossips, private research, GPT-4

(27:40) why Ethan was did not update on PaLM, hardware bottleneck

(29:56) the fastest path, the best funding model for supercomputers

(31:14) EA, OpenAI, Anthropics, publishing research, GPT-4

(33:45) a zillion language model startups from ex-Googlers

(38:07) Ethan's journey in scaling, early days

(40:08) making progress on an academic budget, scaling laws research

(41:22) all alignment is inverse scaling problems

(45:16) predicting scaling laws, useful ai alignment research

(47:16) nitpicks aobut Ajeya Cotra's report, compute trends

(50:45) optimism, conclusion on alignment

10. Peter Wildeford on Forecasting Apr 13, 2022

Show notes

Peter is the co-CEO of Rethink Priorities, a fast-growing non-profit doing research on how to improve the long-term future. On his free time, Peter makes money in prediction markets and is quickly becoming one of the top forecasters on Metaculus. We talk about the probability of London getting nuked, Rethink Priorities and why EA should fund projects that scale.

Check out the video and transcript here: https://theinsideview.github.io/peter

9. Emil Wallner on Building a €25000 Machine Learning Rig Mar 23, 2022

Show notes

Emil is a resident at the Google Arts & Culture Lab were he explores the intersection between art and machine learning. He recently built his own Machine Learning server, or rig, which costed him €25000.

Emil's Story: https://www.emilwallner.com/p/ml-rig

Youtube: https://youtu.be/njbPpxhE6W0

00:00 Intro

00:23 Building your own rig

06:11 The Nvidia GPU rder hack

15:51 Inside Emil's rig

21:31 Motherboard

23:55 Cooling and datacenters

29:36 Deep Learning lessons from owning your hardware

36:20 Shared resources vs. personal GPUs

39:12 RAM, chassis and airflow

42:42 Amd, Apple, Arm and Nvidia

51:15 Tensorflow, TPUs, cloud minsdet, EleutherAI

8. Sonia Joseph on NFTs, Web 3 and AI Safety Dec 22, 2021

Show notes

Sonia is a graduate student applying ML to neuroscience at MILA. She was previously applying deep learning to neural data at Janelia, an NLP research engineer at a startup and graduated in computational neuroscience at Princeton University.

Anonymous feedback: https://app.suggestionox.com/r/xOmqTW

Twitter: https://twitter.com/MichaelTrazzi

Sonia's December update: https://t.co/z0GRqDTnWm

Sonia's Twitter: https://twitter.com/soniajoseph_

Orthogonality Thesis: https://www.youtube.com/watch?v=hEUO6pjwFOo

Paperclip game: https://www.decisionproblem.com/paperclips/

Ngo & Yudkowsky on feedback loops: https://bit.ly/3ml0zFL

Outline

00:00 Intro

01:06 NFTs

03:38 Web 3

21:12 Digital Copy

29:09 ML x Neuroscience

43:44 Limits of the Orthogonality Thesis

01:01:25 Goal of perpetuating Information

01:08:14 Compressing information

01:10:52 Feedback loops are not safe

01:17:43 Another AI Safety aesthetic

01:23:46 Meaning of life

7. Phil Trammell on Economic Growth under Transformative AI Oct 24, 2021

Show notes

Phil Trammell is an Oxford PhD student in economics and research associate at the Global Priorities Institute. Phil is one of the smartest person I know, when considering the intersection of the long-term future and economic growth. Funnily enough, Phil was my roomate, a few years ago in Oxford, and last time I called him he casually said that he had written an extensive report on the econ of AI. A few weeks ago, I decided that I would read that report (which actually is a literature review), and that I would translate everything that I learn along the way to diagrams, so you too can learn what’s inside that paper. The video covers everything from MacroEconomics 101 to self-improving AI in about 30-ish diagrams.

Outline:

- 00:00 Podcast intro

- 01:19 Phil's intro

- 08:58 What's GDP

- 13:42 Decreasing growth

- 15:40 Permanent growth increase

- 19:02 Singularity of type I

- 22:58 Singularity of type II

- 23:24 Production function

- 24:10 The Economy as a two-tubes factory

- 25:09 Marginal Products of labor/capital

- 27:48 Labor/capital-augmenting technology

- 29:13 Technological progress since Ford

- 38:18 Factor payments

- 41:30 Elasticity of substitution

- 48:34 Production function with substitution

- 53:18 Perfect substitutability

- 54:00 Perfect complements

- 55:44 Exogenous growth

- 59:56 How to get long-run growth

- 01:05:40 Endogenous growth

- 01:10:40 The research feedback parameter

- 01:17:35 AI as an imperfect substitute for human labor

- 01:25:25 A simple model for perfect substitution

- 01:33:09 AI as a perfect substitute

- 01:36:07 Substitutability in robotics production

- 01:40:43 OpenAI automating coding

- 01:44:38 Growth impacts via impacts on savings

- 01:46:44 AI in task-based models of good productions

- 01:53:26 AI in technology production

- 02:03:55 Limits of the econ model

- 02:09:00 Conclusion

6. Slava Bobrov on Brain Computer Interfaces Oct 06, 2021

Show notes

In this episode I discuss Brain Computer Interfaces with Slava Bobrov, a self-taught Machine Learning Engineer applying AI to neural biosignals to control robotic limbs. This episode will be of special interest to you if you're an engineer who wants to get started with brain computer interfaces, or just broadly interested in how this technology could enhance human intelligence. Fun fact: most of the questions I asked were sent by my Twitter followers, or come from a Discord I co-created on Brain Computer Interfaces. So if you want your questions to be on the next video or you're genuinely interested in this topic, you can find links for both my Twitter and our BCI discord in the description.

Outline:

00:00 introduction
00:49 defining brain computer interfaces (BCI)
03:35 Slava's work on prosthetic hands
09:16 different kinds of BCI
11:42 BCI companies: Muse, Open BCI
16:26 what Kernel is doing (fNIRS)
20:24 EEG vs. EMG—the stadium metaphor
25:26 can we build "safe" BCIs?
29:32 would you want a Facebook BCI?
33:40 OpenAI Codex is a BCI
38:04 reward prediction in the brain
44:04 what Machine Learning project for BCI?
48:27 Slava's sleep tracking
51:55 patterns in recorded sleep signal
54:56 lucid dreaming
56:51 the long-term future of BCI
59:57 are they diminishing returns in BCI/AI investments?
01:03:45 heterogeneity in intelligence after BCI/AI progress
01:06:30 is our communication improving? is BCI progress fast enough?
01:12:30 neuroplasticity, Neuralink
01:16:08 siamese twins with BCI, the joystick without screen experiment
01:20:50 Slava's vision for a "brain swarm"
01:23:23 language becoming obsolete, Twitter swarm
01:26:16 brain uploads vs. copies
01:29:32 would a copy be actually you?
01:31:30 would copies be a success for humanity?
01:34:38 shouldn't we change humanity's reward function?
01:37:54 conclusion

5. Charlie Snell on DALL-E and CLIP Sep 16, 2021

Show notes

We talk about AI generated art with Charlie Snell, a Berkeley student who wrote extensively about AI art for ML@Berkeley's blog (https://ml.berkeley.edu/blog/). We look at multiple slides with art throughout our conversation, so I highly recommend watching the video (https://www.youtube.com/watch?v=gcwidpxeAHI).

In the first part we go through Charlie's explanations of DALL-E, a model trained end-to-end by OpenAI to generate images from prompts. We then talk about CLIP + VQGAN, where CLIP is another model by OpenAI matching prompts and images, and VQGAN is a state-of-the art GAN used extensively in the AI Art scene. At the end of the video we look at different pieces of art made using CLIP, including tricks for using VQGAN with CLIP, videos, and the latest CLIP-guided diffusion architecture. At the end of our chat we talk about scaling laws and how progress in art relates to other advances in ML.

4. Sav Sidorov on Learning, Contrarianism and Robotics Sep 05, 2021

Show notes

I interview Sav Sidorov about top-down learning, contrarianism, religion, university, robotics, ego , education, twitter, friends, psychedelics, B-values and beauty.

Highlights & Transcript: https://insideview.substack.com/p/sav

Watch the video: https://youtu.be/_Y6_TakG3d0

3. Evan Hubinger on Takeoff speeds, Risks from learned optimization & Interpretability Jun 08, 2021

Show notes

We talk about Evan’s background @ MIRI & OpenAI, Coconut, homogeneity in AI takeoff, reproducing SoTA & openness in multipolar scenarios, quantilizers & operationalizing strategy stealing, Risks from learned optimization & evolution, learned optimization in Machine Learning, clarifying Inner AI Alignment terminology, transparency & interpretability, 11 proposals for safe advanced AI, underappreciated problems in AI Alignment & surprising advances in AI.

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected