Scaling Theory

Latest Episodes:

#10 – Allison Stanger: Political Science Behind Large Tech Companies Sep 26, 2024

My guest today is Allison Stanger. Allison is a Middlebury Distinguished Endowed Professor; an Affiliate at the Berkman Klein Center for Internet and Society, Harvard University; the Co-Director (with Danielle Allen) of the GETTING-Plurality⁠ Research Network, Harvard University; founding member of the Digital Humanism Initiative (Vienna); and an External Professor at the Santa Fe Institute. Allison’s next book, Who Elected Big Tech? is under contract with Yale University Press.

In this conversation, Allison and I delve into the political science surrounding large tech companies. We explore their effects on consumers and democracy, the interplay between capitalism and democracy, the dangers of fragmented regulation, what the effective governance of social media entails, how to scale and measure it, potential areas of cooperation with China, and the relevance of public choice theory, complexity science, and power laws in shaping our understanding of technology. I hope you enjoy our discussion.

***

References

Stanger, Allison. "The Real Cost of Surveillance Capitalism: Digital Humanism in the United States and Europe." Perspectives on Digital Humanism (2022): 33-40. https://library.oapen.org/bitstream/handle/20.500.12657/51945/978-3-030-86144-5.pdf

Werthner, Hannes, et al. "Digital humanism: The time is now." Computer 56.1 (2023): 138-142. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10008968
Soros, George. "Fallibility, reflexivity, and the human uncertainty principle." Journal of Economic Methodology 20.4 (2013): 309-329. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10008968

#9 – Arvind Narayanan: Myths and Policies in Scaling AI Aug 26, 2024

My guest is Arvind Narayanan, a Professor of Computer Science at Princeton University, and the director of the Center for Information Technology Policy, also at Princeton. Arvind is renowned for his work on the societal impacts of digital technologies, including his textbook on fairness and machine learning, his online course on cryptocurrencies, his research on data de-anonymization, dark patterns, and more. He has already amassed over 30,000 citations on Google Scholar.

In just a few days, in late September 2024, Arvind will release a book co-authored with Sayash Kapoor titled “AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference.” Having had the privilege of reading an early version, our conversation delves into some of the book’s key arguments. We also explore what Arvind calls AI scaling myths, the reality of artificial general intelligence, how governments can scale effective AI policies, the importance of transparency, the role that antitrust can, and cannot play, the societal impacts of scaling automation, and more. I hope you enjoy our conversation.

Find me on X at @⁠⁠⁠ProfSchrepel⁠⁠⁠⁠⁠⁠. Also, be sure to subscribe.

References:

➝ AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (2024)

➝ AI scaling myths (2024)

➝ AI existential risk probabilities are too unreliable to inform policy (2024)

➝ Foundation Model Transparency Reports (2024)

#8 – Sara Hooker: Big AI, The Compute Frenzy, and Grumpy Models Aug 05, 2024

My guest today is ⁠Sara Hooker⁠, VP of Research at Cohere, where she leads Cohere for AI, a non-profit research lab that seeks to solve complex machine learning problems with researchers from over 100 countries. Sara is the author of numerous research papers, some of which focus specifically on scaling theory in AI. She has been listed as one of AI’s top 13 innovators by Fortune.

In our conversation, we first delve into the scaling laws behind foundation models. We explore what powers the scaling of AI systems and the limits to scaling laws. We then move on to discussing openness in AI, Cohere’s business strategy, the power of ecosystems, the importance of building multilingual LLMs, and the recent change in terms of access to data in the space. I hope you enjoy our conversation.

Find me on X at @⁠⁠ProfSchrepel⁠⁠⁠⁠⁠. Also, be sure to subscribe.

References:

➝ Sara Hooker, On the Limitations of Compute Thresholds as a Governance Strategy (2024)

➝ Sara Hooker, The Hardware Lottery (2020)

➝ Sara Hooker, Moving beyond “algorithmic bias is a data problem” (2021) ➝ Longpre et al., Consent in Crisis: The Rapid Decline of the AI Data Commons (2024)

#7 – Michael Mauboussin: The Fascinating World of Increasing Returns Jul 15, 2024

My guest today is Michael Mauboussin (@mjmauboussin), one of the world’s leading experts in finance. Michael serves as Head of Consilient Research at Counterpoint Global, Morgan Stanley. He has authored three books and regularly appears in the Wall Street Journal, Financial Times, New York Times, and other publications. Since 1993, Michael has been an adjunct professor of finance at Columbia Business School and is also the chairman emeritus of the board of trustees at the Santa Fe Institute.

In our conversation, we delve into the dynamics of markets, discuss all sorts of increasing returns, and explore topics such as Charles Darwin, policymaking, AI and Web3, and the Santa Fe Institute. I hope you enjoy our discussion.

Find me on X at @⁠ProfSchrepel⁠⁠⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast.

References:

Michael J. Mauboussin & Dan Callahan, "Increasing Returns: Identifying Forms of Increasing Returns and What Drives Them" (2024) https://perma.cc/Y3DN-LNMY
Michael J. Mauboussin & Dan Callahan, "Stock Market Concentration: How Much Is Too Much?" (2024) https://perma.cc/7EEX-ZY9T
Charles Darwin, The Autobiography of Charles Darwin: 1809-1882 https://www.amazon.com/Autobiography-Charles-Darwin-1809-1882/dp/0393310698
David Warsh, Knowledge and the Wealth of Nations: A Story of Economic Discovery (2007) https://www.amazon.com/Knowledge-Wealth-Nations-Economic-Discovery/dp/0393329887
James Bessen, The New Goliaths: How Corporations Use Software to Dominate Industries, Kill Innovation, and Undermine Regulation (2022) https://www.amazon.nl/-/en/James-Bessen/dp/0300255047
Chris Dixon, Read Write Own: Building the Next Era of the Internet (2023) https://readwriteown.com
Anu Bradford, Digital Empires: The Global Battle to Regulate Technology (2023) https://global.oup.com/academic/product/digital-empires-9780197649268
Kenneth J. Arrow, "The Economic Implications of Learning by Doing" (1962) https://www.jstor.org/stable/2295952
J. Doyne Farmer, Making Sense of Chaos (2024) https://www.penguin.co.uk/books/284357/making-sense-of-chaos-by-farmer-j-doyne/9780241201978

#6 – Glen Weyl: Scaling Plurality Jun 20, 2024

My guest is Glen Weyl, an influential economist and social technologist known for his interdisciplinary work at the intersection of economics, computer science, sociology, and political science. He is a Principal Researcher at Microsoft Research, a co-author of the books “Radical Markets” and the recently published “Plurality: The Future of Collaborative Technology and Democracy” which he co-authored with Audrey Tang, who has served as the 1st Minister of Digital Affairs of Taiwan. Glen co-founded the RadicalxChange Foundation, the Plural Technology Collaboratory, and the Plurality Institute. He taught economics at the University of Chicago, Princeton, and Yale.

Glen’s work frequently explores the potential of technology and market mechanisms to drive social change and address systemic inequalities. In his new book, Glen lays the foundation for a new societal vision: plurality.

Our conversation is structured into three distinct parts. We begin by situating the concept of plurality in today’s philosophical and technological landscape. We then discuss the technologies that will enable plurality to materialize, and finally, we conclude with a discussion of the challenges ahead. By the end of this discussion, you will know what plurality is, how different it is from the vision of today’s tech CEOs, the role open source AI and blockchain can play to scale plurality, but also the role legal rules and standards are playing, which of the scaling theory concepts (including fractals, fragility, robustness, chaos, etc.) helped Glen and Audrey define their strategy to take over other visions, and more. I hope you enjoy the conversation.

Find me on X at @ProfSchrepel⁠⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

#5 – Yann LeCun: AI Dynamics and Regulation May 27, 2024

My guest is Yann LeCun, a pioneering French-American computer scientist, known for his groundbreaking work in machine learning, computer vision, and neural networks. Yann is the Silver Professor at the Courant Institute of Mathematical Sciences at New York University and serves as the Vice President and Chief AI Scientist at Meta.

Yann is one of the world’s most influential computer scientists. He has accumulated over 350,000 citations on Google Scholar, he is one of the founding figures in the field of deep learning thanks to its contribution to convolutional neural networks and backpropagation algorithms, and he is a vocal proponent of open source. In recognition of his significant contributions to artificial intelligence, he was awarded the Turing Award in 2018, often referred to as the “Nobel Prize of Computing.”

Our conversation is structured into three distinct parts. We begin by discussing the overarching dynamics in the AI space, then narrow our focus to the firm level, and finally, we conclude with an exploration of the challenges that lie ahead. By the end of this discussion, you will learn whether open source has a chance to make it in AI, the key factors for scaling an AI foundation model, the role ecosystems play in market dynamics, Meta long term strategy in the space, how concentration among chip manufacturers impacts AI companies, the current effect of the European AI Act on AI companies, what Yann would like to see regulators doing, and more. I hope you enjoy the conversation.

#4 – Doyne Farmer: Chaos Theory & Complexity Economics May 06, 2024

J. Doyne Farmer is the Director of the Complexity Economics programme at the Institute for New Economic Thinking at the Oxford Martin School, a Professor at the Mathematical Institute of Oxford University, and an External Professor at the Santa Fe Institute.

In this episode, we explore Doyne’s latest book, “Making Sense of Chaos.” We focus on the relationship between chaos and scaling theory, and more specifically, how chaos can be factored into scaling theory. By the end of this conversation, you will learn why it might be easier to predict the long distant future than predicting tomorrow, how Moore’s Law conflicts with other scaling laws that underpin technological progress, how agent-based modeling can help all scientists and policymakers, how to dominate the world with your theories (...), and even how to trick casinos. I hope you enjoy the conversation.

Find me on X at @⁠⁠ProfSchrepel⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

#3 – Thomas Wolf: Open-Source AI Scaling Laws Apr 15, 2024

Thomas Wolf is the co-founder and Chief Science Officer of Hugging Face, the company at the center of the open-source AI ecosystem. He has a Ph.D. in statistical & quantum physics.

In this episode, we explore why open-source (“OS”) AI may be preferable to closed-source, whether OS has a real chance to take over the space, the challenges OS developers must overcome to scale their foundational models, the role of data and infrastructure in scaling dynamics, how big tech companies are positioning themselves in the ecosystem and responding to other companies' strategies in a true complexity science fashion, and overall, the power laws that undermine the growth of the ecosystem. I hope you enjoy the conversation. I hope you enjoy the conversation.

Find me on X at @⁠ProfSchrepel⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

#2 – Geoffrey West: The Scaling Laws Behind Living Organisms, Cities, Businesses, and Technologies Mar 25, 2024

Geoffrey West is a physicist, former president and distinguished professor of the Santa Fe Institute. His book, “Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Businesses” (2017), is a masterpiece.

In this episode, we talk about the power laws behind living organisms, cities, businesses, and technologies. By the end of this episode, you will know more about the power law behind the heartbeat of all mammals, the number of patents and crime in big cities compared to small cities, innovation, the way technology scales, and more. I hope you enjoy the conversation.

Find me on X at @ProfSchrepel. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

References

Geoffrey West, ⁠Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Businesses⁠ (Penguin Books, 2017)
George J. Stigler, “The Economies of Scale”, The Journal of Law & Economics 1 (1958): 54–71
Michael HR Stanley, et al. “⁠Scaling Behaviour in the Growth of Companies”, Nature 379.6568 (1996): 804-806.APA
W. Brian. Arthur, “⁠Competing Technologies, Increasing Returns, and Lock-In by Historical Events”, The Economic Journal 99.394 (1989): 116-131.
Madeleine IG Daepp, et al. “⁠⁠⁠⁠⁠⁠⁠⁠⁠The Mortality of Companies⁠”, Journal of the Royal Society Interface 12.106 (2015): 20150120.APA
Jiang Zhang, et al. “⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Scaling Laws and a General Theory for the Growth of Public Companies ⁠⁠”, arXiv preprint arXiv:2109.10379 (2021)

Why “Scaling Theory” Mar 16, 2024

In this first episode, Dr. Thibault Schrepel (@⁠ProfSchrepel⁠) introduces “Scaling Theory”, a podcast dedicated to the power laws behind the growth of companies, technologies, legal and living systems.

***

References:

➝ Charles Darwin, On the Origin of Species (1859)

➝ Melanie Mitchell, Complexity: A Guided Tour (2011)

➝ Mitchell Waldrop, Complexity The Emerging Science at the Edge of Order and Chaos (2019)

➝ John H. Miller & Scott Page, Complex Adaptive Systems: An Introduction to Computational Models of Social Life (2007)

➝ W. Brian Arthur, Complexity and the Economy (2014)

➝ Geoffrey West, Scale: The Universal Laws of Life and Death in Organisms, Cities and Companies (2017)