Scaling Theory

*/ ?>

Latest Episodes:

#16 – David Krakauer: Scaling Intelligence Feb 27, 2025

David Krakauer is an American evolutionary biologist. He is the President and William H. Miller Professor of Complex Systems at the Santa Fe Institute. As you will hear in today’s episode, David's research centers around a series of fundamental questions, such as: How did life and intelligence evolve in the universe? How do ideas evolve and how do they encode natural and cultural life?

In this conversation, David and I explore the evolving landscape of complexity science. We discuss its foundational theories, emerging patterns, and intersections with AI and machine learning. We delve into the paradigm shift complexity science represents, its most significant contributions across disciplines, and how computational advances are reshaping its trajectory. We also talk about AI’s potential to scale towards AGI through a complexity lens, the limits imposed by evolutionary principles, and what this means for artificial systems. Finally, as President of the Santa Fe Institute, David discusses SFI’s unique interdisciplinary model. I hope you enjoy the conversation.

You can follow me on X (@ProfSchrepel) and BlueSky (@ProfSchrepel) to receive regular updates.

References:

Unifying complexity science and machine learning (2023) https://www.frontiersin.org/journals/complex-systems/articles/10.3389/fcpxs.2023.1235202/full
The debate over understanding in AI’s large language models (2023) https://static1.squarespace.com/static/5f29a430a2b6a34680879cc0/t/672467763ec35e0639db8457/1730439030537/DK-DebateOverUnderstandingInAIsLLMs2023.pdf
Darwinian demons, evolutionary complexity, and information maximization (2011) https://static1.squarespace.com/static/5f29a430a2b6a34680879cc0/t/6725792b7d0d4f0e4e7ca2fe/1730509104265/DK-DarwinianDemonsEvolutionaryComplexity%26InformationMaximization2011.pdf

#15 – Larry Lessig: Code, Law, and Business Models in the Age of AI Feb 03, 2025

My guest today is Larry Lessig, Professor of Law and Leadership at Harvard Law School. Larry is the author of numerous influential books and articles, including Code 2.0 (2006), which we discuss at length in this episode. If you have been listening to Scaling Theory since the very beginning, you probably remember that I cited a couple of books that changed my perception of everything in the first episode. Code 2.0 is one of these books. Larry Lessig develops what he calls the “pathetic dot theory,” in which he explains that all things are influenced by four constraints: the law, economic forces, norms, and architecture.

In this conversation, Larry and I talk about the importance of these four constraints in the digital economy and assess which ones have scaled the most in recent years. We also explore how complexity science can contribute to Larry’s theory by seeing the dots and their constraints as a complex network. We then steer our conversation toward open source in AI, examine how regulation at the hardware layer could solve software issues, and consider whether we can trust our institutions and current regulations to do so, or if we need to scale other institutions for that purpose. I hope you enjoy our discussion.

References:

Code 2.0 (2006) https://lessig.org/product/codev2/
Code (1999) https://lessig.org/product/code/

You can follow me on X (@ProfSchrepel) and BlueSky (@profschrepel) to receive regular updates.

#14 – Eric Beinhocker: “New Economics” Is Coming For You Jan 13, 2025

My guest today is Eric Beinhocker, Professor of Practice in Public Policy at the Blavatnik School of Government, University of Oxford, and the founder and Executive Director of the Institute for New Economic Thinking at the University’s Oxford Martin School. Eric is the author of numerous academic articles and books, including The Origin of Wealth: Evolution, Complexity, and the Radical Remaking of Economics (2007).

In our conversation, Eric and I contrast traditional economics (neoclassical theory) with new economics (complexity economics). We also explore the policy implications of these differing economic theories, discussing topics ranging from aggressive growth strategies to complexity catastrophes in digital economies. I hope you enjoy our conversation.

References:

The origin of wealth: Evolution, complexity, and the radical remaking of economics (2007) ⁠https://moldham74.github.io/AussieCAS/papers/Origins⁠ of Wealth.pdf
Getting Big Too Fast: Strategic Dynamics with Increasing Returns and Bounded Rationality (2007) ⁠https://pubsonline.informs.org/doi/pdf/10.1287/mnsc.1060.0673⁠

Fair Social Contracts and the Foundations of Large-Scale Collaboration (2022) ⁠https://oms-inet.files.svdcdn.com/staging/files/Fair-Social-Contracts-Beinhocker-v8-22-22.pdf⁠
Reflexivity, complexity, and the nature of social science (2013) ⁠https://www.tandfonline.com/doi/full/10.1080/1350178X.2013.859403⁠

#13 – Kevin Kelly: How Technology Evolves, And What To Do About It Dec 19, 2024

My guest today is Kevin Kelly, the author of 14 books, a public speaker who has delivered TED talks with tens of millions of views, and a technology expert. In 1983, Kevin was hired by Whole Earth founder Stewart Brand to edit several later editions of the Whole Earth Catalog, the Whole Earth Review, and Signal. He later on served as the founding executive editor of the magazine Wired.

In our conversation, Kevin and I talk about the scaling laws behind all technologies, but also how these laws intersect with biology, society, and policy. We explore themes from What Technology Wants, we focus on the 'Triad of Evolution' and the concept of convergence, and connect these ideas to antitrust and innovation policy. I also touch on his earlier work, including New Rules for the New Economy, where we discuss the dynamics of trust in network economies and its implications for technology adoption. Finally, we delve into the inevitability of technological evolution, its accelerating diffusion, and what happens when technology becomes ubiquitous in society. These questions feel increasingly urgent as we approach 2025, a pivotal moment for revisiting these ideas in light of modern developments. I hope you enjoy our discussion.

Find me on X (@ProfSchrepel) and BlueSky (@profschrepel.bsky.social).

References

Kevin Kelly, What Technology Wants (2010)
Kevin Kelly, New Rules for the New Economy (1998)
Rishi Bommasani et al., Considerations for Governing Open Foundation Models (2023) https://hai.stanford.edu/issue-brief-considerations-governing-open-foundation-models

#12 – Rory Linkletter: Scaling Up to the Olympics Nov 28, 2024

My guest today is Rory Linkletter, a professional athlete who recently ran the Paris Olympic Marathon and the New York Marathon. Rory’s current personal best in the marathon is an impressive 2:08:01, which makes him the top Canadian marathon runner and the third-best Canadian performance ever.

This episode, as you might guess, is different from the others. I wanted to talk to Rory because he inspired me greatly when I went to Paris to watch the race. Most importantly, I am convinced that there is much we can learn from professional athletes, especially marathon runners.

In our conversation, we explore how Rory scaled his mental and physical abilities. I draw many parallels with the academic and policy worlds, delving into what we can learn from his process, the power laws he has identified, and his relationship with science.

Scaling Theory is not turning into a running podcast, but, true to its mission, it remains focused on exploring the scaling laws behind everything—be it economic, technical, or biological systems. Rory opens new doors regarding this last subject. I hope you enjoy our discussion.

#11 – Stefan Thurner: The Scaling of Everything Nov 08, 2024

My guest is Stefan Thurner, A Professor of theoretical physics, and the President of the Complexity Science Hub in Vienna. Stefan has published over 240 scientific articles and he was elected Austrian Scientist of the Year 2017. He is also an external professor at the Santa Fe Institute.

In our conversation, we first delve into the scaling laws of everything. We explore social, financial, biological, and economic dynamics—for example, how to make the economy more resilient by targeting some unique companies, how social bubbles form, the strength of networks of friends and foes in social contexts, and how the methodology of physics can help us understand other fields, etc. I hope you enjoy our discussion.

Find me on X at @⁠⁠⁠⁠ProfSchrepel⁠⁠⁠⁠⁠⁠⁠. Also, be sure to subscribe.

***

References:

➝ Measuring social dynamics in a massive multiplayer online game (2010)

➝ How women organize social networks different from men (2013)

➝ Multirelational Organization of Large-Scale Social Networks in an Online World (2010)

➝ What is the minimal systemic risk in financial exposure networks? (2020)

➝ Scaling laws and persistence in human brain activity (2003)

➝ New Forms of Collaboration Between the Social and Natural Sciences Could Become Necessary for Understanding Rapid Collective (2024)

➝ Quantifying firm‐level economic systemic risk from nation‐wide supply networks (2022)

➝ Fitting Power-laws in Empirical Data with Estimators that Work for All Exponents (2017)

➝ Complex Systems: Physics Beyond Physics (2017)

➝ Systemic Financial Risk: Agent-based Models to Understand the Leverage Cycle on National Scales and its Consequences (2011)

➝ Peer-review in a world with rational scientists: Toward selection of the average (2010)

#10 – Allison Stanger: Political Science Behind Large Tech Companies Sep 26, 2024

My guest today is Allison Stanger. Allison is a Middlebury Distinguished Endowed Professor; an Affiliate at the Berkman Klein Center for Internet and Society, Harvard University; the Co-Director (with Danielle Allen) of the GETTING-Plurality⁠ Research Network, Harvard University; founding member of the Digital Humanism Initiative (Vienna); and an External Professor at the Santa Fe Institute. Allison’s next book, Who Elected Big Tech? is under contract with Yale University Press.

In this conversation, Allison and I delve into the political science surrounding large tech companies. We explore their effects on consumers and democracy, the interplay between capitalism and democracy, the dangers of fragmented regulation, what the effective governance of social media entails, how to scale and measure it, potential areas of cooperation with China, and the relevance of public choice theory, complexity science, and power laws in shaping our understanding of technology. I hope you enjoy our discussion.

***

References

Stanger, Allison. "The Real Cost of Surveillance Capitalism: Digital Humanism in the United States and Europe." Perspectives on Digital Humanism (2022): 33-40. https://library.oapen.org/bitstream/handle/20.500.12657/51945/978-3-030-86144-5.pdf

Werthner, Hannes, et al. "Digital humanism: The time is now." Computer 56.1 (2023): 138-142. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10008968
Soros, George. "Fallibility, reflexivity, and the human uncertainty principle." Journal of Economic Methodology 20.4 (2013): 309-329. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10008968

#9 – Arvind Narayanan: Myths and Policies in Scaling AI Aug 26, 2024

My guest is Arvind Narayanan, a Professor of Computer Science at Princeton University, and the director of the Center for Information Technology Policy, also at Princeton. Arvind is renowned for his work on the societal impacts of digital technologies, including his textbook on fairness and machine learning, his online course on cryptocurrencies, his research on data de-anonymization, dark patterns, and more. He has already amassed over 30,000 citations on Google Scholar.

In just a few days, in late September 2024, Arvind will release a book co-authored with Sayash Kapoor titled “AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference.” Having had the privilege of reading an early version, our conversation delves into some of the book’s key arguments. We also explore what Arvind calls AI scaling myths, the reality of artificial general intelligence, how governments can scale effective AI policies, the importance of transparency, the role that antitrust can, and cannot play, the societal impacts of scaling automation, and more. I hope you enjoy our conversation.

Find me on X at @⁠⁠⁠ProfSchrepel⁠⁠⁠⁠⁠⁠. Also, be sure to subscribe.

References:

➝ AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (2024)

➝ AI scaling myths (2024)

➝ AI existential risk probabilities are too unreliable to inform policy (2024)

➝ Foundation Model Transparency Reports (2024)

#8 – Sara Hooker: Big AI, The Compute Frenzy, and Grumpy Models Aug 05, 2024

My guest today is ⁠Sara Hooker⁠, VP of Research at Cohere, where she leads Cohere for AI, a non-profit research lab that seeks to solve complex machine learning problems with researchers from over 100 countries. Sara is the author of numerous research papers, some of which focus specifically on scaling theory in AI. She has been listed as one of AI’s top 13 innovators by Fortune.

In our conversation, we first delve into the scaling laws behind foundation models. We explore what powers the scaling of AI systems and the limits to scaling laws. We then move on to discussing openness in AI, Cohere’s business strategy, the power of ecosystems, the importance of building multilingual LLMs, and the recent change in terms of access to data in the space. I hope you enjoy our conversation.

Find me on X at @⁠⁠ProfSchrepel⁠⁠⁠⁠⁠. Also, be sure to subscribe.

References:

➝ Sara Hooker, On the Limitations of Compute Thresholds as a Governance Strategy (2024)

➝ Sara Hooker, The Hardware Lottery (2020)

➝ Sara Hooker, Moving beyond “algorithmic bias is a data problem” (2021)➝ Longpre et al., Consent in Crisis: The Rapid Decline of the AI Data Commons (2024)

#7 – Michael Mauboussin: The Fascinating World of Increasing Returns Jul 15, 2024

My guest today is Michael Mauboussin (@mjmauboussin), one of the world’s leading experts in finance. Michael serves as Head of Consilient Research at Counterpoint Global, Morgan Stanley. He has authored three books and regularly appears in the Wall Street Journal, Financial Times, New York Times, and other publications. Since 1993, Michael has been an adjunct professor of finance at Columbia Business School and is also the chairman emeritus of the board of trustees at the Santa Fe Institute.

In our conversation, we delve into the dynamics of markets, discuss all sorts of increasing returns, and explore topics such as Charles Darwin, policymaking, AI and Web3, and the Santa Fe Institute. I hope you enjoy our discussion.

Find me on X at @⁠ProfSchrepel⁠⁠⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast.

References:

Michael J. Mauboussin & Dan Callahan, "Increasing Returns: Identifying Forms of Increasing Returns and What Drives Them" (2024) https://perma.cc/Y3DN-LNMY
Michael J. Mauboussin & Dan Callahan, "Stock Market Concentration: How Much Is Too Much?" (2024) https://perma.cc/7EEX-ZY9T
Charles Darwin, The Autobiography of Charles Darwin: 1809-1882 https://www.amazon.com/Autobiography-Charles-Darwin-1809-1882/dp/0393310698
David Warsh, Knowledge and the Wealth of Nations: A Story of Economic Discovery (2007) https://www.amazon.com/Knowledge-Wealth-Nations-Economic-Discovery/dp/0393329887
James Bessen, The New Goliaths: How Corporations Use Software to Dominate Industries, Kill Innovation, and Undermine Regulation (2022) https://www.amazon.nl/-/en/James-Bessen/dp/0300255047
Chris Dixon, Read Write Own: Building the Next Era of the Internet (2023) https://readwriteown.com
Anu Bradford, Digital Empires: The Global Battle to Regulate Technology (2023) https://global.oup.com/academic/product/digital-empires-9780197649268
Kenneth J. Arrow, "The Economic Implications of Learning by Doing" (1962) https://www.jstor.org/stable/2295952
J. Doyne Farmer, Making Sense of Chaos (2024) https://www.penguin.co.uk/books/284357/making-sense-of-chaos-by-farmer-j-doyne/9780241201978

#6 – Glen Weyl: Scaling Plurality Jun 20, 2024

My guest is Glen Weyl, an influential economist and social technologist known for his interdisciplinary work at the intersection of economics, computer science, sociology, and political science. He is a Principal Researcher at Microsoft Research, a co-author of the books “Radical Markets” and the recently published “Plurality: The Future of Collaborative Technology and Democracy” which he co-authored with Audrey Tang, who has served as the 1st Minister of Digital Affairs of Taiwan. Glen co-founded the RadicalxChange Foundation, the Plural Technology Collaboratory, and the Plurality Institute. He taught economics at the University of Chicago, Princeton, and Yale.

Glen’s work frequently explores the potential of technology and market mechanisms to drive social change and address systemic inequalities. In his new book, Glen lays the foundation for a new societal vision: plurality.

Our conversation is structured into three distinct parts. We begin by situating the concept of plurality in today’s philosophical and technological landscape. We then discuss the technologies that will enable plurality to materialize, and finally, we conclude with a discussion of the challenges ahead. By the end of this discussion, you will know what plurality is, how different it is from the vision of today’s tech CEOs, the role open source AI and blockchain can play to scale plurality, but also the role legal rules and standards are playing, which of the scaling theory concepts (including fractals, fragility, robustness, chaos, etc.) helped Glen and Audrey define their strategy to take over other visions, and more. I hope you enjoy the conversation.

Find me on X at @ProfSchrepel⁠⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

#5 – Yann LeCun: AI Dynamics and Regulation May 27, 2024

My guest is Yann LeCun, a pioneering French-American computer scientist, known for his groundbreaking work in machine learning, computer vision, and neural networks. Yann is the Silver Professor at the Courant Institute of Mathematical Sciences at New York University and serves as the Vice President and Chief AI Scientist at Meta.

Yann is one of the world’s most influential computer scientists. He has accumulated over 350,000 citations on Google Scholar, he is one of the founding figures in the field of deep learning thanks to its contribution to convolutional neural networks and backpropagation algorithms, and he is a vocal proponent of open source. In recognition of his significant contributions to artificial intelligence, he was awarded the Turing Award in 2018, often referred to as the “Nobel Prize of Computing.”

Our conversation is structured into three distinct parts. We begin by discussing the overarching dynamics in the AI space, then narrow our focus to the firm level, and finally, we conclude with an exploration of the challenges that lie ahead. By the end of this discussion, you will learn whether open source has a chance to make it in AI, the key factors for scaling an AI foundation model, the role ecosystems play in market dynamics, Meta long term strategy in the space, how concentration among chip manufacturers impacts AI companies, the current effect of the European AI Act on AI companies, what Yann would like to see regulators doing, and more. I hope you enjoy the conversation.

#4 – Doyne Farmer: Chaos Theory & Complexity Economics May 06, 2024

J. Doyne Farmer is the Director of the Complexity Economics programme at the Institute for New Economic Thinking at the Oxford Martin School, a Professor at the Mathematical Institute of Oxford University, and an External Professor at the Santa Fe Institute.

In this episode, we explore Doyne’s latest book, “Making Sense of Chaos.” We focus on the relationship between chaos and scaling theory, and more specifically, how chaos can be factored into scaling theory. By the end of this conversation, you will learn why it might be easier to predict the long distant future than predicting tomorrow, how Moore’s Law conflicts with other scaling laws that underpin technological progress, how agent-based modeling can help all scientists and policymakers, how to dominate the world with your theories (...), and even how to trick casinos. I hope you enjoy the conversation.

Find me on X at @⁠⁠ProfSchrepel⁠⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

#3 – Thomas Wolf: Open-Source AI Scaling Laws Apr 15, 2024

Thomas Wolf is the co-founder and Chief Science Officer of Hugging Face, the company at the center of the open-source AI ecosystem. He has a Ph.D. in statistical & quantum physics.

In this episode, we explore why open-source (“OS”) AI may be preferable to closed-source, whether OS has a real chance to take over the space, the challenges OS developers must overcome to scale their foundational models, the role of data and infrastructure in scaling dynamics, how big tech companies are positioning themselves in the ecosystem and responding to other companies' strategies in a true complexity science fashion, and overall, the power laws that undermine the growth of the ecosystem. I hope you enjoy the conversation. I hope you enjoy the conversation.

Find me on X at @⁠ProfSchrepel⁠. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

#2 – Geoffrey West: The Scaling Laws Behind Living Organisms, Cities, Businesses, and Technologies Mar 25, 2024

Geoffrey West is a physicist, former president and distinguished professor of the Santa Fe Institute. His book, “Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Businesses” (2017), is a masterpiece.

In this episode, we talk about the power laws behind living organisms, cities, businesses, and technologies. By the end of this episode, you will know more about the power law behind the heartbeat of all mammals, the number of patents and crime in big cities compared to small cities, innovation, the way technology scales, and more. I hope you enjoy the conversation.

Find me on X at @ProfSchrepel. Also, be sure to subscribe to the Scaling Theory podcast; it helps its growth.

***

References

Geoffrey West, ⁠Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Businesses⁠ (Penguin Books, 2017)
George J. Stigler, “The Economies of Scale”, The Journal of Law & Economics 1 (1958): 54–71
Michael HR Stanley, et al. “⁠Scaling Behaviour in the Growth of Companies”, Nature 379.6568 (1996): 804-806.APA
W. Brian. Arthur, “⁠Competing Technologies, Increasing Returns, and Lock-In by Historical Events”, The Economic Journal 99.394 (1989): 116-131.
Madeleine IG Daepp, et al. “⁠⁠⁠⁠⁠⁠⁠⁠⁠The Mortality of Companies⁠”, Journal of the Royal Society Interface 12.106 (2015): 20150120.APA
Jiang Zhang, et al. “⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Scaling Laws and a General Theory for the Growth of Public Companies ⁠⁠”, arXiv preprint arXiv:2109.10379 (2021)

Why “Scaling Theory” Mar 16, 2024

In this first episode, Dr. Thibault Schrepel (@⁠ProfSchrepel⁠) introduces “Scaling Theory”, a podcast dedicated to the power laws behind the growth of companies, technologies, legal and living systems.

***

References:

➝ Charles Darwin, On the Origin of Species (1859)

➝ Melanie Mitchell, Complexity: A Guided Tour (2011)

➝ Mitchell Waldrop, Complexity The Emerging Science at the Edge of Order and Chaos (2019)

➝ John H. Miller & Scott Page, Complex Adaptive Systems: An Introduction to Computational Models of Social Life (2007)

➝ W. Brian Arthur, Complexity and the Economy (2014)

➝ Geoffrey West, Scale: The Universal Laws of Life and Death in Organisms, Cities and Companies (2017)