Complex Systems with Patrick McKenzie (patio11)

Latest Episodes:

AI and the great developer speed-up, with Joel Becker of METR Aug 21, 2025

Show notes

This week on Complex Systems, Patrick McKenzie (patio11) is joined by Joel Becker from METR. They discuss groundbreaking research on AI coding assistants.

Joel et al’s randomized controlled trial of 16 expert developers working on major open source projects revealed a counterintuitive finding: despite predictions of 24-40% speed improvements, developers actually took 19% longer to complete tasks when using AI tools, even though they retrospectively believed they were 20% faster. The conversation explores why even sophisticated professionals struggle to accurately assess their own productivity with AI tools, the industrial organization of software development, and the implications for AI's recursive self-improvement in research and development. It also touches on other perspectives from software developers using these tools professionally, and where we can expect them to improve rapidly.

–

Full transcript available here: www.complexsystemspodcast.com/the-great-developer-speed-up-with-joel-becker/

–
Sponsor:
This episode is brought to you by Mercury, the fintech trusted by 200K+ companies — from first milestones to running complex systems. Mercury offers banking that truly understands startups and scales with them. Start today at Mercury.com

Mercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group, Column N.A., and Evolve Bank & Trust; Members FDIC.
–

Recommended in this episode:

METR: https://metr.org/
Joel Becker’s site: https://joel-becker.com/

–

Timestamps:

(00:00) Intro

(00:34) Understanding AI evaluation methods

(02:04) METR's unique approach to AI evaluation

(03:10) The evolution of AI capabilities

(06:44) AI as coding assistants

(09:15) Research on AI's impact on developer productivity

(13:55) Sponsor: Mercury

(15:07) Challenges in measuring developer productivity

(20:38) Insights from the research paper

(31:26) The formalities of software development

(32:07) Automated tools and human discussions

(32:47) AI and style transfer in software

(34:35) The role of comments in AI coding

(36:51) The future of AI in software engineering

(40:25) Economic implications of AI in software

(46:53) Challenges and risks of AI in software

(59:03) Security concerns with AI-generated code

(01:04:59) Wrap

How banks actually work (and don't work) Aug 14, 2025

Show notes

In this solo episode, Patrick McKenzie reads his classic essay "Seeing Like a Bank," exploring why financial institutions often appear to have no memory of previous customer interactions despite being excellent at tracking money itself. He breaks down the complex web of legacy systems, tiered support structures, and regulatory constraints that create Kafka-esque experiences for bank customers. Using the lens of institutional legibility borrowed from "Seeing Like a State," Patrick explains how banks' technical architecture and organizational design choices—from core processing systems to customer service tiers—systematically generate the dysfunction that customers experience when things go wrong.

–

Full transcript available here:
www.complexsystemspodcast.com/how-banks-actually-work/

–

Recommended in this episode:

Patrick’s Bits about Money essay, Seeing like a bank: https://www.bitsaboutmoney.com/archive/seeing-like-a-bank/

–

Timestamps:

(00:00) Intro
(03:52) Recordkeeping systems
(10:20) Sponsor: Safebase
(11:50) Human accountability and its malcontents
(22:57) Two embedded surprises about bank staffing
(27:47) Society has goals which conflict with banks being good at banking
(30:52) So what can be done about this?

Building software in Japan, with Jim Weisser Aug 07, 2025

Show notes

Patrick McKenzie (patio11) is joined by Jim Weisser, a serial entrepreneur and founder of SignTime, for an in-depth exploration of Japan's software market and startup ecosystem. They discuss the unique challenges of building software in a culture that prizes stability over rapid iteration, the dominance of systems integrators, and how Japan's economic stagnation shaped its relationship with technology. The conversation covers everything from Excel-driven development processes to the recent transformation of Japan's VC landscape, offering insights for anyone considering doing business in the world's third-largest economy.

–
Full transcript: https://www.complexsystemspodcast.com/building-software-in-japan-with-jim-weisser/

–

Sponsor: BoxWorks 2025

Discover hands-on AI data extraction techniques, network with industry experts, and hear featured speakers Vinod Khosla, Jared Kaplan, and Aaron Levie at BoxWorks 2025 in San Francisco (Sept 11-12). Use code BW25-ComplexSystems for 50% off at this link: https://bit.ly/452036h

–

Links:

Signtime:: https://www.signtime.com/en/

–

Timestamps:

(00:00) Intro

(00:24) Exploring the Japanese software market

(01:25) Challenges in the Japanese startup ecosystem

(02:37) Jim Weisser's background and journey

(05:14) The evolution of Japan's economy and business practices

(10:15) Understanding Japan Inc. and its impact on software

(10:28) The role of systems integrators in Japan
(19:01) Sponsor: BoxWorks Conference 2025 (*discounted tickets for Complex Systems listeners)

(20:29) Cultural differences in software development

(26:06) Digital transformation and efficiency in Japan

(32:44) Labor market dynamics and employment practices

(35:50) Startup ecosystem and equity culture in Japan

(42:53) Success stories and market potential in Japan

(43:42) Corporate relationships in Japanese companies

(44:10) Salesforce's success in Japan

(45:44) E-commerce boom in Japan

(46:33) Rakuten's evolution and strategy

(48:27) Amazon's journey in Japan

(49:13) Cloud services and security concerns

(51:18) Sansan: Business card management

(55:14) Venture capital landscape in Japan

(01:05:02) Personal reflections on living in Japan

(01:16:21) Wrap

Startup investing in Tokyo, with Coral Capital’s James Riney Jul 31, 2025

Show notes

Patrick McKenzie (patio11) is joined by James Riney, partner at Coral Capital, to explore Japan's transformation from a $700 million startup ecosystem to today's $5-10 billion market. They discuss the cultural and structural factors that initially limited venture activity in Japan. The conversation covers unique aspects of building startups in Japan, from the quirks of being a foreign professional to why Japanese engineers love Twitter but ignore LinkedIn, plus insights into Japan's "time machine advantage" and why American-developed dev tools are going viral in Tokyo.
–
Read full transcript here:
www.complexsystemspodcast.com/startup-investing-in-tokyo/

–

[Patrick notes: Complex Systems now produces occasional video episodes! You can access them directly on YouTube: www.youtube.com/@patio11podcast. My kids inform me that I’m supposed to tell you to like and subscribe.]
–

Sponsor: Safebase
Leading companies use SafeBase to eliminate up to 98% of inbound security questionnaires, automate workflows, and accelerate pipeline. Go to safebase.io/podcast

–

Links:

Coral Capital: https://coralcap.co/

–

Timestamps:

(00:00) Intro

(01:57) The early days of Japanese startups

(04:08) The rise of Coin Check

(05:54) Challenges and opportunities in the Japanese startup ecosystem

(16:09) Cultural and structural differences in hiring
(19:47) Sponsor: Safebase

(21:22) The role of content and communication in Japanese startups

(31:29) LinkedIn vs. Facebook in Japanese work culture

(32:38) LinkedIn's social capital issues in Japan

(33:58) Cultural differences in asking for permission

(34:31) Navigating Japanese regulatory clarity

(36:49) The evolution of VC investment in Japan

(39:54) The rise of SaaS in Japan

(45:26) System integrators and software development in Japan

(50:39) Challenges in Japanese tech companies

(54:36) Opportunities for foreign companies in Japan

(55:03) The importance of commitment in the Japanese market

(57:08) Dev tools and viral adoption in Japan

(59:31) Japan's influence on global tech

(01:03:23) Wrap

How AI reshapes the craft of software engineering, with Yoav Tzfati Jul 24, 2025

Show notes

Patrick McKenzie is joined by AI researcher Yoav Tzfati to discuss “vibe coding” - using LLMs to delegate software engineering work to AI models. Yoav runs a bootcamp teaching programming novices to build full-stack web applications using AI, without them ever looking at code. Patrick and Yoav discuss the fundamental shift in software engineering, where humans increasingly act as product managers directing AI "junior engineers," and explore the implications for the future of programming careers and the democratization of software development.

–

Read full transcript here: www.complexsystemspodcast.com/how-ai-reshapes-software-engineering/
–

[Patrick notes: Complex Systems now produces occasional video episodes like this one!You can access them directly on YouTube: https://www.youtube.com/@patio11podcast. My kids inform me that I’m supposed to tell you to like and subscribe.]
–

Links:

Follow Yoav Tzfati: https://x.com/yoavtzfati
Yoav's bootcamp: https://www.code-bloom.app/

–

Timestamps:
(00:00) Intro

(00:32) Defining vibe coding

(01:35) The evolution of software engineering with LLMs

(04:07) Practical applications of vibe coding

(09:37) Teaching vibe coding to novices

(18:30) Future of AI in software development

(21:42) Discussing timelines and model capabilities

(22:12) Flappy Bird and the evolution of game development

(23:27) The impact of LLMs on software engineering

(24:46) Future of coding and human roles

(29:47) Monitoring and error handling in software

(31:20) The role of LLMs in code review and maintenance

(35:12) Wireframing and project management with LLMs

(36:40) The future of software engineering careers

(43:07) Practical tips for software engineers

(44:38) Wrap

Stacking the odds: Cate Hall on agency and outlier success Jul 17, 2025

Show notes

Patrick McKenzie (patio11) is joined by Cate Hall, CEO of Astera Institute and author of a forthcoming book on agency, to explore how individuals can systematically develop higher agency in their lives. They discuss the selection effects that draw agentic people to fields like poker and startups, the importance of being comfortable with ignorance and feedback, and practical strategies like asking "Is there a better way to do this?" ten times daily. Cate shares insights from her journey from Supreme Court lawyer to world champion poker player, including how peer groups, emergencies, and what addicts call "the gift of desperation" can trigger step-changes in personal agency.
–

Full transcript available here:
https://www.complexsystemspodcast.com/stacking-the-odds-cate-hall/

–

[Patrick notes: Complex Systems now produces occasional video episodes like this one.You can access them directly on YouTube: https://www.youtube.com/@patio11podcast. My kids inform me that I’m supposed to tell you to like and subscribe.]

–

Links:

Cate Hall’s Writing: https://usefulfictions.substack.com/
Pre-order Cate's book on agency: https://mailchi.mp/44578760e686/book-preorder-signup
Astera: https://astera.org/

–

Timestamps:

(00:00) Intro

(00:28) Exploring the concept of agency

(00:56) Selection effects in different fields

(01:31) The tech industry evolution

(02:54) Y Combinator's changing demographics

(04:23) Parallels between poker and startups

(15:48) The role of networking and relationships

(19:12) Embracing feedback and continuous improvement

(23:49) Exploring growth mindset

(24:35) AI and personal improvement

(27:04) Learning with AI: strategies and cautions

(27:53) The value of human interaction

(30:19) Professional gamblers and their unique insights

(35:05) Understanding agency and its development

(38:45) Triggers for increased agency

(45:23) Wrap

Think like a trader, with Ricki Heicklen Jul 10, 2025

Show notes

Patrick McKenzie (patio11) is joined again by Ricki Heicklen to discuss the evolution of her trading education business, Arbor, one year after their first conversation. They dive deep into the pedagogy of trading, exploring how simulated markets teach concepts like adverse selection, team dynamics, and risk management through hands-on experience. Ricki shares war stories from the bootcamp trenches—infinite loop bugs that mirror Knight Capital's disaster, WiFi outages that create unexpected trading opportunities, and that the most successful trading teams often focus on internal team communication even more than trade execution or technical acumen.

See the full transcript:
https://www.complexsystemspodcast.com/think-like-a-trader-ricki-heicklen/

–

[Patrick notes: Complex Systems now produces occasional video episodes.You can access them directly on YouTube: https://www.youtube.com/@patio11podcast. My kids inform me that I’m supposed to tell you to like and subscribe.]

–

Links:

Trading Camp : https://trading.camp/
Metagame: https://www.metagame.games/#tickets
Story of Knight Capital: https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf

–

Timestamps:
(00:00) Intro

(00:46) Ricki's journey from trading to teaching

(01:25) The birth of Arbor and first bootcamps

(03:32) Developing a trader's mindset

(05:53) Understanding heuristics in trading

(08:21) Adverse selection in everyday life

(15:40) Insights from teaching trading bootcamps

(21:07) Pedagogical approach: learning by doing

(32:00) Handling mistakes and learning opportunities

(36:17) Unplanned bugs and real-world lessons

(39:47) Learning from Knight Capital's bug

(40:24) Understanding exchange-side bugs

(43:10) Risk limits and strategy separation

(44:41) Importance of UI in trading bots

(46:53) The Madagascar button

(48:20) The big red button in manufacturing

(49:45) Simulated trading and information aggregation

(50:29) Sibling trading game explained

(53:24) Modeling and hidden information

(01:01:15) Trading behavior and market updates

(01:04:38) Real-world applications and lessons

(01:13:58) Surprises and market opportunities

(01:16:24) Pedagogical approaches in trading education

(01:17:08) Market dynamics and counterparty behavior

(01:17:53) Retail vs. institutional order flow

(01:19:23) Simplifying trading concepts for beginners

(01:21:27) Introducing market characters and their roles

(01:31:31) Team dynamics and communication in trading

(01:39:13) The importance of redundancy in trading systems

(01:47:52) Future of trading education and online classes

(01:53:47) Wrap

Achieving results in the physical world, with Adam Jarvis of “Public Service” Jul 03, 2025

Show notes

Patrick McKenzie is joined by Adam Jarvis, author of the Public Service substack and a New Zealand civil engineer and public sector veteran. They discuss how political capital constraints, funding misalignment across government levels, and accumulated regulatory "scar tissue" make infrastructure projects extraordinarily difficult. The conversation reveals why replacing a water pipe now costs more in planning than the entire project did a decade ago, and how talent sorting has drained capacity from public institutions. Despite these challenges, Patrick and Adam find reasons for optimism about reforming government capacity.

Complex Systems now has video episodes as well. Watch this episode and subscribe at: https://www.youtube.com/@patio11podcast
–
Full transcript: www.complexsystemspodcast.com/achieving-results-in-the-physical-world-with-adam-jarvis-of-public-service/

–

Links:

Public Service Substack: https://alethios.substack.com/

–

Timestamps:

(00:00) Introduction to Complex Systems
(01:13) Understanding local government functions
(03:41) Challenges in public sector project delivery
(06:31) Funding complexities in public projects
(09:14) The burden of regulatory and legislative constraints
(17:04) Appreciating the infrastructure we take for granted
(23:02) Historical and modern infrastructure challenges
(30:04) Talent mobility and its impact on public sector
(33:35) Public sector hiring practices
(34:11) Agglomeration effects and brain drain
(36:04) Aging population and resource allocation
(36:59) Structural factors in public sector layoffs
(37:49) Hiring and firing in the public sector
(40:39) Labor mobility and job security
(46:34) AI and automation in public sector jobs
(52:56) Risk aversion and process overload
(01:01:30) Optimism for public sector reform

The AI infrastructure stack with Jennifer Li, a16z Jun 26, 2025

Show notes

In this episode, Patrick McKenzie (@patio11) is joined by Jennifer Li, a general partner at a16z investing in enterprise, infrastructure and AI. Jennifer breaks down how AI workloads are creating new demands on everything from inference pipelines to observability systems, explaining why we're seeing a bifurcation between language models and diffusion models at the infrastructure level. They explore emerging categories like reinforcement learning environments that help train agents, the evolution of web scraping for agentic workflows, and why Jennifer believes the API economy is about to experience another boom as agents become the primary consumers of software interfaces.
–
Full transcript:
www.complexsystemspodcast.com/the-ai-infrastructure-stack-with-jennifer-li-a16z/

–

Sponsor: Vanta

Vanta automates security compliance and builds trust, helping companies streamline ISO, SOC 2, and AI framework certifications. Learn more at https://vanta.com/complex

–

Links:

Jennifer Li’s writing at a16z https://a16z.com/author/jennifer-li/

–

Timestamps:

(00:00) Intro
(00:55) The AI shift and infrastructure
(02:24) Diving into middleware and AI models
(04:23) Challenges in AI infrastructure
(07:07) Real-world applications and optimizations
(15:15) Sponsor: Vanta
(16:38) Real-world applications and optimizations (cont’d)
(19:05) Reinforcement learning and synthetic environments
(23:05) The future of SaaS and AI integration
(26:02) Observability and self-healing systems
(32:49) Web scraping and automation
(37:29) API economy and agent interactions
(44:47) Wrap

Getting better at LLMs, with Zvi Mowshowitz Jun 12, 2025

Show notes

In this episode, Patrick McKenzie (patio11) recorded with Zvi Mowshowitz (TheZvi) live at the LessOnline conference. They explore practical strategies for getting better results from large language models. Zvi explains how to customize AI behavior through thoughtful system prompts, while Patrick shares techniques for using LLMs as writing partners and research assistants. They discuss the evolving relationship between content creators and AI training data, touching on the emerging field of "generative engine optimization" (GEO). The conversation also covers multimodal capabilities, recursive AI use, and strategies for avoiding common failure modes like hallucination and sycophancy.
–
Full transcript: www.complexsystemspodcast.com/getting-better-at-llms-with-zvi-mowshowitz/

–

Sponsor: Vanta

Vanta automates security compliance and builds trust, helping companies streamline ISO, SOC 2, and AI framework certifications. Learn more at https://vanta.com/complex

–

Links: