Show notes
BONUS: The Chip Race Behind AI With Rakesh Kumar
AI feels like a software revolution, but Rakesh Kumar argues that the next limits may come from chips, energy, and the physical systems underneath our code. In this BONUS episode, Rakesh explains why AI progress is tied to hardware, what software teams need to relearn about efficiency, and why the economics of data centers should matter to product leaders.
Why Chips Became a Society Topic "The race about AI essentially becomes the race about chips."
Rakesh traces his path from a small town in India to computer science, chip research, entrepreneurship, and eventually writing The Chip Age. The turning point came during the COVID-era chip shortages, when cars, appliances, and everyday electronics became harder to buy. That moment made chips visible to people outside engineering. AI then raised the stakes further: the promise of new medicines, new materials, and major productivity gains depends on models that require enormous amounts of computation. For software people, that means chips are no longer background infrastructure. They are part of the product strategy, the cost model, and the operating constraint.
AI Progress Depends on Computation "To the first order, the more chips you can use and the more data you throw in at training AI models, the more powerful an AI model becomes."
Vasco frames the shift with a sharp comparison: AlexNet used two GPUs in 2012, while newer frontier models use tens or hundreds of thousands of GPUs. Rakesh connects that growth to AI scaling laws. The problems AI is asked to solve have changed, from recognizing cats and dogs to scientific discovery and complex reasoning tasks. That creates pressure to train larger models with more data and more chips. The result is a world where software teams cannot assume that compute is cheap, abundant, or invisible. AI roadmaps are now connected to chip availability, data center capacity, energy, and the ability to use hardware well.
Software Teams Need to Care About Efficiency Again "Software developers now have to refocus on performance, because that dictates the quality of AI you can serve and the cost of AI when you're serving AI."
For years, software teams could often rely on hardware getting faster underneath them. Rakesh says AI changes that habit. If two teams have access to the same number of chips, the team that extracts more useful work from those chips has an advantage. That advantage shows up in model quality, inference cost, latency, and energy consumption. Efficiency is no longer a niche concern for low-level specialists. It becomes a product concern. Teams building with AI need to understand where computation goes, how much energy it consumes, and which architectural choices make expensive infrastructure work harder than it needs to.
The Hardware and Software Boundary Is Moving "Because of AI, the hardware design and the software design at some level, those abstractions have started merging."
Rakesh points to new AI-assisted chip design workflows, including the Jalapeno chip example Vasco mentions, as a sign that software and hardware work are moving closer together. Chip design that once took years can now be accelerated with AI-supported automation and specialized partners. At the same time, hardware choices are becoming more tailored to AI workloads. General-purpose chips offer flexibility, but specialized chips may offer much better efficiency. That creates a trade-off familiar to software leaders: optimize too narrowly and the target may move; stay too general and the system may be too costly to run.
The Data Center Constraint "I think we have to figure out how to do AI efficiently. The current way of doing AI seems pretty unsustainable to me."
Rakesh warns that AI data centers are running into physical constraints. Large facilities can consume gigawatts of power, and a significant share of that power is used to move heat out of the system. There are also inefficiencies in generation, transmission, distribution, cooling, and in the short useful life of expensive GPUs when model requirements change quickly. For agile teams and product leaders, this matters because energy and infrastructure are becoming part of the real cost of AI features. The question is not only whether an AI feature works. It is whether it can be served responsibly, affordably, and at scale.
What Breaks First in the AI Supply Chain? "The capacity of the grid grows really slowly. We have this slowly increasing supply now hitting against the AI exponential."
When Vasco asks what may break first if AI demand keeps rising, Rakesh ranks energy supply as the biggest challenge. New transmission lines and grid connections can take years, while AI demand grows much faster. Chip supply comes next: can the industry build enough chips fast enough? Cost is also a pressure point, especially for smaller companies, developing countries, and individual consumers. Rakesh also flags a broader economic risk: data centers can cost billions, and some financing is backed by older chips as collateral. If the economics fail, the impact may reach beyond individual AI companies.
What Software Developers Should Prepare For "Generating functional code will be a lot easier in future than generating functional code that meets your performance and energy objectives and constraints."
Rakesh argues that every software engineer needs to become fluent with AI, but that is only the starting point. The valuable work may shift toward areas where AI still struggles: physical AI, robotics, IoT, sensors, performance, energy, and verification. If AI writes more code, teams still need ways to build confidence in that code. Vasco connects this to familiar engineering practices such as testing, performance testing, security testing, and maturation. The future may include much more software than today, but the scarce skill may be knowing how to make software trustworthy, efficient, and suitable for the physical systems it touches.
About Rakesh Kumar Rakesh Kumar is a Professor and John Bardeen Faculty Scholar in Computer Engineering at Illinois and author of The Chip Age. He studies chips and computer systems, co-founded chip companies, advised companies and governments, and developed technologies now used in phones, tablets, and computers.
You can follow Rakesh Kumar's work on his University of Illinois webpage and read The Chip Age.
Our TOPPODCAST Picks 
Stay Connected
