The AI boom is running into the physical world. The constraints shaping AI’s next phase is increasingly physical, and many of the hardest infrastructure problems remain unsolved.
I covered why we’re in the AI infrastructure gold rush, and recently sat down with Amin Vahdat from Google, Santosh Janardhan from Meta, and Sajiv Dalal from TSMC to understand the demand signal for AI infrastructure from three different vantage points at the 2026 Silicon Catalyst Summit, and they painted a remarkably clear picture of where AI infrastructure is headed.
The bottlenecks are physical, the opportunities are massive, but many of the hard problems remain unsolved.
Here is what they said, and what it means if you are building the future of AI infrastructure.
Amin framed that the Industrial Revolution mattered because it created capabilities no one had imagined, not because it pumped water out of coal mines.
In the 100x Collaborative Intelligence era, the truly net-new applications of AI have not arrived yet. We are still in the faster-better-cheaper phase, where AI augments and automates what already exists. World-changing inventions are still ahead, just as Uber and DoorDash came a few years after the iPhone, not on day one.
If AI is a nine-inning baseball game, Santosh said that we are in innings two or three. Two timelines are colliding: Software moves fast, but chips, data centers, power, and turbines do not. You can have a software idea in the shower and ship it in two weeks. You cannot do that with a data center. A gigawatt of capacity will take three years to build.
What this means for founders: the AI infrastructure layer, the picks and shovels of this era, needs to be built now, because the physical world runs on multi-year timelines.
Bottom line: The world-changing innovations in AI are still ahead. Build for the physical timeline, not the software one.
Power and cooling are becoming the defining long-term constraints of the AI era. The numbers are stark. According to Santosh, the U.S. adds 30-40 GW of net new power capacity per year. China adds 550 GW of capacity each year, more than India’s entire annual output. That gap is why China can run frontier AI models on underpowered chips and still compete.
Turbine lead times already stretch to 2029 and 2030. Data centers planned today will not come online until 2027. Google is on record for roughly $185 billion in capex this year, a significant share going to power, cooling, and space.
The short-term answer is behind-the-meter generation – think gas turbines, solar, batteries – but even that capacity is maxed out. The longer-term answer is nuclear and better grid orchestration.
The opportunity goes beyond generating more power. The biggest levers are using existing power more intelligently and cooling infrastructure. Hyperscalers historically provisioned for 99.999% availability, leaving massive amounts of power sitting idle, while cooling is increasingly becoming the limiting factor in extracting full performance from modern AI systems.
A massive percentage of the power arriving at the data center never reaches the chips. The inefficiency between the grid connection and the actual compute is enormous. Google has committed a gigawatt of demand response back to utilities, agreeing to reduce consumption during peak grid stress in exchange for more flexible access the rest of the time. A win for the grid, the community, and Google.
What this means for founders: The unglamorous work inside the data center for power, cooling, transmission, and workload orchestration is wide open. Hyperscalers, chip makers, cloud providers, and enterprise data centers are all looking for this. You don’t have to win Google to win big.
Bottom line: Power and cooling are the gate everything else passes through.
Amin pointed out that the most expensive part of AI computation is not the compute. It is moving the bits.
SRAM, HBM, DRAM, SSDs, and the networks connecting them are all becoming constraints at once. Most people use the wrong mental model: optimize the chip, optimize the compute cycles. The right question is how to get data to the compute unit as quickly and efficiently as possible.
The next frontier is memory architecture: lower latency, 3D stacking, direct GPU-to-storage connectivity, and fundamentally rethinking how compute connects to memory.
Sajiv laid out where TSMC is already pushing: logic-on-logic stacking, logic-on-DRAM stacking, and co-packaged photonics, swapping electrons for photons at the packaging level. Amin went further, calling for radical micro-architectural reinvention. His framing on 3D stacking: if you can place data across three dimensions instead of two, the physical distance the data has to travel drops dramatically. He pointed to architectures with no HBM at all as an early, provocative signal that the field is willing to rethink from first principles.
The memory industry, Santosh noted, has historically been cyclical, with three major players wary of overbuilding that could create a glut. While demand is rising rapidly, fab capacity is growing much more slowly. By the time supply catches up to today’s demand, demand will have moved further.
What this means for founders: if you have techniques to unlock more from existing memory, or alternatives to HBM, or approaches that increase fab efficacy, build them.
Bottom line: The bottleneck moved from the chip to the data path. Build for the data path.
The whole game in AI infrastructure, Santosh said, is making every chip in a rack, in a row, in a data center behave like a single chip.
Google just announced a hundred-thousand-chip non-blocking Virgo cluster. Amin described the ICI interconnect behind Google’s individual TPU supercomputers: 9,600 TPUs arranged in a three-dimensional torus, with switching silicon integrated directly into each TPU and shared memory across the entire pod at terabytes per second of aggregate bandwidth. The scale is hard to internalize until you hear the specifics.
Network planning has to work like highway planning. Build two lanes where you needed four, and you’ll be tearing it up later at enormous cost and disruption. The right move is to provision for the most demanding, latency-sensitive workloads, because retrofitting a network is as painful as retrofitting a highway.
Co-packaged optics (CPO) has been a year or two out for years. CPO is difficult to deploy, maintain, and service, which is exactly why it hasn’t happened yet and exactly why there is room for someone to get it right. The device-level innovation at the rack is finally arriving.
What this means for founders: within-rack, in-datacenter, and cross-datacenter networking are all open to disruption. Any optical breakthrough will likely be the industry’s biggest shift.
Bottom line: Copper capped out. The optical company that nails deployment captures the industry’s biggest shift.
A year ago, training accounted for most of the infrastructure. Today, Amin puts it at roughly 50/50 between training and inference, driven by autonomous agents running continuously across massive stacks. Santosh’s long-term call: 75-80% of deployed chips will eventually run inference, not training. Most companies are doing more training than they need. When they find product-market fit, the mix shifts.
Google just released two specialized TPUs at once for the first time, one optimized for inference and one for training. Same programming model, different hardware. It is the first time the tradeoff has been clear enough to justify separate silicon.
Sajiv added the TSMC view: by 2030, the semiconductor industry should reach $1.5 trillion, with AI and high-performance computing accounting for about 55 percent of that. The edge is growing too, in smartphones, cars, smart glasses, and humanoids, but infrastructure will be the dominant share of silicon demand for the foreseeable future.
What this means for founders: general-purpose compute is giving way to workload-specific architectures. The more specialized the hardware, the more efficient for its target workload, a principle that made no economic sense fifteen years ago and is now table stakes.
Bottom line: Inference is becoming the main event. Specialized silicon and a credible small first deployment matter to hyperscalers.
Sajiv described that TSMC has fewer people in the fab nowadays. Inspection, maintenance scheduling, process optimization, multi-chip platform planning – all AI-driven. What once required a human standing in front of a machine now has one technician monitoring twenty pieces of equipment from a dashboard. AI is not just the thing being manufactured. It’s how the manufacturing happens.
On the org side, Santosh described what Meta is already experimenting with: smaller, AI-native teams in which the emerging power role is not a PM or an engineer. It is the AI Captain. The AI Captain’s full-time job is to remove friction: privacy reviews, approvals, process bottlenecks, and anything that keeps the team from operating autonomously.
What this means for founders: your customers are already restructuring how they work. When you pitch, the product is table stakes. What wins the deployment is ease of integration, smooth operations, and the confidence that you will be there when something breaks.
Bottom line: The product gets you in the room. Operational trust wins the deployment.
Silicon is back in Silicon Valley. The physical world is the constraint, and the AI infrastructure gold rush in power, cooling, memory, and networking is happening right now.
The hyperscalers are spending hundreds of billions of dollars and telling you directly what they cannot solve fast enough on their own.
My key takeaways for founders building in the AI infrastructure layer:
Start building for the physical world.
