No one has benefitted from the scaling hypothesis quite like NVIDIA. On the back of an AI boom and GPU monopoly, they’ve become the fastest scaling hardware company in history–adding $2T of value in 13 months with SaaS-like margins. 

While the H100 generation likely represents peak pricing power (new B200s have lower margins and higher COGS), an immediate lack of alternatives means they’ll continue to print cash. 

The open question is long-term (>6yrs) durability. Hyperscalers (Google, Microsoft, Amazon, and Meta) are aggressively consolidating AI demand to become the dominant consumers of AI accelerators; while developing competitive, highly-credible chip efforts. 

Simultaneously, the sheer scale of compute needs has hit limits on capex, power availability, and infrastructure development. This is driving an enormous shift towards distributed, vertically-integrated, and co-optimized systems (chips, racks, networking, cooling, infrastructure software, power) that NVIDIA is ill-prepared to supply.

In this paradigm, NVIDIA can lose with the highest-performing GPUs; the implications will reverberate at every level of the AI stack–from fabs and semiconductors, to infrastructure, clouds, model developers, and the application layer. 

Demand Consolidation

NVIDIAs predicament has been driven by hyperscalers’ consolidation of AI workloads and accelerator demand–setting the stage for toothier custom silicon and evolving infrastructure requirements. 

Already, ~50% of NVIDIA’s datacenter demand is from hyperscalers, the other half comes from a large number of startups, enterprises, VCs, and national consortiums. 

That share is set to shrink–the tidal wave of startup spending on GPUs was a transient phenomenon to secure access in a fiercely competitive market. Today, most startups simply don’t have unusual control or infrastructure requirements and are better served by the cloud. 

As such, early purchases were ill-fated; this is borne out by low utilization and abysmal ROI’s for small/short term GPU rentals (usually offered by startups that over-provisioned and are forced to rent out at a loss). This is eerily reminiscent of dot com era startups defending costly server hardware as the world moved to the cloud.

Share

More fundamentally, and contrary to early expectations, the model rollout has aggressively consolidated around a few closed source APIs. Even open source and edge models are now the domain of the hyperscalers. Custom, small-midsize models trained with unique data for specific uses have struggled (e.g. Bloomberg GPT). Scaled frontier models can be cheaper while performing and generalizing better – especially with effective RAG and widely available fine-tuning. Thus, the value proposition for most companies training proprietary models is unclear. Going forward, demand from this long tail of buyers looks shaky; considerably consolidating NVIDIA’s revenue base.

Meanwhile, the smaller independent clouds (Coreweave, Lambda, Crusoe, Runpod etc) have very uncertain futures. NVIDIA propped these businesses up with direct investments and preferential GPU allocations in order to drive fragmentation and reduce their reliance on the hyperscalers. Yet, they face long term headwinds without the product variety, infrastructure, and talent to cross-sell and establish lock-in; forcing them to sell commoditized H100 hours. NVIDIA’s production ramp has eroded scarcity and attractive margins baked into initial assumptions, while undermining the “moat” of favorable allocation. These companies are also extremely leveraged on 3rd party demand, and have been relying on GPU-secured debt + heroic fundraising to expand fast enough to reach competitive economies of scale. It’s an open question if this will work, but things look ugly. The effects of lukewarm third party demand are already visible from the high availability and declining GPU/hour costs at small clouds. 

Price cuts have reduced costs by 40%+ since last year and show no signs of stopping. This is disastrous for the durability and unit economics of independent clouds. Currently, you can rent GPUs for $1.99/hour. At those prices providers are getting <10% ROE; if prices dip below ~