The Unexpected Rise of Legacy GPUs in the AI Era

Enduring Value: The Longevity of Older GPU Generations in AI

Despite the rapid advancements in AI hardware, not all AI servers are exclusively powered by the newest chips. CoreWeave's co-founder and chief development officer, Brannin McBee, recently shared insights on this trend, noting that older-generation GPUs are still actively and profitably utilized in the market. This indicates a broader application spectrum for AI compute, extending beyond just the latest and most powerful offerings.

Profitable Operations: How Older SKUs Sustain AI Cloud Infrastructure

McBee elaborated that even GPUs from the late 2010s are integral to cloud infrastructure, proving that these older units are not operated at a loss. He pointed out a significant rise in pricing for Ampere architecture GPUs over the past year, while the latest Hopper chips maintained stable pricing. This suggests that the market for these older components is far from depreciating, showcasing their continued relevance and profitability.

Workload Diversity: Matching Compute Power to AI Tasks

The persistent demand for older GPUs, particularly those from the Ampere generation (Nvidia's RTX 30-series equivalent for servers), is largely driven by the varied requirements of AI inference tasks. Not every AI workload necessitates the most advanced computational power. Instead, the market efficiently matches specific AI tasks with the most suitable infrastructure, irrespective of its vintage. This strategic alignment ensures that older, yet capable, GPUs find their niche in the diverse AI landscape.

Sustained Demand: Ampere's Unexpected Resurgence

The Ampere generation, launched around 2020 before the major AI boom, has witnessed a remarkably robust demand. This contrasts with the rapid evolution seen with subsequent architectures like Hopper (RTX 40-series) and Grace Blackwell (RTX 50-series), and the upcoming Rubin server chips. The ongoing high demand for Ampere chips, alongside accelerating demand for newer infrastructure, highlights a comprehensive market need that transcends generational divides.

Market Dynamics: Supply Constraints and Rising Costs

The AI industry's substantial financial inflows continue to fuel an intense demand for GPUs, leading to significant price increases across the board, including memory prices. The market faces considerable supply constraints, turning the acquisition of any available chip into a strategic advantage. This is exemplified by situations such as China's capped access to Hopper chips and outright exclusion from Blackwell, underscoring that in a highly constrained market, even less powerful or older components are highly valued and sought after.