"If You Can't Ship Without Trimming Down, It's All for Nothing"... AI Giants Ditch 'Top Performance' for Forced HBM Diet [Chip Talk]

AI Giants Forced into 'HBM Diet' Amid Memory Shortage
NVIDIA Considers Downgrading Rubin Ultra's HBM Specs
"Memory Supply Now Determines Final AI Accelerator Specifications"

An unexpected phenomenon is unfolding amid the increasingly fierce global race for artificial intelligence (AI) performance. The world's leading AI accelerator manufacturers are considering a so-called "despec" strategy, which entails lowering the memory specifications of their next-generation products. Analysts say that as memory shortages intensify, the era is shifting to one in which the physical supply of memory, rather than pure design intent, is now the determining factor in the final specifications of AI semiconductors.

Jensen Huang, CEO of Nvidia, introduced the Vera Rubin Architecture during his keynote speech at a side event of the Computex Trade Show held in Taipei, Taiwan on June 1. Photo by Reuters Yonhap News

According to industry sources on August 8, Nvidia recently revised down the high-bandwidth memory (HBM) specification for its next-generation AI accelerator "Rubin Ultra" from the initially planned 16-stack HBM4E to 12 stacks, and is now even considering options such as an 8-stack HBM4E, or adopting 8- or 12-stack HBM4 as in the previous model. AMD is also reportedly likely to release its next-generation accelerator MI400 in both 8-stack and 12-stack HBM4 versions.

Until now, the AI semiconductor market has operated on the basic premise of a cutthroat race for "higher bandwidth" and "more memory stacks." Even Nvidia maintained its baseline design for the Rubin Ultra at 12-stack HBM4E through the first half of this year.

However, the recent supply shortage throughout the global memory semiconductor market has now reached even the most advanced HBM segment, causing a dramatic turnaround. No matter how high-performing a graphics processing unit (GPU) is designed, the sheer lack of HBM die available for installation is creating a bottleneck that blocks product shipments. As a result, it appears that global big tech companies are shifting their approach, opting to maximize product output—even if it means partially compromising on performance.

Market research firm TrendForce explained, "Due to DRAM shortages expected next year, wafer capacity that memory manufacturers can allocate for HBM production is severely limited. Moreover, the verification schedule for 12-stack HBM4E and uncertainty regarding yield ramp-up for mass production have combined to prompt key players in the AI chip ecosystem to begin lowering memory specifications."

Nvidia faces a delicate calculation between technical bandwidth and GPU shipment volumes. If verification for 12-stack HBM4E is completed on time and mass production succeeds, the Rubin Ultra's input/output (I/O) speed could reach 14–16Gbps, far surpassing the previous generation's 8–11.7Gbps. Conversely, if Nvidia lowers its sights to optimize for HBM4 as an alternative, the I/O speed will remain at only 11–12Gbps.

One key reason for seriously considering lower specs is that reducing the number of HBM stacks per GPU dramatically boosts the total number of GPUs that can be released to the market.

Injun Son, a researcher at Eugene Investment & Securities, said, "Reducing the number of memory stacks is a practical choice to produce as many accelerators as possible with the limited HBM supply. For the same amount of DRAM material input, assembling with 12 stacks instead of 16 increases HBM output by about 33%, and reducing from 12 to 8 stacks provides an approximately 50% boost." He added that fewer stacks also lead to lower defect rates (yield increase) and shorter manufacturing lead times.

The "despec" trend due to memory shortages is now spreading beyond HBM to the entire AI infrastructure. According to TrendForce, major cloud service providers (CSPs) and server original equipment manufacturers (OEMs) already reduced DRAM module (RDIMM) capacity in their server configurations during the first half of this year. In June, a report by global semiconductor research firm SemiAnalysis revealed that Nvidia was slashing the amount of low-power DRAM (LPDDR5X) used in its upcoming AI computing platform, the Vera Rubin NVL72, by 50% from 54TB to 27TB.

Despite the three major memory firms—Samsung Electronics, SK hynix, and Micron—announcing accelerated factory completion schedules and expanded capital expenditure plans, most forecasts say the memory shortage will only worsen next year. This dynamic is expected to strengthen the pricing power of memory suppliers in the ongoing HBM price negotiations for the coming year, potentially boosting their earnings.

Jaejun Kim, Executive Vice President and Head of Memory Strategy & Marketing at Samsung Electronics, recently said during the company’s second quarter earnings conference call, "We anticipate that third-quarter HBM4 sales will more than triple compared to the previous quarter. In the second half of this year, HBM4 sales are expected to comfortably account for over 60% of Samsung Electronics’ total HBM revenue." This underscores the company's goal of raising its HBM market share to a level comparable to its DRAM market share by the end of this year.

Analyst Son estimates that as the expansion of advanced foundry and packaging capacity outpaces HBM supply, the competition among AI accelerator manufacturers to secure memory will only intensify. He projects next year’s HBM operating profits for Samsung Electronics and SK hynix to skyrocket year-on-year by 539% and 286% to 85 trillion won and 89 trillion won, respectively.

An industry insider commented, "Whereas GPU or processor computing speeds previously drove memory adoption, we are now shifting to an era in which the physical supply of memory determines the final specifications of AI semiconductors."

This content was produced with the assistance of AI translation services.