[Chip Talk] "Slim Down to Ship"... AI Giants Forced into an 'HBM Diet' Amid Memory Shortage

Nvidia Considers Lowering HBM Specs for Rubin Ultra

"Memory Supply Becomes Decisive Factor for Final AI Accelerator Specs"

An unexpected phenomenon is unfolding amid the increasingly intense global competition for artificial intelligence (AI) performance. The world’s leading AI accelerator manufacturers are reportedly considering a so-called 'despec' strategy—deliberately lowering the memory specifications for their next-generation products. As the shortage of memory supply intensifies, analysts say the era is shifting: the physical supply of memory, rather than technical ambitions, is increasingly deciding the final specs of AI semiconductors.

As of August 8, industry sources say Nvidia has downgraded the initially planned high-bandwidth memory (HBM) for its next-generation AI accelerator, 'Rubin Ultra', from a 16-stack HBM4E to a 12-stack version and is even considering options with 8-stack HBM4E or adopting previous-generation HBM4 in 8-stack and 12-stack configurations. AMD is also expected to launch its upcoming MI400 accelerator with both 8-stack and 12-stack HBM4 variants.

Until now, the AI semiconductor market has been defined by fierce specification battles emphasizing 'wider bandwidth' and 'greater number of memory stacks.' Even Nvidia maintained a default design with a 12-stack HBM4E for Rubin Ultra through the first half of this year.

However, the severe supply shortage rippling through the global memory chip industry has now reached the cutting-edge HBM sector, upending the situation. No matter how powerful the graphics processing unit (GPU) design may be, the absolute shortage of available HBM dies has created a bottleneck, stalling product shipments. In response, global Big Tech firms appear to be pivoting their strategies—preferring to maximize shipment volumes, even if it means some compromise on performance.

According to market research firm TrendForce, "Due to persistent DRAM shortages next year, memory manufacturers face a hard cap on the amount of wafer capacity they can allocate to HBM." The company added, "Adding to this, uncertainties in the verification and mass-production ramp-up schedule for 12-stack HBM4E have driven major AI chip ecosystem players to begin lowering memory specifications."

Nvidia’s dilemma is a delicate trade-off between technical bandwidth and overall GPU shipment volume. If verification for 12-stack HBM4E is completed on schedule and mass production succeeds, Rubin Ultra’s input/output (I/O) speeds could sharply exceed those of the previous generation (8–11.7Gbps), potentially reaching 14–16Gbps. On the other hand, switching to a more optimized HBM4 design would mean settling for I/O speeds around 11–12Gbps.

Nevertheless, the reason the reduction in specification is being seriously considered is that by decreasing the number of HBM stacks per GPU, the total number of GPUs supplied to the market can increase dramatically.

Injun Son, a researcher at Eugene Investment & Securities, explained, "Reducing the number of memory stacks is a pragmatic choice to maximize the number of accelerators produced under tight HBM supplies. For the same amount of DRAM material, building 12 stacks instead of 16 results in about a 33% increase in HBM output, while dropping from 12 stacks to 8 yields an approximately 50% increase." In addition, lower stack counts reduce defect rates (yield) and shorten processing times (lead times).

This 'despec' trend, driven by memory supply strain, is now spreading beyond HBM to impact all of AI infrastructure. TrendForce reports that major cloud service providers (CSPs) and server OEMs already cut back RDIMM DRAM module capacity in their server configurations during the first half of this year. In June, global semiconductor research firm SemiAnalysis reported that Nvidia plans to halve the amount of low-power LPDDR5X DRAM in its next-gen AI computing platform, 'Vera Rubin NVL72,' from 54TB to 27TB.

In response, the 'Big 3' in memory manufacturing (Samsung Electronics, SK hynix, and Micron) have accelerated facility completion schedules and announced new capital expenditure plans. Still, most forecasts indicate that the memory shortage will worsen in 2027 compared to this year. This is strengthening the memory suppliers' pricing power in ongoing negotiations for next year's HBM contracts and is expected to buoy their financial results.

Jaejun Kim, Executive Vice President and Head of Memory Strategy Marketing at Samsung Electronics, recently stated during the second quarter earnings call, "We expect HBM4 revenue in the third quarter of this year to be more than triple that of the previous quarter," adding, "By the second half of this year, HBM4 will comfortably account for more than 60% of Samsung’s total HBM revenue." This underscores Samsung’s aim to match its HBM market share to its DRAM market share as early as the latter half of 2026.

Researcher Son estimates that, as the pace of advanced foundry and packaging expansion outstrips HBM supply, memory procurement battles among AI accelerator manufacturers are intensifying. He projects Samsung Electronics’ and SK hynix’s HBM operating profits next year will skyrocket by 539% and 286%, respectively, reaching 85 trillion won and 89 trillion won compared to last year.

An industry insider assessed, "If, until now, GPU or processor computational speed has driven memory adoption, the initiative is now shifting to an era where the physical availability of memory determines the final specifications of AI semiconductors."

This content was produced with the assistance of AI translation services.