Nvidia Uses "Modular Data Centers" to Control Its Ecosystem... "An Opportunity" for Samsung and SK hynix [Chip Talk]

Vera Rubin-Based 'DSX POD' Unveiled
Turnkey Supply Aims to Counter Big Tech's In-House Chips
'Custom HBM' Puts Samsung and SK hynix to the Test

Nvidia has unveiled a “modular data center” offering that it pre-validates as a finished product at its factories before delivery. The plan is to cut the time needed to build infrastructure based on its next-generation Vera Rubin architecture from six months to as little as 14 days. As Nvidia’s strategy to expand its ecosystem and control the entire data center through a turnkey supply model, beyond just providing AI chips, takes shape, the calculations of domestic semiconductor companies such as Samsung Electronics and SK hynix are becoming more complex.

Nvidia CEO Jensen Huang delivers a keynote address at GTC Berlin in Berlin, Germany, on the 21st (local time). Yonhap News.

Finished Data Centers Delivered Directly...What Is Jensen Huang's Aim?

According to semiconductor industry sources and foreign media reports on October 31, Nvidia recently unveiled the “DSX POD,” a next-generation design standard for modular data centers, at GTC Berlin, a GPU (graphics processing unit) technology conference held in Berlin, Germany. Assuming that the necessary infrastructure for power, cooling, and networking is in place, the design can cut on-site installation time from six months to as little as 14 days. Nvidia aims to begin deliveries early next year.

Until now, building AI data centers has taken months, even when the latest GPUs and high-bandwidth memory (HBM) were secured, because racks had to be assembled on-site and complex liquid-cooling pipes and high-speed networks had to be tested. With heat generation and power consumption rising explosively, on-site installation has been a major bottleneck to expanding AI infrastructure. Nvidia's strategy is to address this bottleneck by bundling chips, racks, cooling modules, and networking at its factories, fully validating them in advance, and then shipping them as a single unit.

This represents a fundamental shift in the data center design paradigm itself. Lee Jong-hwan, a professor in the Department of System Semiconductor Engineering at Sangmyung University, said, “Whereas conventional data centers were ‘building-centric,’ focused on land and civil engineering, modular facilities are designed to create high-density, GPU-centric AI factories. They can rapidly deliver the ultra-high-efficiency infrastructure that big tech companies want, so there is a high likelihood that the center of gravity in the data center market will shift substantially toward modular facilities.”

Industry observers believe Nvidia is using modular systems to counter the shift by big tech companies toward their own chips (ASICs). If Google, Meta, Amazon, and others develop their own accelerators and build their infrastructure directly, it would take substantial time and money to move from design to deployment. By contrast, adopting Nvidia's pre-validated DSX POD could enable operations to begin in just two weeks, keeping big tech companies within Nvidia's ecosystem.

Professor Lee said, “Even if big tech companies develop their own chips, in practice they cannot stop using high-performance GPUs altogether. If modular systems increase the speed and efficiency of GPU server deployment, customers will use fewer of their own chips while maintaining the share of GPUs, creating a kind of countervailing effect.”

It could also be an attractive alternative for “neocloud” providers, which rent out small- and medium-scale AI computing infrastructure and face challenges building hyperscale data centers spanning thousands to tens of thousands of pyeong. Existing data centers can also immediately boost performance by replacing only specific modular units, without shutting down the entire facility, making the system well-suited to attracting demand for partial upgrades, observers say.

From Off-the-Rack to Custom-Made...Challenges and Opportunities for Korea's Memory Industry

As Nvidia strengthens its turnkey model of bundling and selling entire server racks and modules, the market environment facing domestic memory companies such as Samsung Electronics and SK hynix is changing rapidly. The shift from supplying HBM in standardized formats to “custom HBM” optimized for system modules is expected to accelerate further.

First, the physical constraints within modules may make memory specifications more demanding. An industry official said, “Because modular systems have clear limits on power, space, and heat, they may require detailed tuning that maintains bandwidth while reducing power consumption and heat, rather than simply prioritizing maximum output.” There are also concerns that Nvidia may propose a standardized base-die specification to memory manufacturers, instead of using their own foundry dies, or require uniform specifications to ensure consistent quality across finished modules.

“The custom memory market is opening in earnest as module-level supply converges with demand tailored to individual big tech companies,” the official said. “Ultimately, how precisely companies meet Nvidia's system validation criteria and module specifications will be the key battleground in the competition for leadership in HBM and next-generation memory.”

Some observers say these changes could create new opportunities for Korea's semiconductor ecosystem. As the number of modules equipped with high-performance GPUs grows, demand for next-generation HBM is set to surge. The benefits could also spread to adjacent parts of the value chain, including advanced packaging, power semiconductors that reduce power waste, and liquid-cooling equipment.

Professor Lee said, “The spread of modular data centers means more high-performance GPU servers, inevitably accompanied by growing demand for next-generation HBM and advances in high-end packaging to support them. In addition, the areas in which domestic companies can benefit will expand across related infrastructure, including power semiconductors that maximize power-conversion efficiency and cooling equipment.”

This content was produced with the assistance of AI translation services.