Jensen Huang's "AI Factory": How Is It Different from a Data Center? [Tech Talk]

AI Factory: Data Centers Designed for Large-Scale Deployment

Nearly Equal Ratio of CPUs to GPUs

Facilities Specialized for Supporting AI Fine Tuning

Editor's NoteThis series aims to explain essential but often unfamiliar technologies—ranging from AI and semiconductors to telecommunications and biotech—in an accessible and easy-to-understand manner.

Jensen Huang, CEO of NVIDIA, revealed plans to build an “AI Factory” during his recent visit to Korea. If realized, this initiative could attract data center investments worth hundreds of trillions of won over the next five years.

Jensen Huang, CEO of NVIDIA, is distributing chicken to citizens on the streets of Samseong-dong, Seoul. Photo by Jin-Young Kim

At first glance, an AI Factory does not appear very different from a regular data center, as both facilities are built with similar hardware such as graphics processing units (GPUs), central processing units (CPUs), and memory. However, there is a key distinction between an AI Factory and the computing facilities that big tech companies have established so far. While AI infrastructure until now has focused on research and development, the AI Factory is designed from the outset for large-scale deployment.

What sets the AI Factory apart? Differences in hardware configuration ratios

The AI Factory is a data center solution ambitiously designed by NVIDIA. According to its official website, its purpose is to “accelerate full-stack AI infrastructure and software,” and to “enable deployment of AI.”

An NVIDIA engineer adjusting a server rack. NVIDIA

Because of this, the AI Factory differs from typical AI data centers in its “computer chip configuration ratio.” Up until now, AI data centers have mainly served as training facilities. In other words, their primary function has been pre-training AI models with massive datasets. This required far more GPUs than CPUs to accelerate the training process. It was common to see data centers with four to eight GPUs for each CPU.

In contrast, the AI Factory is oriented toward deployment, with a focus on running AI agents. NVIDIA’s next-generation Rubin platform for AI Factories offers a more balanced ratio, with one CPU for every one or two GPUs. NVIDIA even sells server racks composed solely of Vera CPUs, specifically for running AI agents.

Turning AI into specialized experts... Data centers specialized for “fine tuning”

Another distinguishing feature of the AI Factory lies in its software capabilities. The AI Factory is a data center specialized in supporting fine tuning. While typical large language models (LLMs) are trained on vast datasets and are generalists by design, they struggle to perform well in fields that require highly specialized knowledge. Therefore, fine tuning is essential to optimize AI for specific professional domains.

NVIDIA's next-generation Vera Rubin platform's core components: Vera CPU, Rubin GPU, and other chipset components. NVIDIA

For example, medical AI needs access to sensitive patient data available only at certain hospitals in order to provide reliable medical advice. Similarly, legal AI and manufacturing AI must be trained on case law databases and manufacturing process data, respectively. Fine-tuned AI models are trained on “proprietary data” held by specific companies or experts.

This is one of the reasons why CEO Huang has expressed optimism about Korea becoming a “physical AI hub.” Korea’s advanced manufacturing sector has accumulated vast amounts of images and numerical data through repeated production processes. By processing this data into high-quality datasets and training them in the AI Factory, it will be possible to develop AI capable of handling a wide variety of manufacturing tasks in the future.

Meanwhile, on June 8, CEO Huang, together with SK hynix, SK Telecom, and SK Group, announced plans to build an AI Factory with a capacity of up to 5 gigawatts (GW). It is estimated that each AI Factory requiring 1 GW of power will cost approximately $60 billion (about 91 trillion won) in facility investment. Completing this plan would require up to $300 billion (about 457 trillion won) in total capital expenditure.

This content was produced with the assistance of AI translation services.