Despite No Physical Chip, $350 Million Deal Struck...Samsung Also Unveils Next-Gen Semiconductor [Tech Talk]

Anthropic Moves to Secure Next-Generation In-Memory Chips
Solving AI’s Core Bottleneck: Data Movement
Eliminating the Gap Between Memory and Compute Cores

Samsung Accelerates Next-Generation PIM Development
Unveils Breakthrough Technology at Hot Chips

Editor's NoteThis series simplifies and explains technology stories—essential but often unfamiliar—that range from AI and semiconductors to telecommunications and biotech, making them more accessible for readers.

According to the US IT media outlet The Information in May, Anthropic began discussions to purchase 250 million dollars’ worth (about 350 billion won) of chips from the semiconductor startup Fractal. The challenge is that these chips only exist at the design stage, with physical units expected to be produced next year. It is exceedingly rare for a big tech company to enter into a multi-billion-won contract for a product that has not even undergone basic performance verification.

The chips Anthropic is eager to secure are so-called "in-memory" semiconductors—a new concept attracting attention as a potential remedy for the bottleneck between computer memory and computation, which remains a core obstacle for today’s artificial intelligence (AI) industry.

Fractal's Enterprise Value Soars After Anthropic Deal

Dario Amodei, CEO of Anthropic. Reuters-Yonhap News

Fractal is a startup founded in 2022 by Walter Goodwin, who holds a Ph.D. from the University of Oxford, and has a workforce of just over 100 employees. Nevertheless, after news broke of Anthropic’s purchase agreement, Fractal’s enterprise value surged to 6.5 billion dollars (about 9 trillion won), and recent reports indicate it is preparing to raise another substantial round of investment totaling 600 million dollars (about 830 billion won).

The actual chip developed by Fractal has yet to be revealed, with physical shipments anticipated next year. This means Anthropic purchased billions of won worth of semiconductors before their performance could be validated. Typically, when vast resources are poured into ventures such as data centers, the core computer chips are meticulously measured for performance. This process is known as "Proof of Concept (PoC)." For an advanced company like Anthropic to skip PoC and purchase chips outright is an unprecedented move.

In-Memory Chips: The AI Bottleneck Breaker

Walter Goodwin, founder and CEO of Fractal. An engineer who earned his Ph.D. from the Robotics Institute at the University of Oxford, he gained attention by revealing that the in-memory chip developed by Fractal achieved inference speeds 25 times faster at a cost 10 times lower than conventional semiconductors. Fractal Homepage

The chip that Anthropic is taking a risk to secure early is referred to as "in-memory" semiconductor. In-memory architecture means that a computer’s memory and compute units are essentially merged as one.

Currently, the foundational semiconductors in AI computers are configured with memory and computation kept separate. Memory components such as DRAM, NAND flash, and solid-state drives (SSD) primarily serve as data storage, while computation units such as central processing units (CPU) and graphics processing units (GPU) handle actual processing tasks.

AI systems store massive model data in memory, then transfer it to computational units for processing. The bandwidth required to move vast amounts of data between memory and processors thus becomes a bottleneck. To address this, major domestic memory companies like Samsung Electronics and SK hynix have focused on producing high-bandwidth memory (HBM). However, these solutions are extremely costly, and challenges such as heat generation and power consumption are growing more acute—presenting ongoing headaches for AI companies.

However, if memory and computation units could be truly merged as one, eliminating the physical time required to move data, there would be no need to rely on HBM. AI computers could scale far more efficiently. This would especially enable unprecedented levels of cost effectiveness in AI inference operations—something previously unthinkable.

Race for Next-Generation Semiconductor Tech...Samsung Joins 'PIM' Development

Conceptual diagram of 'PIM (Processing In Memory)' unveiled by Samsung Electronics. Samsung has been researching in-memory computers that combine memory and compute units for several years. Samsung Electronics website

However, achieving the fusion of memory and computation is extremely difficult. Integrating both memory and compute cores within nanometer-scale circuits requires an innovation that overturns the traditional semiconductor design paradigm. Walter Goodwin, CEO of Fractal, claims to have overcome this challenge through geometric innovations in chip topology, but whether such a small startup has truly beaten the industry giants to fully realize in-memory technology remains to be seen.

It is highly likely that Anthropic is aggressively preempting the "ideas" of innovative chip companies like Fractal. Even if other startups have not progressed as far as Fractal, the number of companies in the US and UK working on in-memory chips is clearly growing. Should any one of them succeed, the landscape of the AI race could change dramatically—making the multi-billion-won early investment a risk worth taking for Anthropic.

Major semiconductor manufacturers are also eager to develop in-memory technology. For several years, Samsung Electronics has been developing a new memory device concept called "PIM (Processing In Memory)." This chip inserts a compute core inside Samsung’s signature DRAM, enabling the memory itself to process data. Last month, Samsung surprised engineers at "Hot Chips," the world’s largest semiconductor conference held in the United States, by unveiling PIM, drawing significant attention from industry professionals.

This content was produced with the assistance of AI translation services.