SK hynix: "Data Movement Between Memory Tiers" a New Bottleneck... 3D Stacked DRAM, HBF to Address It

SK hynix President and Head of Development Ahn Hyun delivers keynote at SEDEX 2026
"AI infrastructure to grow nearly 20-fold by 2030; next-generation memory tiers beyond HBM are essential"
"We will lead standardization with global par

SK hynix said the growing demand for artificial intelligence (AI) inference is creating new bottlenecks as data moves between tiers of memory, and identified 3D Stacked DRAM and High Bandwidth Flash (HBF) as key next-generation technologies to address the problem.

Ahn Hyun, president and head of development at SK hynix, speaks on “The Evolution of Memory Technology in the AI Era” during the keynote address at the 28th Semiconductor Expo (SEDEX 2026), held at COEX in Seoul on the 14th. Photo by Kwon Hyunji

Ahn Hyun, president and head of development at SK hynix, made the remarks during his keynote address on "The Evolution of Memory Technology in the AI Era" at the 28th Semiconductor Expo (SEDEX 2026), held at COEX in Seoul on October 14.

Ahn said the AI market is not stagnating but accelerating at an even faster pace. In particular, with the emergence of agentic AI, the number of tokens required by agents is surging from 15 times to more than 100 times the number generated by humans. He forecast that the scale of AI infrastructure in 2030 will be nearly 20 times larger than this year.

The biggest bottleneck emerging in this process is "data movement between memory tiers." Ahn said, "Context length, which was only about 128K as recently as 2024, has recently grown to around 1 million tokens and will increase even more rapidly." He also pointed to a trend in which context length is expanding to around 100 million (100M) tokens. As a result, existing high-bandwidth memory (HBM) and DRAM capacities are no longer sufficient to hold all the data, causing a "spillover" of data onto solid-state drives (SSDs).

"The performance gap between memory tiers, from SRAM to HBM, DRAM and SSD, and the movement of data between them have now become new bottlenecks," Ahn said. "To overcome the limitations whereby increasing performance reduces capacity and increasing capacity lowers performance, we must continue to create entirely new memory tiers."

SK hynix presented new memory tiers, including 3D Stacked DRAM and HBF, as innovative technologies to address these memory bottlenecks. 3D Stacked DRAM involves vertically stacking DRAM on logic semiconductors such as graphics processing units (GPUs) or neural processing units (NPUs) and connecting them directly using through-silicon vias (TSVs). Compared with existing HBM, the technology can increase the number of input/output (I/O) pathways dozens of times, boosting data transfer speeds and significantly reducing power consumption.

HBF is an ultra-high-capacity memory that applies to NAND flash the TSV-stacking expertise SK hynix has developed in producing HBM. The company is developing it with the goal of achieving more than 10 times the capacity of HBM, while offering about half its bandwidth. Ahn said that AI inference involves reading already-stored data hundreds of thousands of times more often than writing data. Based on this, he emphasized that NAND-based HBF, which is well suited to read operations, could become a key alternative for large-scale AI infrastructure.

"We are already working to standardize HBF in collaboration with global partners such as SanDisk and major cloud service providers (CSPs) such as Google," Ahn said. "We will finalize the specifications next January and proactively build a global ecosystem." He also shared updates on pooled memory based on Compute Express Link (CXL) and "Storage-Next," which increases storage speed by four to eight times.

Meanwhile, SK hynix set up the largest booth at SEDEX 2026, unveiling a broad range of customized memory solutions for applications from AI infrastructure to on-device AI. The exhibition space, themed "An AI Journey That Never Stops Moving Forward" and designed in the shape of a racetrack, featured models of HBM and HBF structures as well as exhibits on the history of HBM development. An NVIDIA Vera Rubin superchip and an HBM wafer signed by NVIDIA CEO Jensen Huang were also on display, showcasing the partnership between the two companies.

In server DRAM, the company showcased a 64GB RDIMM using a 10-nanometer-class sixth-generation (1c) process for the first time in the world, the industry's highest-capacity 256GB 3DS RDIMM, and a high-speed 128GB MRDIMM. In enterprise SSDs (eSSDs), it displayed products supporting direct liquid cooling (DLC), as well as the PS1101, the company's first quad-level cell (QLC)-based enterprise SSD. In the PC and mobile segments, it also unveiled a range of next-generation solutions targeting the AI ecosystem, including the next-generation CSODIMM module, GDDR7, LPDDR6 and UFS 5.0.

This content was produced with the assistance of AI translation services.