Professor Kim Jeong-ho, known as the "Father of HBM" from the Korea Advanced Institute of Science and Technology (KAIST), has presented a paradigm-shifting view: the essence of artificial intelligence lies in memory technology, not the graphics processing unit. In a recent video interview, Professor Kim from KAIST's School of Electrical Engineering provided a comprehensive overview of HBM's evolution, the AI computing landscape, and the future architecture of semiconductors. Professor Kim, who collaborated with SK hynix on the initial HBM1 development in the early 2010s and has since spearheaded foundational research, shared insights that are resonating across technology and investment circles, highlighting a structural tension in the current AI compute race.

Key Insight: GPU Idle Time

Professor Kim offered a startling statistic: "Even with a million GPUs installed, they are only actively computing for about 10% of the time." He explained that for every word ChatGPT generates, the system must read data from HBM, perform calculations, and write it back. "Reading and writing consume almost all the time, leaving the GPU idly waiting," he stated. Even with algorithmic optimizations, GPU utilization struggles to exceed 30%. This observation underpins his core thesis: "AI equals Memory."

GPU's Limitations and the Rise of Memory

Professor Kim offered a pointed assessment of NVIDIA's situation, suggesting recent activities by its CEO indicate underlying concerns. "My judgment is that the technical growth of GPUs is nearly stalled," he said. "The evolution of AI computers is now in the hands of memory." He argued GPUs face a physical bottleneck: to boost performance, they need more area for compute units, but heat dissipation requires a bulky heatsink on the back, preventing the vertical stacking that is possible with memory. "The GPU feels stuck in a dead end," he remarked. As the focus shifts from training to inference, memory is being revalued. "In the inference era, what matters is how much data you can cram into the AI, and the semiconductor that determines this is memory," he asserted. He further contends that the competition in AI capability is ultimately a competition in memory capability, suggesting that the performance of models from Google's Gemini, OpenAI, and Anthropic's Claude is dictated by memory.

The Dual Pillars of HBM: Capacity and Bandwidth

Professor Kim defined HBM's value through two dimensions. First is capacity, driven by context engineering, multimodal inputs, and agentic AI, leading to memory demand doubling annually—a 1000-fold increase over a decade. With physical scaling nearing quantum limits, the only path forward is "stacking up." Second is bandwidth. He likened traditional memory to an 8-lane highway, while HBM is a 1024-lane road, now moving to 2048 lanes, with the potential for millions of lanes in the future. This massive parallel data transfer is essential to match AI's computational speed.

HBF: The Stacking Era for NAND Flash

While HBM addresses speed, capacity remains a constraint. Professor Kim detailed the next technological path: HBF (High Bandwidth Flash), which involves vertically stacking NAND Flash memory like HBM. DRAM is fast but limited in capacity; NAND Flash offers large capacity and persistent storage, with speed sufficient for "cold data" in inference. He envisions a future where HBM and HBF coexist, forming a complex "city" of memory that supplies data to the GPU. "We are in the HBM era now, but in 10 years, the market demand for NAND Flash and HBF will surpass that of HBM," he predicted, urging Samsung Electronics Co Ltd and SK hynix Inc to prepare. He noted that companies developing HBF include SK hynix, Western Digital Corp (through SanDisk), Samsung, and Japan's Kioxia Holdings Corp.

HBS: A Forward-Looking Third Path

Professor Kim also proposed HBS (High Bandwidth SRAM), a more advanced concept. SRAM is about 1000 times faster than DRAM but has lower density and higher cost. His idea is to fabricate entire 12-inch wafers with SRAM and stack them 12 to 16 layers high, expanding capacity from 100GB to 1600GB. "Then it makes sense—speed is 1000 times faster, and capacity is sufficient," he said. He described the ultimate AI chip as a "100-story 3D building" with layers of HBM, HBF, and HBS, with the GPU on top for cooling. The primary engineering challenge, he noted, is not computation but power delivery and thermal management, which will become a core competitive differentiator.

Custom HBM and Shifting Market Dynamics

The advent of HBM4 is changing supply-demand dynamics, Professor Kim explained. Moving from standardized products to "Custom HBM" tailored for accelerators from NVIDIA, Google, and Advanced Micro Devices Inc (AMD) means memory makers require long-term agreements before development. "AI companies are so desperate for high-performance HBM that they are queuing up. Suppliers are starting to set prices—this is a paradigm shift," he said. He also anticipates future HBM chips incorporating communication functions, allowing "HBM modules to talk to each other," further elevating the strategic position of memory manufacturers.

Samsung and SK hynix's Unique Position

Professor Kim emphasized that globally, only Samsung and SK hynix can mass-produce both DRAM (for HBM) and NAND Flash (for HBF). "SanDisk and Kioxia, despite their soaring stock prices, can only make HBF, not HBM. Samsung and SK hynix possess the most powerful tools to lead the future," he stated. When asked about forecasts of combined operating profits reaching 500-600 trillion won for the two Korean giants this year, he called it "realistic," noting increased engagement with their executives, though he acknowledged competitive pressure from Micron Technology Inc and Western Digital securing orders from NVIDIA and Google.

AI PCs and Smartphones: Memory Dictates Price

Professor Kim extended the memory narrative to end-user devices. He predicted that future AI PCs requiring true personal AI computation could cost around 10 million won, with memory determining the price. For AI smartphones priced between 3-5 million won, memory could account for 2-3 million won of the cost. "The continuous evolution of AI infrastructure and models requires ever more memory. AI PCs and AI smartphones are another major axis of this trend," he said.

Exploding Demand from Agentic and Physical AI

Looking ahead, Professor Kim believes the advent of Agentic AI and Physical AI (embodied AI) could increase memory usage by about 1000 times compared to today. "AI agents work 24/7, unlike humans who sleep, so workloads explode, and memory demand naturally follows. That will be the era not of HBM, but of 'Ultra HBM,'" he concluded.