Professor Kim Jeong-ho of the Korea Advanced Institute of Science and Technology (KAIST), known as the "Father of HBM," has presented a perspective-shifting view: the essence of AI is memory, not the GPU.

In a recent video interview, Professor Kim Jeong-ho from KAIST's Department of Electrical Engineering provided a systematic overview of HBM technology evolution, the AI computing power landscape, and future semiconductor architecture. Credited as the "Father of HBM," Kim collaborated with SK Hynix on HBM1 development in the early 2010s and has since led foundational architectural research. His core arguments, now widely discussed in tech and investment circles, point to a structural contradiction in the current AI computing race.

Professor Kim presented a startling figure: "Even with a million GPUs installed, they are only actively computing for about 10% of the time." He explained that for every word ChatGPT outputs, the system must read data from HBM, perform calculations, and write it back to memory. "Reading and writing consume almost all the time, leaving the GPU idly waiting." Even with algorithmic optimizations, GPU utilization struggles to exceed 30%. This reality underpins his long-held core thesis: "AI equals Memory."

Why GPUs Face a 'Bottleneck Dilemma'

Professor Kim offered a sharp critique of NVIDIA's current situation. He noted that CEO Jensen Huang's frequent visits to Korea, TV appearances, casual meetings, and networking "indicate he is not at ease."

"GPU technological growth is nearly stalled, in my judgment," Kim stated. "The evolution of AI computers is in the hands of memory." His logic is clear: to boost performance, GPUs can only increase chip area and add more compute units. However, GPUs run too hot, requiring heatsinks on the back, preventing the vertical stacking possible with memory. "GPUs are caught in a bottleneck dilemma."

In contrast, as the focus shifts from the training era to the inference era, the importance of memory is being re-evaluated. Kim stated, "In the inference era, what matters more is how much data you can feed into the AI, and the semiconductor that determines this is memory." He further argued that competition in AI capability is ultimately a competition in memory capability: "Whether Google's Gemini, OpenAI, or Anthropic's Claude is stronger is determined by memory—that is my assertion."

The Two Pillars of HBM: Capacity and Bandwidth

Professor Kim defined HBM's value by two dimensions. First is capacity. With the rise of context engineering, multimodal input, and Agentic AI, memory demand is doubling annually—"a 1000-fold increase over a decade." The traditional method of shrinking transistors to increase density is nearing quantum mechanical limits. Therefore, the only path is to "stack upwards."

Second is bandwidth. Kim used an analogy: "If traditional memory is an 8-lane highway, HBM is a 1024-lane highway, now 2048 lanes, and potentially a million lanes in a few years." Only by transmitting massive data simultaneously through parallel channels can the speed demands of AI computing be met.

HBF: The Stacking Era for NAND Flash

While HBM solves speed, capacity remains limited. Professor Kim detailed the next technological path: HBF (High Bandwidth Flash). Simply put, HBF involves vertically stacking NAND Flash memory like HBM. DRAM is fast but capacity-limited; NAND Flash offers large capacity and long-term data retention. Though slower, it suffices for storing "cold data" in inference scenarios.

Kim believes HBM and HBF will coexist, akin to urban planning: "Like having a department store surrounded by duplex apartments and regular houses, various forms of HBM and HBF will combine into a complex to supply data to the GPU." He made a clear long-term prediction: "Now is the era of HBM, but in 10 years, the market demand for NAND Flash and HBF will surpass HBM. Samsung and SK Hynix must prepare for the HBF era."

He noted companies currently developing HBF include SK Hynix, SanDisk, Samsung Electronics, and Japan's Kioxia. Kioxia's market cap recently surpassed Toyota to become Japan's top stock, SanDisk's share price continues to rise, while Samsung and SK Hynix maintain leading market capitalizations in Korea.

HBS: A More Advanced Third Path

Professor Kim also proposed a forward-looking concept—HBS (High Bandwidth SRAM). SRAM (Static Random-Access Memory) is about 1000 times faster than DRAM but has lower density and higher cost, traditionally used only as small on-chip caches. Kim's idea is to fabricate an entire 12-inch wafer with SRAM and then vertically stack 12 to 16 layers, expanding capacity from 100GB to 1600GB. "That would offer speed 1000 times faster with sufficient capacity, making sense," he said.

He described the ultimate AI chip as a "100-story 3D building": "HBM, HBF, and HBS each form multi-layered structures, with the GPU placed on the top for cooling. This 3D semiconductor structure is, in my current judgment, inevitable for future AI computers." He acknowledged the biggest engineering challenge is not computation but power delivery and thermal management: "Supplying thousands of amps to the GPU and stacked memory; designing the power delivery network will be the hardest technology, becoming the true core competency between companies."

Custom HBM: Shifting Power Dynamics

Professor Kim specifically discussed the structural changes HBM4 brings. Previously, memory was a standardized product; manufacturers produced first, then customers purchased. Buyers dictated price, and inventory risk lay with memory makers—the essence of the "memory cycle."

Starting with HBM4, which requires custom design for accelerator architectures from clients like NVIDIA, Google, and AMD, memory makers must secure volume commitments from customers before initiating development—through Long-term Agreements (LTAs). "AI companies desperately need high-performance HBM, so they are queuing up. Suppliers are starting to set prices; this is a paradigm shift."

He also anticipates future HBM chips will integrate communication functions, enabling "HBM modules to talk to each other," forming a coalition-like structure: "We communicate among ourselves, allocating more memory to those who treat us better; uncooperative GPUs get less." This further elevates the systemic position of memory manufacturers.

Samsung and Hynix: The Sole Dual-Capability Players

Professor Kim repeatedly emphasized that globally, only Samsung Electronics and SK Hynix can mass-produce both DRAM (for HBM) and NAND Flash (for HBF). "While SanDisk and Kioxia's stocks are soaring, they can only make HBF, not HBM. Samsung and SK Hynix possess the most powerful tools to lead the future."

When asked if the forecast of combined operating profit reaching 500-600 trillion won for Samsung and SK Hynix this year is realistic, Kim replied, "It is realistic." He added that he frequently holds technical discussions with executives from both companies and "their eyes are getting brighter." However, he noted real competitive pressure exists, with Micron and SanDisk receiving orders from NVIDIA and Google.

AI PCs and AI Phones: Memory Dictates Device Price

Professor Kim extended the memory demand narrative to end-user devices. He predicted that for future AI PCs to achieve true personal AI computing, the required memory scale could make "a PC cost 10 million won, with memory price determining the PC price." In AI smartphones priced at 3-5 million won, 2-3 million won could be the cost of memory. "The continuous evolution of AI infrastructure and models requires ever more memory. AI PCs and AI phones are another major axis of this trend."

Agentic AI and Physical AI: Memory Demand to Soar 1000-Fold

Professor Kim's view on AI's evolution is also noteworthy. He believes that with the advent of Agentic AI and Physical AI (embodied AI), memory usage could be about 1000 times higher than now. "AI agents work 24/7, unlike humans who sleep, so workload explodes, and memory demand naturally follows. That will herald the era not of HBM, but of 'Ultra HBM'."

The Research Journey: 50 Years of Accumulation and 'Luck'

At the interview's conclusion, Professor Kim reflected on his academic path. He earned his Ph.D. in 1993, researching femtosecond-level ultra-fast electrical signal measurement; his advisor later won a Nobel Prize in Physics. He joined Samsung Electronics' memory division in 1994, returned to KAIST in 1996, and spent about a decade on foundational memory and HBM research before it became a commercial product.

In 2015, he first heard the term "deep learning" at a university meeting and realized AI algorithms and HBM architecture used the same underlying mathematics—linear algebra and matrix operations. "I loved matrices in my sophomore year; both sides happened to use the same math—that's luck," he said with a laugh. He initially envisioned HBM for televisions to make images more vivid, never imagining it would become AI era infrastructure: "I didn't know back then; you could call that luck too."