HBM father says AI equals memory as GPU utilization hits 10%

KAIST professor Kim Jung-ho, the engineer credited with inventing HBM, argues the AI era's defining bottleneck has shifted from compute to memory — a claim backed by Micron's 84.9% gross margin and $100 billion in locked-in customer contracts.

The engineer credited with inventing high-bandwidth memory says the AI industry's defining constraint is no longer GPU compute but memory bandwidth, with GPU utilization stuck at 10% because processors spend most of their time waiting for data.

"AI equals memory," Kim Jung-ho, a professor of electrical engineering at KAIST and the architect behind the first HBM specification, said in a video interview. "The evolution of AI computers is in the hands of memory."

Kim's argument finds support across the memory supply chain. Micron Technology posted an 84.9% non-GAAP gross margin in its fiscal third quarter, with CEO Sanjay Mehrotra disclosing 16 Strategic Customer Agreements covering roughly 20% of DRAM volume and one-third of NAND volume, carrying remaining performance obligations of about $100 billion. Samsung Electronics and SK Hynix, the two companies Kim identified as uniquely positioned to manufacture both DRAM and NAND, announced a combined 800 trillion won ($518 billion) investment in four new mega-fabs in South Korea's Honam region, targeting completion by the mid-2030s.

The structural shift threatens Nvidia's dominance in AI hardware. Kim said Nvidia's GPU technology growth "has almost stopped" because chips cannot be stacked vertically due to heat dissipation constraints, while memory can. If Kim's thesis holds, the roughly $3 trillion in market capitalization concentrated in GPU makers could begin migrating to memory manufacturers as AI workloads shift from training to inference.

HBM4 Marks a Shift in Supplier Power

Starting with HBM4, memory is no longer a standardized commodity. Kim said each major AI customer — Nvidia, Google, AMD — now requires a custom HBM design tailored to its accelerator architecture. That shift has inverted the traditional buyer-supplier relationship. Memory manufacturers now demand long-term agreements and volume commitments before beginning development, effectively setting prices rather than accepting them.

"AI companies desperately need high-performance HBM, so they line up," Kim said. "The supplier starts to decide the price. This changes the model entirely."

The numbers bear this out. Micron's Mehrotra told investors that gross margins at the floor of these agreements "will be well beyond the peaks we experienced in prior cycles." Micron shares have gained 296.9% year to date, while Samsung and SK Hynix have seen their market capitalizations swell as investors price in sustained pricing power.

HBF and HBS: The Next Decade of Memory Architecture

Kim outlined two additional memory architectures he expects to reshape AI hardware over the next decade. The first, High Bandwidth Flash (HBF), stacks NAND flash vertically in the same manner as HBM, offering roughly 10 times the capacity at lower speeds — suitable for storing the "cold data" that inference workloads accumulate. Companies developing HBF include SK Hynix, Samsung, Sandisk, and Japan's Kioxia, which recently surpassed Toyota in market value to become Japan's largest company by market capitalization.

The second, High Bandwidth SRAM (HBS), represents a more radical departure. Kim proposes fabricating an entire 12-inch wafer as SRAM — which is roughly 1,000 times faster than DRAM — then stacking 12 to 16 layers vertically to reach 1.6 terabytes of capacity. A GPU would sit on top, with cooling integrated into the stack's uppermost layer.

"Power delivery will be the hardest technology," Kim said. "Supplying thousands of amps through this 3D structure — that will become the true core competitiveness between companies."

The Investment Case for Memory Over Compute

Kim's long-term forecast is unambiguous: "It's the era of HBM now, but 10 years from now, NAND flash and HBF market demand will surpass HBM. Samsung and SK Hynix must prepare for the HBF era."

For investors, the thesis translates into a structural re-rating of memory stocks. Samsung and SK Hynix are forecast to generate a combined 500 trillion to 600 trillion won ($324 billion to $389 billion) in operating profit this year, according to Kim, who said he regularly meets with executives at both companies. "Their eyes are getting brighter," he said.

The risk is cyclical. Memory downturns have historically punished even the strongest manufacturers. But Kim argues that custom HBM contracts with floor margins above prior cycle peaks provide unprecedented downside protection. "The model has changed," he said.

This article is for informational purposes only and does not constitute investment advice.