● Memory Factory Arms Race
The Battleground After GPU Growth Slows: Why AI Data Centers Are Becoming “Memory Factories”
The core points you must look at in this article are exactly five.
First, we summarize why Samsung’s zHBM, or GHBM architecture, unveiled at FMS 2026, is faster than conventional HBM.
Second, we explain why HBF, which SK hynix, SanDisk, and Google are pushing, is drawing attention as the next-generation memory after HBM.
Third, we examine why Nvidia is considering an external memory architecture based on optical communication instead of immediately adopting HBF.
Fourth, we summarize why tHBM, first mentioned by KAIST Professor Kim Jeong-ho, is an idea that turns the thermal problem of zHBM on its head.
Fifth, we explore from an economic perspective how AI semiconductor competition is shifting from GPUs to memory-centric computing, AI infrastructure, data center power, and semiconductor packaging competition.
1. The core message of FMS 2026: The HBM war is not over, it is only beginning
On August 4 in Santa Clara, California, FMS 2026, the Future of Memory and Storage event, was held.
The core theme of this event was not simply “How much faster can HBM get?”
The real message was that “the bottleneck in the AI era has shifted from GPU computation to memory capacity and bandwidth.”
Until now, the AI semiconductor market has been centered on Nvidia GPUs.
But as generative AI becomes more advanced, and companies move beyond simple adoption to actually measuring ROI, the situation is changing.
Now the important question is not “How many GPUs did you buy?”
The core point is whether AI can remember longer contexts, answer more accurately, and perform inference at lower cost.
At the center of this issue are next-generation memory technologies such as HBM, HBF, zHBM, and tHBM.
2. Why HBF suddenly became necessary: The real bottleneck in the AI inference era is the KV cache
Today’s AI is no longer at the level of answering one short question at a time as in the past.
It must read hundreds of pages of documents, reference YouTube content, synthesize search results, and even remember previous conversations.
The input unit AI processes in this case is called a token.
In the past, processing only a few thousand tokens was enough, but now long contexts of hundreds of thousands of tokens, and in many cases over 1 million tokens, are becoming important.
The problem is that as the number of tokens increases, it is not only the amount of computation the GPU must perform that grows.
The KV cache that AI must store in the middle also grows explosively.
Simply put, the KV cache is the internal memory AI temporarily stores to understand and remember long contexts.
Up to now, this KV cache has mainly been stored in HBM.
But if the context length increases by 1,000 times, memory capacity and speed must also increase dramatically to match.
HBM is fast, but it has capacity limits.
Even if you attach 8 or 16 HBM stacks to a GPU, it will still be insufficient in the long run.
That is why the concept of HBF emerged.
3. What is HBF: High-bandwidth flash aiming for far greater capacity than HBM
HBM is a memory that achieves high bandwidth by vertically stacking DRAM.
HBF, on the other hand, is an approach that seeks much larger capacity based on NAND flash.
Compared with DRAM, NAND flash is generally much more advantageous in terms of capacity.
As Professor Kim Jeong-ho explains, if HBM is stacking DRAM, then HBF is about stacking NAND flash in a high-bandwidth architecture.
As demand for KV cache and long-term memory storage surges in AI inference, the need for HBF is growing.
In particular, SK hynix is moving together with SanDisk and Google to standardize HBF and build an ecosystem around it.
The important point here is that HBF is not a technology that will immediately and completely replace HBM.
HBM is likely to continue playing the role of close-range memory needed for ultra-high-speed computation.
HBF is more likely to take shape as a complement for large-capacity KV cache, long-term memory, and AI inference data storage that HBM alone cannot handle.
In the end, the structure of AI data centers is shifting from a simple form made up only of GPUs and HBM to a complex structure combining HBM, HBF, SSDs, optical communication, and power infrastructure.
4. Samsung’s zHBM or GHBM: The idea of stacking memory on top of the GPU
The technology Samsung emphasized at FMS 2026 is described as zHBM or GHBM.
The terminology is somewhat mixed, but the core idea is a structure that brings the GPU and HBM closer together in a vertical direction.
In the conventional HBM structure, HBM is attached next to the GPU.
For data to move from HBM to the GPU, it must move down vertically first and then move sideways again.
Professor Kim Jeong-ho compared this to “taking an elevator down to the first floor and then moving to the neighboring building.”
This is similar to already creating ultra-high-speed roads at the 1,024-lane or 2,048-lane level.
But road lanes cannot be expanded indefinitely.
So the idea that emerged was to stack HBM directly on top of the GPU.
If you do that, data can move directly in a vertical direction without moving sideways.
The advantage is that memory bandwidth can be greatly increased.
But there is a critical problem.
GPUs get very hot.
If HBM is placed on top of the GPU, it may be difficult for the memory to operate stably because of heat.
Professor Kim Jeong-ho said this structure may be more suitable for purpose-specific AI accelerators such as NPUs and LPUs, which generate less heat than general-purpose high-performance GPUs.
5. tHBM revealed by Professor Kim Jeong-ho: A structure that flips Samsung’s zHBM
The most interesting part of this interview is tHBM.
Professor Kim Jeong-ho revealed for the first time that tHBM is under study in his lab.
The core point of tHBM is simple but powerful.
Instead of stacking HBM on top of the GPU, it places the GPU on top of HBM.
In other words, it reverses the structure.
Why is this idea important?
The answer becomes clear when you think about data center cooling architecture.
Hot chips are better off being directly adjacent to the cooling device.
If the GPU is at the top, it is relatively easier to apply thermal management solutions such as cooling fans, water cooling, and immersion cooling.
On the other hand, if the hot GPU is underneath and the memory is on top of it, there is a greater risk that heat will be transferred to the memory side.
tHBM is a structure that aims to improve cooling efficiency by moving the GPU, the center of heat generation, to the top.
This idea is not just a packaging technology; it is an approach that could change the thermal design direction of AI semiconductors.
It could become an important hint for future competition in semiconductor packaging, advanced cooling, and data center power efficiency.
6. Why does Nvidia look complicated about HBF?
Just because HBF becomes technically necessary does not mean Nvidia will immediately adopt it.
From Nvidia’s perspective, dependence on SK hynix and Samsung Electronics is already high in HBM.
If HBF is also led by Korean memory companies and the SanDisk ecosystem, Nvidia’s strategic burden could grow even more.
So Nvidia may consider other alternatives rather than adopting HBF as is.
One commonly discussed direction is connecting external large-capacity storage and GPUs via optical communication.
Simply put, instead of building a high-rise apartment right next to the GPU like HBM, this approach places a large warehouse outside and connects it with ultra-fast optical communication highways.
This structure has the advantage of reducing dependence on memory companies.
However, because data must travel farther, limits arise in latency and bandwidth.
Professor Kim Jeong-ho believes that to fully leverage GPU performance, memory should be placed as close to the GPU as possible.
From this perspective, Nvidia’s optical communication strategy is more like a detour to reduce dependence on memory.
That said, as AI data centers grow from the rack level to the cluster level, optical communication and externally expandable memory architectures are also likely to advance together.
7. The meaning of cHBM: Memory is no longer a general-purpose commodity
Another important concept mentioned by Professor Kim Jeong-ho is cHBM.
cHBM stands for Custom HBM, meaning customer-specific HBM.
Traditional memory semiconductors were close to standard products.
Anyone could make memory to the same specification, and customers would choose the cheapest product among them.
In such a market, the semiconductor cycle of rising and falling prices repeats itself.
But if computation functions, CPU functions, and data movement control functions are added to the base die of HBM, the story changes.
Memory is no longer just a component; it takes on the character of a system semiconductor designed together with the GPU.
In that case, production volume, price, performance, and packaging structure must be aligned from the early design stage with the customer company.
Then it becomes possible to move away from the structure where prices collapse like a general-purpose memory product.
This is very important from an investment perspective.
As HBM and cHBM spread, the earnings structure of memory companies may become more closely tied to AI infrastructure investment and long-term supply contracts than to the old DRAM price cycle.
In other words, when evaluating the valuation of memory semiconductor companies, relying only on the old way may cause you to miss important points.
8. From GPU-centric computing to memory-centric computing: The shift in power within AI semiconductors
Professor Kim Jeong-ho said GPU development has nearly stopped.
Of course, this does not mean GPU performance improvement has completely halted.
More precisely, it means that simply increasing GPU computing performance is no longer enough to solve bottlenecks in AI infrastructure.
Most AI models are based on the transformer architecture.
Differences between the models themselves are increasingly shrinking.
As open-source models advance rapidly, the gap among models in the United States, China, and Korea is narrowing compared with the past.
Now value is generated more in inference infrastructure, memory, power, and data center operational efficiency than in the model itself.
Companies operating AI services may earn more revenue, but if infrastructure costs are too high, profitability becomes difficult.
The original text explains that even if an AI company earns 1,000,000 won, 700,000 won may go to infrastructure costs.
The core of those costs is memory and power.
That is why future AI data centers will resemble memory factories more than simple server farms.
When you open up the inside of the AI factory that Jensen Huang talks about, it ultimately means that high-performance memory and power systems are the key.
9. The key change from an economic perspective: AI infrastructure has become a capital battle
AI semiconductor competition is now not only a technology competition but also a capital competition.
Cutting-edge GPUs, HBM, HBF, advanced packaging, liquid cooling, power grids, and data center sites all require massive investment.
Realistically, the countries that can sustain this battle at the national level are mainly the United States and China.
The United States has big tech, capital markets, and cloud infrastructure.
China has state-led investment and a huge domestic market.
Korea, in terms of capital scale alone, cannot directly keep up with the United States and China.
But Korea has a decisive choke point.
That is precisely its ability to produce HBM and high-performance memory semiconductors.
As AI data centers shift toward memory factories, Korea’s SK hynix and Samsung Electronics can take on even more important positions in the global supply chain.
In particular, even amid uncertain global economic prospects and volatile rates, exchange rates, and stock markets, AI infrastructure investment is likely to remain a long-term growth pillar.
That said, the semiconductor cycle does not disappear entirely.
General-purpose DRAM and NAND can still experience price fluctuations.
But as the share of high-value-added custom products such as HBM, HBF, and cHBM grows, the earnings structure of memory companies can become more stable than before.
10. The strategy Korea must pursue: Selling only memory is not enough, and it must secure at least part of the full stack
Professor Kim Jeong-ho believes Korea must create an irreplaceable position in memory.
The idea is to make it so that any country in the world would have no choice but to cooperate with Korean memory companies in order to do AI.
But stopping there would be disappointing.
Korea must secure at least part of the full stack, not only memory but also power infrastructure, AI models, and service monetization models.
Samsung has smartphones and TVs.
Hyundai Motor has automotive platforms.
LG has home appliances and display ecosystems.
If AI services embedded in these devices rely only on overseas platforms, Korea may end up remaining merely a hardware supplier.
Conversely, if Korean companies connect device-based AI services, on-device AI, automotive AI, home appliance AI, and industrial AI solutions, the picture changes.
It becomes possible to build a structure that expands from memory into AI infrastructure and services.
This is the realistic path by which Korea could aim to become one of the top three AI powers alongside the United States and China.
11. The real core point that other news often misses
First, HBF is not just the next product after HBM; it is a revaluation point for the NAND industry.
Until now, AI benefits have been interpreted mainly around HBM.
But if HBF becomes fully established, NAND flash may also be revalued as a core component of AI inference infrastructure.
This is an issue connected to SanDisk, Kioxia, Samsung Electronics’ NAND business, and SK hynix’s Solidigm strategy.
Second, the contest between zHBM and tHBM may be decided more by thermal management than by speed.
Stacking memory on top of the GPU shortens the data transfer distance.
But if heat cannot be controlled, commercialization becomes difficult.
Going forward, AI semiconductor competition is likely to be determined not simply by nanoscale process nodes, but by packaging, cooling, power efficiency, and materials technology together.
Third, cHBM could change the business model of the memory industry.
As custom HBM expands, memory is no longer a commodity sold cheaply in large volume.
It becomes a strategic component designed together with GPUs, AI accelerators, and cloud data centers.
This change is a very important variable for the long-term profitability of Samsung Electronics and SK hynix.
Fourth, Nvidia’s optical communication strategy is both a technology strategy and a bargaining strategy.
The reason Nvidia emphasizes optical communication and external memory architectures is not only about performance.
There is also a strategic purpose of reducing dependence on Korean memory companies.
Therefore, the HBF standardization race is not only a competition over technical standards but also a competition over supply chain leadership.
Fifth, the next bottleneck in the AI investment cycle is power.
As memory grows, power consumption grows as well.
AI data centers cannot run just because they have high-performance semiconductors.
They also need power grids, cooling water, transformers, energy storage systems, and data center sites.
That is why AI infrastructure investment is connected not only to semiconductor companies but also to power equipment, cooling, and energy infrastructure companies.
12. Checkpoints that investors and companies should watch
1) Confirm whether HBM demand expands into HBF demand.
As the AI inference market grows, the need for HBF may increase further.
In particular, as long-context use, AI agents, and enterprise search-style AI spread, demand for large-capacity memory increases.
2) The speed of HBF standardization by SK hynix, SanDisk, and Google is important.
Companies that secure the standard first can gain leadership in the ecosystem.
Conversely, if customer adoption is delayed, market opening may be postponed even if the technology exists.
3) Samsung Electronics’ zHBM and GHBM strategies must be viewed together with thermal management technology.
The vertical stacking structure is attractive, but the heat issue is the key variable.
What to watch is how Samsung combines advanced packaging with cooling solutions.
4) Nvidia’s choice could significantly shake up the market direction.
If Nvidia adopts HBF, market adoption will accelerate.
On the other hand, if it pushes an optical communication-based external memory structure, the HBF ecosystem may grow centered on Google, AMD, and companies developing their own AI accelerators.
5) The AI semiconductor value chain is expanding from GPUs into memory, packaging, and power.
When looking at AI beneficiaries now, it is not enough to look only at GPUs.
You must also consider HBM, NAND, HBF, interposers, substrates, cooling, power equipment, and data center operational efficiency.
< Summary >
The center of AI semiconductor competition is moving from GPUs to memory.
HBM is fast, but it has capacity limits when it comes to handling the long contexts and KV caches of the AI inference era.
That is why SK hynix, SanDisk, and Google are pushing HBF as the new large-capacity high-bandwidth memory.
Samsung Electronics proposed zHBM and GHBM structures that stack the GPU and memory vertically, but heat is the key variable.
Professor Kim Jeong-ho’s tHBM reverses this structure by placing the GPU on top of HBM to improve cooling efficiency.
cHBM is an important trend that turns memory from a general-purpose product into a customer-specific system semiconductor.
In the future, AI data centers are likely to become closer to memory factories centered on memory and power rather than simple GPU factories.
Korea must use HBM and HBF as a base to capture the choke points of the global AI infrastructure supply chain and expand into device-based AI services as well.
[Related Articles…]
The Next Battleground in the AI Semiconductor Market After HBM
AI Data Center Power Infrastructure and the Global Investment Trend
*Source: [ 티타임즈TV ]
– “GPU 발전은 멈췄다! 이제 데이터센터는 메모리 팩토리” (김정호 카이스트 교수)
