● HBM To HBF Shift Boosts AI Memory Hierarchy
HBM Next, HBF? The Real Shift in AI Memory Semiconductors Seen by SK hynix and SanDisk
The key takeaway of this issue is not simply whether “HBM becomes unnecessary.”
The truly important point is that, inside AI servers, memory architecture is moving beyond an era sustained by HBM alone and into a layered memory era where HBM, HBF, and SSDs each share roles.
In particular, the fact that SK hynix is advancing HBF technology together with SanDisk, even though it could threaten HBM, its core source of profit, can be seen as a signal that the memory semiconductor market is changing fundamentally.
In this article, let’s take a single look at what HBF is, whether it can actually reduce HBM demand, how it affects AI semiconductors and data center investment trends, and what investors and industry watchers should focus on.
1. One-line news summary: HBF is not a replacement for HBM, but an “intermediate memory layer”
HBF, reportedly being developed through collaboration between SK hynix and SanDisk, is a high-bandwidth memory structure based on NAND Flash.
Simply put, it is a method of stacking NAND Flash, originally used in SSDs, into multiple layers and placing it close to the AI chip.
In the existing AI server structure, HBM sits right next to the GPU, while SSDs are farther away.
HBM is fast, but expensive and limited in capacity.
SSDs are cheap and large in capacity, but from the GPU’s perspective they are too far away and too slow.
HBF is a new memory layer that sits between the two.
In other words, it is not as fast as HBM, but it is much closer than SSDs and provides much larger capacity than HBM.
2. Understanding the HBM·HBF·SSD structure through a kitchen analogy
If you think of an AI server as a kitchen, the GPU is the chef.
The chef calculates, runs the AI model, and produces the answer.
HBM is the small worktable right in front of the chef.
Because the ingredients can be grabbed immediately with just a quick reach, it is extremely fast.
But the worktable is narrow and expensive.
SSD is the large warehouse far away in the kitchen.
It can store an enormous amount of ingredients and is relatively inexpensive.
But if the chef has to keep going back to the warehouse, it takes a long time.
HBF is closer to a large mobile shelf placed between the worktable and the warehouse.
It is not as fast as the worktable, but it is much closer than the warehouse and can hold much more than the worktable.
3. Why HBF suddenly became necessary: agentic AI is increasing the memory bottleneck
In the past, AI often received one question and produced one answer.
But today’s AI trend is completely different.
Now AI agents remember long conversations, read documents, execute code, call external tools, and continue through multi-step tasks.
It is becoming possible not just to finish a task quickly, but to continue for 1 hour, 5 hours, or even 10 hours as a long-running job.
In this structure, the data AI must remember, namely the context, grows explosively.
As context grows, more memory is needed around the GPU.
Ultimately, the spread of agentic AI is a key factor driving memory semiconductor demand.
The recent rise in HBM prices and the expansion of data center investment are also underpinned by this trend.
4. Core technology: KV cache offloading is the reason HBF exists
One of the most important keywords to watch in recent AI infrastructure is KV cache offloading.
The KV cache is critical data used when a large language model remembers context and predicts the next token.
The problem is that this KV cache keeps growing as AI models perform longer and more complex tasks.
If all KV caches are placed in HBM, speed is fast but the cost becomes too high.
Conversely, if they are moved to SSD, costs drop but a bottleneck appears because the GPU has to wait for data.
That is why an intermediate layer became necessary.
Frequently used data that must respond immediately is kept in HBM, larger but less urgent data is kept in HBF, and long-term storage data is kept in SSD.
When this structure becomes widespread, AI servers will no longer be a competition simply to install faster chips, but rather a competition over how well the memory hierarchy is designed.
5. Why HBF’s specs matter: its capacity is overwhelmingly larger than HBM’s
Based on the original text, HBF is being discussed with a maximum capacity of around 512GB.
Compared with the benchmark that one HBM4 stack is mentioned at around 24GB, the capacity gap could widen to roughly 20 times.
Of course, the numbers may vary depending on the actual product configuration and generation-specific specs.
What matters is the direction.
HBF is a technology aimed at placing much larger capacity than HBM close to the GPU.
Bandwidth is also discussed at roughly 0.4TB per second up to as much as 3TB per second, depending on the class.
At the top end, there is also expectation that bandwidth comparable to HBM4 may be possible.
In addition, a structure is being discussed that can be attached to various CPUs and GPUs through common connection rules in the UCI family.
This is important because HBF may not remain a component for a specific company, but could expand across the entire AI semiconductor ecosystem.
6. “If HBF appears, is HBM over?” That interpretation is only half right
If HBF emerges, some HBM demand may be adjusted.
In particular, data such as AI model weights that are large but do not change frequently may move to HBF.
Right now, too much data is being placed on narrow and expensive HBM.
It is reasonable for part of that to move down to a wider shelf called HBF.
But HBF cannot completely replace HBM.
HBM is DRAM-based, while HBF is NAND Flash-based.
Structurally, NAND Flash is slower than DRAM in response speed, and it has lifespan issues due to repeated writing and erasing.
Data just calculated, data that changes continuously, and data that would stall the GPU even with a slight delay still need to remain in HBM.
So the emergence of HBF should be seen not as the end of HBM, but as a change that makes the criteria for what should go into HBM more stringent.
7. Why would SK hynix develop a technology that could shake its own cash cow?
This is the most interesting part of this issue.
From SK hynix’s perspective, HBM is currently its most powerful growth engine.
Even so, pushing HBF can be interpreted as a strategy to look at the entire AI memory ecosystem rather than just short-term product revenue.
It is similar to Apple’s case of putting music functions into the iPhone instead of only protecting the iPod.
As iPhones sold more, iPod demand declined, but Apple captured a larger smartphone market.
SK hynix can also be seen as moving away from a strategy of selling as much HBM as possible and instead trying to dominate the memory structure of the entire AI server.
It sells HBM in the fastest layer, HBF in the middle layer, and enterprise SSDs in the larger storage layer.
In this way, it becomes not just competition over a single product, but a strategy to capture the memory budget of the entire AI data center.
8. From a global economic outlook perspective: HBF is a technology that lowers AI service costs
One of the biggest concerns in the AI industry today is cost.
GPUs are expensive, HBM is expensive, power consumption is high, and data center investment is getting larger.
For AI service companies, they want to improve model performance, but if inference costs are too high, profitability deteriorates.
If HBF is properly established, not all data needs to be placed in expensive HBM.
That means the memory cost structure per AI server can be made more flexible.
If AI service costs fall, companies can deploy more AI services.
As usage increases, data center investment increases again.
In this process, the AI semiconductor, memory semiconductor, and the entire semiconductor supply chain can all grow together.
In other words, HBF is not just a new component technology, but an infrastructure technology connected to improving AI economics.
9. The real key other news often misses: HBF strengthens the “lock-in structure”
Many reports discuss HBF only in terms of whether it is or is not a replacement for HBM.
But the more important point is customer lock-in.
In the past, during the general-purpose DRAM era, if prices did not fit, it was relatively easy for customers to switch suppliers.
But HBM and HBF are not simple memory components.
GPU, CPU, packaging, interconnects, software, and cooling structures all need to be designed together.
In particular, if HBF is attached next to the GPU, works together with HBM, and expands to optical-communication-based connections, then system-level optimization becomes essential.
When that happens, customers become deeply embedded in the technical ecosystem of a specific memory company.
Going forward, it may no longer be easy to say, “Let’s switch to another company because the memory is cheaper.”
This is the core of the long-term strategy pursued by memory companies such as SK hynix and Samsung Electronics.
10. What it means for Samsung Electronics and SK hynix
SK hynix is aiming to lead not only the HBM market, where it already has a strong position, but also the middle layer called HBF.
If this strategy succeeds, it could rise beyond a simple HBM supplier to become a key design partner in the memory architecture of AI servers.
For Samsung Electronics, not only regaining competitiveness in HBM but also next-generation memory hierarchy strategy becomes important.
Simply making good HBM may not be enough.
An integrated strategy connecting AI model structure, data movement methods, packaging, foundry, and NAND Flash is needed.
Ultimately, competition in memory semiconductors is evolving beyond the distinction between DRAM and NAND into a battle over who can design the entire AI server more efficiently.
11. Why SanDisk matters: the role of NAND Flash grows again
SanDisk is a company with strong technological capabilities in NAND Flash and SSDs.
If HBF is based on NAND Flash, the role of companies like SanDisk becomes very important.
Until now, the protagonist of the AI memory market has been HBM, and HBM has moved mainly around DRAM companies.
But if HBF becomes fully established, NAND Flash companies can also enter the center of the AI semiconductor market.
This is a positive change for the NAND market as well.
Even during periods when NAND demand centered on smartphones and PCs is slowing, a new high-value demand source called AI data centers can emerge.
In the end, HBF could become an opportunity to expand the collaborative model between DRAM companies and NAND companies.
12. Three checkpoints that must be watched going forward
The first is actual sample performance.
It must be confirmed whether the bandwidth and capacity presented in standards or press materials are maintained in actual samples.
For AI servers, real-world performance is far more important than announced specs.
The key is how much bottleneck reduction it achieves while operating stably next to the GPU.
The second is heat and lifespan.
Because HBF is based on NAND Flash, heat and write endurance are important issues.
The area around the GPU has high temperatures and strong power density.
Whether HBF can operate stably for long periods in this environment is the key to commercialization.
In particular, how durability is secured in workloads that frequently write and erase data is crucial.
The third is ecosystem adoption.
No matter how good the memory is, if GPU makers, cloud companies, server manufacturers, and software platforms do not adopt it, the market will not open.
It is necessary to see whether NVIDIA, AMD, Intel, and major cloud companies will actually put HBF into real systems.
Also, software optimization that separates HBM, HBF, and SSD usage at the AI framework and model design stages must be possible.
13. From an investor perspective: HBF is a bigger theme than the HBM peak-out debate
The market is very interested in when HBM demand will peak.
But the HBF issue shows a trend that is bigger than a simple peak-out debate.
As AI advances, the total amount of memory required will continue to increase.
However, not all of that memory needs to be HBM.
Going forward, HBM, HBF, and SSDs will each share roles in different price and performance ranges.
For memory companies, this is an opportunity to broaden their product portfolios.
It also suggests, from a global economic outlook perspective, that the AI infrastructure investment cycle may continue not as a short-lived trend but as a structural investment.
This is why we must also watch data center investment, power infrastructure, cooling technology, advanced packaging, and foundry demand together.
14. Conclusion: The rise of HBF is not a crisis for HBM, but a signal of AI memory market expansion
Interpreting HBF’s emergence as meaning HBM will disappear is too simplistic.
More precisely, the work HBM must handle and the work HBF must handle are being separated.
HBM will continue to serve as the ultra-fast worktable that handles the fastest and most sensitive data.
HBF will take on the role of a large shelf that places large but relatively less sensitive data near the GPU.
SSD will handle long-term storage and large-capacity retention.
As the memory hierarchy becomes more segmented in this way, AI servers become more efficient.
And if AI service costs fall, overall AI demand can actually grow even larger.
In the end, HBF should be seen not as a technology that kills HBM, but as a technology that expands the entire AI memory semiconductor market.
< Summary >
HBF is a next-generation high-capacity memory based on NAND Flash and placed close to the GPU.
It plays the role of an intermediate memory layer, with larger capacity than HBM and closer to the GPU than SSDs.
As agentic AI spreads, context and KV cache are growing, increasing the need for HBF.
HBF may replace some HBM demand, but fast computational data still requires HBM.
The real key takeaway is not the end of HBM, but the start of a layered AI memory structure divided among HBM, HBF, and SSD.
SK hynix’s push into HBF can be seen as a strategy to dominate the entire AI server memory ecosystem rather than relying on HBM revenue alone.
Going forward, sample performance, heat and lifespan, and whether the ecosystem of GPU and cloud companies adopts it will be the most important checkpoints.
[Related Articles…]
The core shift in AI memory after HBM
Analyzing the AI semiconductor and data center investment cycle
*Source: [ 월텍남 – 월스트리트 테크남 ]
– 삼전, 닉스 HBM이제 필요 없다고?! 완전히 새로운 메모리가 등장..ㄷㄷㄷ
