High Bandwidth Flash (HBF): The New Frontier in AI Memory
As artificial intelligence models grow in size and complexity, the hardware required to run them is hitting a critical bottleneck: memory. While High Bandwidth Memory (HBM) provides the speed necessary for training, its limited capacity and high cost make it difficult to scale for massive inference workloads.
Enter High Bandwidth Flash (HBF). This emerging technology aims to bridge the gap between the extreme speed of DRAM and the massive capacity of NAND flash, offering a memory tier specifically optimized for AI inference.
What is High Bandwidth Flash (HBF)?
HBF is a specialized memory architecture that leverages advanced NAND flash technology to provide significantly higher bandwidth than traditional storage, while maintaining a capacity far greater than HBM. By optimizing the interface and controller, HBF allows AI accelerators to stream massive model weights and KV caches more efficiently.
The performance leap is substantial. According to Sandisk, HBF can achieve the following throughput:
- First Generation: 1.2 TB/s
- Second Generation: 2 TB/s
- Third Generation: 3.2 TB/s
In terms of capacity, HBF offers a dramatic increase over HBM:
- First Generation: 1 TB
- Second Generation: 1 TB
- Third Generation: 1.5 TB
While HBF has higher latency than DRAM, Sandisk simulations show that when reading pretrained weights for a Llama 3.1 405B model, HBF performed within 2.2% of a hypothetical unlimited-capacity HBM setup. This makes it highly effective for streaming large model weights.
Who is driving the standardization of HBF?
The move toward HBF is gaining momentum through high-level industry partnerships. In February 2026, Sandisk and SK hynix began global standardization efforts for HBF at Sandisk’s Milpitas headquarters, working through the Open Compute Project.
The HBF technical advisory board is chaired by David Patterson, a University of California, Berkeley emeritus professor and Google distinguished engineer. Patterson, a 2017 ACM Turing Award winner, stated that HBF could drive down costs for AI applications that are currently unaffordable by delivering unprecedented capacity at high bandwidth.
The advisory board also includes Raja Koduri, founder and CEO of Oxmiq Labs and former AMD chief architect. Koduri suggests that HBF will revolutionize edge AI by allowing sophisticated models to run locally in real time.
The involvement of SK hynix is particularly notable. Despite holding roughly 62% of the HBM market, SK hynix is collaborating on the HBF specification. SK hynix President and Chief Development Officer Ahn Hyun framed this as an ecosystem optimization strategy rather than a competition between individual technologies.
What are the implications for edge AI and enterprises?
HBF’s ability to maintain data when power is lost (persistence) and its stability at high operating temperatures offer unique advantages for edge computing. Ilkbahar suggests this could allow smartphones to make real-time decisions and seamlessly retrieve old context from previous queries without needing to communicate with the cloud.
For the enterprise sector, HBF could democratize access to high-level AI. Because HBF-enabled accelerators are expected to be more cost-effective, smaller companies may finally have the resources to fine-tune large, pre-trained models for domain-specific uses, a task previously reserved for hyperscale providers.
Pro Tip: When evaluating AI infrastructure, focus on the “inference-to-training” ratio. As models move from development to deployment, memory capacity and cost-per-terabyte become more critical than raw DRAM latency.
How will HBF impact the Australian data center market?
The shift toward more efficient memory tiers arrives as Australia experiences a massive influx of data center investment. Amazon has committed A$20 billion to Australian data centers between 2025 and 2029, marking one of the largest tech investments in the country’s history.
Additionally, OpenAI and NextDC are developing an A$7 billion AI campus in Sydney. With the local hyperscale market projected to grow from $6.27 billion in 2026 to $16.18 billion by 2031, the ability to manage power and cost via technologies like HBF will be essential for the economic viability of these large-scale clusters.
Frequently Asked Questions
Is High Bandwidth Flash (HBF) a replacement for DRAM?
No. HBF is aimed at AI inference workloads where high capacity and bandwidth are needed, but extreme low latency is less critical. It is intended to complement rather than replace DRAM.
When will HBF be available for commercial use?
Sandisk expects first samples to arrive in the second half of 2026, with the first AI-inference devices utilizing HBF expected to hit the market in early 2027.
Why is SK hynix supporting a technology that competes with its HBM business?
SK hynix views the move as an ecosystem play. By helping define the HBF standard, they aim to optimize the entire AI infrastructure rather than focusing solely on the performance competition of individual technologies.
What is the main benefit of HBF for large language models (LLMs)?
HBF supports large KV caches, which allows chatbots to handle long, complex user prompts and maintain context over many messages more efficiently.
Want to stay updated on the latest in AI hardware and semiconductor trends? Subscribe to our newsletter or leave a comment below with your thoughts on the future of memory architecture.