HBM Is Indispensable, But Sandisk HBF Solves Its Greatest Weakness — Low Capacity

HBM DRAM · Sandisk HBF · Memory Capacity · AI Accelerators · Inference Infrastructure

HBM Is Indispensable, But Sandisk HBF Solves Its Greatest Weakness — Low Capacity

A comprehensive B2B analysis of why High Bandwidth Memory remains the irreplaceable engine of AI acceleration, why its limited capacity has become the single most expensive constraint in AI infrastructure, and how Sandisk High-Bandwidth Flash (HBF) is positioned to remove that constraint without displacing HBM from its rightful role.

Editorial note: This article is intended for AI datacenter operators, cloud and neocloud providers, GPU and accelerator server buyers, semiconductor and memory supply-chain professionals, storage solution vendors, enterprise infrastructure architects, system integrators, distributors, and technology investors. It treats HBM DRAM and Sandisk HBF as complementary technologies. References to HBF specifications reflect publicly stated roadmap positioning and should be validated against shipping product documentation.

The argument in one sentence: HBM wins on bandwidth and latency and will keep winning, but it loses on capacity and cost per gigabyte, and HBF is the first technology designed to fix exactly that loss while sitting in the same physical position on the accelerator package.

1. Introduction: The Paradox at the Heart of AI Memory

Every modern AI accelerator worth deploying is built around High Bandwidth Memory. Without HBM, the thousands of matrix engines inside a GPU or custom AI chip would starve, waiting for data that conventional memory could never deliver fast enough. HBM is the reason large language models can generate text at interactive speeds, the reason training runs complete in weeks rather than years, and the reason the entire AI infrastructure economy functions. Its position is not under threat from any technology on the horizon. In the truest sense of the word, HBM is indispensable.

And yet, the most common complaint heard from anyone operating AI infrastructure at scale is that there is never enough of it. Not enough HBM to hold the model. Not enough to hold the key-value cache for long contexts. Not enough to serve many tenants and their fine-tuned variants. Not enough to keep a mixture-of-experts model’s full expert set resident. Not enough to avoid spreading a single model across eight, sixteen, or more accelerators when the compute of two or three would have sufficed. HBM is the best memory technology ever built for AI, and its scarcity is the single most expensive constraint in the AI datacenter.

This is the paradox. HBM’s greatest strength—extreme bandwidth achieved by placing stacked DRAM millimeters from the compute die—is inseparable from its greatest weakness. The physical constraints that make HBM fast also cap how much of it can fit on a package, and the economics of DRAM ensure that every gigabyte is expensive. Bandwidth and capacity are in tension, and for a decade the industry has resolved that tension by choosing bandwidth and paying for capacity through brute force: more accelerators, more nodes, more power, more cost.

Sandisk High-Bandwidth Flash, or HBF, is the first technology designed to break the tension rather than choose a side. By applying HBM-style stacking and packaging to high-density NAND flash, HBF aims to deliver a memory stack that fits in the same physical position as HBM, connects through a comparable wide interface, provides bandwidth in the same class, and offers roughly eight to sixteen times the capacity at a fraction of the cost per bit. It does not replace HBM. It solves HBM’s capacity problem by placing a large, persistent, high-bandwidth pool right next to it.

This article examines that proposition in depth. It first establishes why HBM is genuinely irreplaceable, then dissects why its capacity is limited and why those limits are structural rather than temporary. It quantifies how AI demand has outrun HBM capacity, what that costs the industry, and why existing workarounds are inadequate. It then explains what HBF is, how it addresses capacity directly, how it divides labor with HBM, which workloads benefit most, what the honest trade-offs are, how the economics work, what software must change, and what the ecosystem and timeline look like. It closes with strategic guidance for buyers, vendors, and investors navigating the transition to a tiered accelerator memory architecture.

2. Why HBM Is Indispensable

To appreciate why HBF complements rather than competes with HBM, one must first be precise about what HBM does that nothing else can. The core of AI computation is matrix multiplication at enormous scale. A transformer model with hundreds of billions of parameters executes, for every generated token, a sequence of matrix operations that touch every weight in the active portion of the model. The arithmetic is performed by thousands of parallel compute units, and those units are only useful if they receive operands continuously. The rate at which operands can be delivered is memory bandwidth, and memory bandwidth is the ceiling on AI performance.

Consider the decode phase of large language model inference, in which tokens are generated one at a time. For each token, the accelerator must read the model’s weights and the accumulated key-value cache from memory. The compute required per token is modest relative to the accelerator’s peak capability; the memory traffic is not. In practice, decode throughput is almost entirely determined by how many bytes per second the memory system can supply. An accelerator with twice the memory bandwidth generates tokens roughly twice as fast, all else equal. This is why every generation of AI accelerator has pushed HBM bandwidth higher, and why bandwidth figures headline product launches.

Training is even more demanding. Forward passes, backward passes, gradient accumulation, and optimizer updates all move data between memory and compute at rates that saturate multi-terabyte-per-second HBM subsystems. Training also involves fine-grained, write-intensive access patterns—updating parameters and optimizer states in place—that are fundamentally unsuited to any non-volatile memory. Training is and will remain an HBM workload.

HBM achieves its bandwidth through physical proximity and interface width. Rather than a 64-bit channel on a motherboard traversing centimeters of trace, each HBM stack presents a roughly 1024-bit interface over a distance of millimeters across a silicon interposer. Wide and short means high bandwidth and low energy per bit transferred. No memory located off-package can match this, because signal integrity and power constraints limit how many bits can be moved across a board or connector. The performance advantage of HBM is a consequence of geometry, and geometry does not change.

HBM also delivers DRAM latency: random access to any location in well under a microsecond, with predictable timing and no wear. Activations, attention computations, scratch buffers, and the hot portion of the KV cache all require this property. These data structures are accessed in patterns that cannot be predicted far enough ahead to prefetch and are too fine-grained to benefit from streaming. They must live in memory that responds to arbitrary requests immediately. That memory is HBM.

For all of these reasons, HBM’s role is secure. Every major accelerator roadmap for the remainder of the decade includes HBM in increasing bandwidth and stack counts. HBF, CXL memory, faster NVMe, and every other emerging technology position themselves around HBM, not against it. The correct mental model is not “HBM or something else” but “HBM plus what?” This article argues that the most compelling answer to “plus what” is HBF, because it targets the one attribute where HBM is weakest.

3. Anatomy of an HBM Stack

Understanding HBM’s capacity ceiling requires understanding how a stack is built. An HBM stack consists of a base logic die at the bottom and a number of DRAM dies stacked vertically above it. Each DRAM die is thinned to tens of micrometers, and the dies are connected vertically by through-silicon vias—thousands of tiny conductive channels etched through the silicon—and by microbumps between layers. The base die manages the interface to the host accelerator, distributes commands to the DRAM layers, and handles testing and repair functions.

The number of DRAM dies per stack has grown across generations: four, then eight, then twelve, with sixteen-high stacks on roadmaps. Each die carries a fixed capacity determined by the DRAM process node and die area. Stack capacity is therefore the product of die count and die capacity. Current high-end stacks reach several tens of gigabytes. An accelerator integrates several stacks—commonly six to eight—around the compute die, yielding total HBM capacity in the range of roughly 100 to 300 gigabytes depending on generation and configuration.

The stack sits on a silicon interposer or advanced organic substrate alongside the compute die. The interposer carries the dense wiring that connects each stack’s wide interface to the accelerator’s memory controllers. The entire assembly—compute die, HBM stacks, interposer, substrate—is a single package that must be manufactured, tested, and cooled as a unit. Package size is limited by the reticle of the lithography tools used for the interposer, by warpage and mechanical reliability of large assemblies, and by the thermal capacity of the cooling solution.

Every element of this construction is a capacity constraint. Die count per stack is limited by stack height, thermal resistance through the stack, and TSV yield. Die capacity is limited by DRAM cell scaling, which has slowed dramatically. Stack count per package is limited by the perimeter of the compute die and by interposer area. Package size is limited by manufacturing. And all of it is expensive: HBM DRAM dies are produced on advanced nodes, TSV processing and stacking add significant cost, yield loss compounds across stacked dies, and the packaging itself is among the most costly in the semiconductor industry.

The consequence is that HBM capacity per accelerator advances incrementally, generation by generation, at a pace set by the slowest of these constraints. Meanwhile, the memory footprint of AI workloads advances at a pace set by model designers, application developers, and users—which is to say, much faster. That gap is the subject of the next several sections.

4. HBM’s Greatest Weakness: Capacity

If HBM were merely fast, its limited capacity would be an inconvenience. Because HBM is the only memory the accelerator can access at full speed, its limited capacity is a hard boundary on what the accelerator can do. Any data the accelerator needs quickly must fit in HBM. Anything that does not fit must be fetched from far slower tiers, and the performance penalty of that fetch is so severe that system designers avoid it by any means available.

The primary means of avoidance is parallelism across accelerators. If a model does not fit in one accelerator’s HBM, it is split across two, four, eight, or more, using tensor parallelism, pipeline parallelism, or expert parallelism. Each accelerator holds a slice of the model in its own HBM, and the slices communicate over high-speed interconnect during execution. This works, and it is how every frontier model is served today. But it means that the number of accelerators deployed for a given model is determined by memory capacity, not by compute requirement. In many inference deployments, the accelerators are compute-underutilized because they were provisioned to provide HBM, not FLOPS.

The second means is compression. Quantizing weights from sixteen bits to eight or four reduces memory footprint proportionally. KV cache can also be quantized. These techniques are widely used and effective, but they have limits: aggressive quantization degrades model quality, and there is a floor below which further compression is not acceptable for production workloads. Compression buys time; it does not remove the constraint.

The third means is eviction and recomputation. When the KV cache for active sessions exceeds HBM, the serving engine evicts cache blocks and recomputes them if the session resumes. Recomputation consumes compute that could have generated tokens and increases time-to-first-token for the affected user. Serving engines have become sophisticated at managing this trade-off, but every eviction is a cost paid because capacity was insufficient.

The capacity weakness manifests differently across workloads but is universal. For dense models, it determines the minimum accelerator count. For mixture-of-experts models, it determines whether the full expert set can be resident. For long-context applications, it determines how many concurrent sessions can be served. For multi-tenant platforms, it determines how many models and adapters can be hot. For retrieval-augmented systems, it forces the index to live elsewhere entirely. In every case, more capacity at HBM-class bandwidth would directly increase throughput, reduce hardware, or improve latency. That is what makes capacity HBM’s greatest weakness: it is the attribute whose improvement would have the largest effect on the economics of AI.

It is important to be clear that HBM manufacturers are working hard on capacity, and each generation adds more. But the increments are bounded by physics and economics in ways that bandwidth increments are not. Bandwidth can be increased by widening the interface and raising signaling rates. Capacity requires more dies or denser dies, and both are constrained by the manufacturing realities described in the previous section. The industry needs a source of capacity that scales on a different curve, and that is where NAND enters the picture.

5. Why HBM Capacity Cannot Scale Fast Enough

Several independent forces prevent HBM capacity from keeping pace with AI demand. Each is worth examining because together they explain why the capacity problem is structural rather than a temporary shortfall that the next generation will resolve.

DRAM cell scaling has slowed. For decades, DRAM density improved as lithography shrank the capacitor and transistor that form each bit cell. That scaling has hit fundamental barriers: capacitors must retain enough charge to be reliably read, and shrinking them further compromises retention and reliability. DRAM manufacturers now advance density through incremental process refinements and, increasingly, through vertical structures, but the era of doubling density every couple of years is over. Die capacity growth is slow, and it directly bounds stack capacity.

Stack height is limited by thermal and mechanical constraints. Each added DRAM die increases the thermal resistance from the bottom of the stack to the heat spreader on top. The lowest dies in a tall stack run hotter, and DRAM refresh requirements increase with temperature, consuming bandwidth and power. Taller stacks also require thinner dies, which are more fragile and harder to handle. Package height limits, especially in systems with tight z-height constraints, add a further ceiling. Sixteen-high stacks are on roadmaps, but progress beyond that is uncertain.

Yield compounds across stacked dies. An HBM stack is only good if every die in it is good and every TSV connection works. Yield loss multiplies with die count. Manufacturers mitigate this with redundancy and known-good-die testing, but the cost of stacking many dies, some fraction of which will be scrapped, is inherent. Taller stacks are disproportionately expensive per gigabyte.

Package area is finite. The number of HBM stacks per accelerator is limited by the perimeter of the compute die along which stacks can be placed and by the maximum interposer size that can be manufactured. Multi-die compute assemblies and larger interposers have expanded the budget, but every additional stack position competes with compute area, I/O, and thermal headroom. Placing sixteen stacks on a package is far harder than placing eight.

Supply is concentrated and allocated. HBM is manufactured by a small number of DRAM suppliers, and those suppliers have shifted wafer capacity toward HBM at the expense of conventional DRAM, contributing to price increases across the memory market. HBM supply is allocated to accelerator vendors under long-term agreements, and buyers frequently cannot obtain as much as they want. Even if a design could accommodate more HBM, the parts may not be available.

Cost per gigabyte is structurally high. Advanced-node DRAM, TSV processing, stacking, testing, and packaging combine to make HBM the most expensive memory per bit in volume production, several times the cost of conventional DRAM and an order of magnitude or more above NAND. Even if all other constraints were relaxed, filling an accelerator with terabytes of HBM would be economically irrational for data that does not need HBM-class latency.

These forces are not going away. HBM capacity will continue to grow, but it will grow on a curve determined by DRAM physics and advanced packaging economics. AI memory demand grows on a curve determined by model architecture and application design. The two curves have diverged and will continue to diverge. Closing the gap requires a technology whose capacity scales on a different curve, and NAND flash, whose density has continued to improve through vertical stacking of hundreds of layers, is the obvious candidate.

6. The Explosion of AI Memory Demand

On the other side of the gap, AI memory demand has grown along every dimension simultaneously. Understanding these dimensions clarifies why no plausible HBM roadmap closes the gap alone.

Model size. Frontier language models have grown from billions to hundreds of billions to over a trillion parameters. Even at 4-bit quantization, a trillion-parameter model requires roughly half a terabyte for weights alone. At 8-bit, a full terabyte. No single accelerator can hold this, and the HBM roadmap does not reach it within the planning horizon of current deployments.

Mixture-of-experts architectures. The frontier has shifted to MoE designs, in which the model contains many expert sub-networks but activates only a few per token. This decouples total parameter count from compute per token, allowing models with enormous total size and modest active size. The consequence for memory is that total parameters—and therefore total memory footprint—can grow much faster than compute requirements. An MoE model may have a trillion parameters but activate only a few tens of billions per token. The compute of a single accelerator could serve it, but the HBM of many is needed to hold it.

Context length. Context windows have expanded from a few thousand tokens to hundreds of thousands and, in some models, over a million. KV cache scales linearly with context length and with the number of concurrent sequences. A single long-context request can produce tens of gigabytes of cache. A platform serving thousands of such requests concurrently needs terabytes of cache capacity. HBM cannot provide it.

Concurrency. Inference platforms serve millions of users. Throughput and cost efficiency improve with batch size, because larger batches amortize weight reads across more tokens. But every sequence in a batch consumes KV cache, so achievable batch size is bounded by HBM capacity after weights are accounted for. Capacity limits concurrency, concurrency limits batch size, and batch size limits efficiency.

Model and tenant multiplicity. Commercial platforms serve many models—different sizes, specializations, modalities—and thousands of customer fine-tunes implemented as adapters. Holding this pool hot requires capacity far beyond HBM. Platforms compensate with cold starts and model rotation, both of which cost latency and utilization.

Retrieval and agent state. Retrieval-augmented generation attaches large vector indexes to inference. Agentic systems accumulate persistent state across long sessions. Neither lives in HBM today, but both are increasingly on the latency-critical path, and both would benefit from residing closer to compute.

Multiply these dimensions together and the memory footprint of a modern inference deployment is measured in terabytes per node and petabytes per fleet, of which HBM can hold a small fraction. The remainder is scattered across host DRAM, NVMe, and network storage, with performance cliffs at every boundary. This is the demand side of the capacity gap, and it is growing faster than HBM supply or HBM density ever will.

7. What the Capacity Ceiling Costs the Industry

The HBM capacity ceiling is not an abstract limitation. It translates into concrete, measurable costs across the AI infrastructure economy.

Excess accelerator count. When a model is spread across more accelerators than its compute requires, the surplus accelerators are effectively purchased as memory carriers. Each carries a full accelerator price, consumes a full accelerator power budget, occupies rack space, and requires cooling and interconnect. For inference deployments of large models, memory-driven overprovisioning can account for a substantial fraction of total accelerator spend.

Interconnect pressure. Splitting a model across accelerators means every token generation involves communication among them. High-speed interconnect fabrics are expensive, power-hungry, and themselves supply-constrained. The more accelerators per model instance, the greater the interconnect requirement and the higher the risk that communication rather than memory becomes the bottleneck.

Reduced batch efficiency. Because KV cache competes with weights for HBM, achievable batch size is smaller than compute would allow. Smaller batches mean lower accelerator utilization during decode, which means more accelerators to reach a throughput target and higher cost per token.

Quality compromises. Pressure to fit models in HBM drives adoption of more aggressive quantization than would otherwise be chosen, with measurable degradation in output quality for some tasks. Capacity constraints thus impose a quality cost as well as a financial one.

Latency penalties. Cold starts for models and adapters not resident in HBM, prefill recomputation for evicted sessions, and network round trips to external retrieval systems all increase time-to-first-token. For interactive applications, latency directly affects user experience and, in commercial contexts, conversion and retention.

Operational fragility. Because HBM is volatile, every restart requires reloading models from storage. For fleets of thousands of nodes, coordinated restarts saturate storage networks and extend recovery time. Rolling updates and failure recovery are slower and more complex than they would be with persistent near-package memory.

Supply exposure. Dependence on HBM for all near-compute memory concentrates supply risk in a handful of suppliers and one packaging ecosystem. When HBM is short, AI capacity expansion stalls regardless of accelerator availability.

Power and facility cost. Every excess accelerator, every interconnect switch, every storage network transfer consumes power. In regions where grid capacity is the binding constraint on datacenter growth, memory-driven inefficiency directly limits how much AI can be deployed.

Summed across the industry, these costs are enormous. They are also largely invisible in component-level analysis, because they show up not as “memory cost” but as accelerator count, power bills, interconnect spend, and latency SLA misses. Any technology that materially raises near-package memory capacity at HBM-class bandwidth attacks all of these costs at once. That is the size of the opportunity HBF addresses.

8. Current Workarounds and Why They Fall Short

The industry has not been idle in the face of the capacity ceiling. Several approaches to extending accelerator memory are in use or development, and it is instructive to understand why none of them fully solves the problem.

Host DRAM offload. Accelerators can access host system memory over PCIe or proprietary CPU-to-GPU links. Serving engines use this to spill KV cache and, in some cases, model weights. The limitation is bandwidth: host links deliver tens to low hundreds of gigabytes per second, one to two orders of magnitude below HBM. Data offloaded to host memory is effectively cold from the accelerator’s perspective. Host DRAM is also volatile, increasingly expensive as DRAM prices rise, and shared with the CPU’s own needs.

CXL memory expansion. Compute Express Link allows memory to be attached via a coherent fabric, enabling large pools shared among hosts and, eventually, accelerators. CXL improves DRAM capacity and flexibility, but it does not change DRAM economics—the memory is still DRAM at DRAM cost—and it sits behind a fabric with bandwidth and latency well below HBM. CXL addresses host-side capacity more than accelerator-side capacity.

GPU-direct NVMe storage. Technologies that allow accelerators to read directly from NVMe SSDs bypass the host CPU and reduce software overhead. They are valuable for loading models and datasets, but SSD bandwidth per device is single-digit to low double-digit gigabytes per second, latency is tens of microseconds, and the data still traverses PCIe. NVMe is a storage tier, not a memory tier, however direct the path.

Unified memory architectures. Some accelerator designs integrate CPU and GPU with a shared memory pool, expanding the memory visible to the accelerator. This helps, but the shared pool is typically conventional DRAM with bandwidth far below HBM, and total capacity is still bounded by DRAM cost and board area.

Model compression and distillation. Reducing model size through quantization, pruning, and distillation is effective but has quality floors and does not address KV cache, model multiplicity, or retrieval data.

Algorithmic KV cache reduction. Techniques such as grouped-query attention, multi-head latent attention, and cache quantization reduce KV cache per token. They are valuable and widely adopted, but context lengths and concurrency continue to grow faster than these techniques shrink the cache.

More HBM. Accelerator vendors continue to add stacks and adopt taller stacks. This is the most direct approach and will continue, but as the previous sections established, it advances slowly and expensively and cannot reach the multi-terabyte capacities that inference demands.

The common shortcoming is that every approach either keeps the memory far from the accelerator, keeps it in DRAM at DRAM cost, or reduces demand without eliminating it. None provides what the accelerator actually wants: terabytes of capacity, on-package, at bandwidth close to HBM, at NAND cost, with persistence. HBF is designed to provide exactly that.

9. Enter Sandisk High-Bandwidth Flash

Sandisk unveiled High-Bandwidth Flash in early 2025, shortly after completing its separation from Western Digital as an independent NAND flash company. The concept is conceptually simple and technically ambitious: take the architectural principles that make HBM fast—vertical stacking, through-silicon vias, a logic base die, a very wide parallel interface, and placement on the accelerator package—and apply them to 3D NAND flash instead of DRAM.

The resulting stack, as described in Sandisk’s public presentations, consists of multiple high-density NAND dies stacked vertically on a logic base die, connected by TSVs, and presented in a footprint and electrical interface intended to be compatible with HBM stack positions on an accelerator package. Sandisk has stated that HBF targets bandwidth comparable to HBM at the stack level while providing roughly eight to sixteen times the capacity per stack, at a significantly lower cost per bit. The stated target application is AI inference, with an explicit acknowledgment that HBF complements rather than replaces HBM.

Several enabling technologies make this plausible. Sandisk’s 3D NAND dies stack hundreds of memory layers in a single die, yielding per-die capacities far beyond any DRAM die. Its wafer-bonding manufacturing approach, in which CMOS logic and the memory array are fabricated separately and bonded, maps naturally onto a design where a base logic die manages a stack of memory dies. NAND’s inherent internal parallelism—many independent planes per die, many dies per stack—provides the concurrency needed to reach high aggregate bandwidth when the interface is wide enough. And the TSV-based stacking techniques pioneered for HBM provide the interconnect density required to expose that parallelism.

Sandisk has surrounded the technology with ecosystem signals. It formed a technical advisory board including prominent computer architects with deep experience in GPU and memory system design. It announced a collaboration with a major HBM manufacturer to standardize the HBF interface, indicating an intent to make HBF an industry category with multiple sources rather than a proprietary product. And it has communicated a timeline of sample availability in the second half of 2026 with first inference devices expected in early 2027, placing HBF on the near-term planning horizon for accelerator and system designers.

The positioning is disciplined. Sandisk does not claim that NAND achieves DRAM latency, nor that HBF is suitable for training or for the hottest inference data. It claims that for the large volume of inference data that is read-mostly and predictable—model pools, experts, idle cache, adapters, indexes—NAND’s latency is tolerable when bandwidth is high, and that the capacity and cost benefits are transformative for that data. In other words, HBF is aimed precisely at HBM’s greatest weakness while leaving HBM’s greatest strengths untouched.

10. How HBF Attacks the Capacity Problem Directly

HBF addresses each of the structural capacity constraints identified earlier, not by changing the physics of DRAM but by using a different memory technology whose physics favor capacity.

The net effect is that an accelerator package that today holds a few hundred gigabytes of HBM could, with a mix of HBM and HBF stacks, hold a few hundred gigabytes of HBM plus multiple terabytes of HBF. The accelerator’s hot working set stays in HBM. Its extended working set—everything it may need within the latency budget of a prefetch—moves from off-package tiers into HBF. The performance cliff that currently sits immediately below HBM is replaced by a gentler step to a tier that is still on-package and still high-bandwidth.

This directly addresses the costs enumerated earlier. A model that previously required eight accelerators for capacity reasons might fit in HBM plus HBF on two or three. KV cache for far more concurrent sessions can be held on-package, raising batch size and utilization. The full model and adapter pool can be hot. Retrieval indexes can be co-located. Restarts warm from local flash rather than network storage. Interconnect requirements shrink with accelerator count. Quantization can be less aggressive because capacity is less scarce.

Crucially, HBF achieves all of this without asking HBM to do anything differently. HBM continues to serve the hottest data at full bandwidth and DRAM latency. HBF sits beside it and absorbs the capacity burden that HBM was never economically suited to carry. This is what it means to solve HBM’s greatest weakness rather than to replace HBM.

11. HBM + HBF: A Division of Labor

The relationship between HBM and HBF is best understood as a division of labor governed by access pattern. Every piece of data an accelerator touches can be classified along two axes: how predictable its access is, and how fine-grained its access is. Data that is accessed unpredictably and at fine granularity must be in HBM. Data that is accessed predictably or at coarse granularity can be in HBF, staged into HBM ahead of need.

In an HBM-plus-HBF accelerator, HBM holds activations, attention scratch space, the weights of the currently executing layers or active experts, and the KV cache for sequences actively generating tokens. HBF holds the full model or model pool, the complete expert set, the KV cache for sequences that are paused between turns, the adapter library, embedding tables, retrieval index partitions, semantic caches, and any other data whose need can be anticipated.

A memory controller or runt ime layer moves data between the two tiers continuously. When the scheduler determines which expert a layer will need, it issues a prefetch from HBF to HBM. When a session goes idle, its cache is written from HBM to HBF. When a request arrives for an adapter, the adapter is staged from HBF before execution begins. HBM functions as a cache for HBF in much the same way that a CPU’s L2 functions as a cache for L3, with the important difference that inference execution is structured enough that most prefetches can be issued deterministically rather than speculatively.

This division allows each technology to operate where it is strongest. HBM is never asked to provide capacity it cannot economically provide. HBF is never asked to provide latency it cannot physically provide. The accelerator, for the first time, has both a fast tier and a large tier within the package boundary.

12. Workload Deep Dive: Large Language Model Inference

Dense large language model inference illustrates the HBM capacity problem in its simplest form. A dense model of several hundred billion parameters at 8-bit precision occupies several hundred gigabytes of weights. Placing this in HBM requires several accelerators even before any KV cache is allocated. Once concurrent sessions are added, the accelerator count rises further, and the fraction of each accelerator’s compute that is actually used during decode falls.

With HBF, the full weight set resides on-package in flash, and the runtime streams each layer’s weights into HBM immediately before that layer executes. Because layer execution order is fixed and known, this prefetch is fully deterministic. If HBF read bandwidth is sufficient to deliver a layer’s weights in the time the preceding layer computes, weights need not be permanently resident in HBM at all; HBM becomes a rolling window over the model. The accelerator count for a given model then drops toward the number required by compute and active KV cache rather than by total weight size.

Whether full weight streaming is achievable depends on the ratio of HBF bandwidth to the model’s per-layer weight volume and compute time. For decode at moderate batch sizes, where compute per layer is short, HBF bandwidth must be very high to keep up. At larger batch sizes, compute per layer is longer and the bandwidth requirement relaxes. This creates a virtuous cycle: HBF enables larger batches by freeing HBM for KV cache, and larger batches make weight streaming from HBF more feasible. Early deployments will likely use partial streaming—keeping some layers resident and streaming others—and tune the ratio to the workload.

Even without weight streaming, HBF benefits dense LLM inference through KV cache tiering. Freeing HBM of idle session cache increases active batch capacity. Prefix caching for shared system prompts and RAG contexts becomes practical at scale because the prefix store lives in persistent flash. Both effects raise tokens per second per accelerator and lower cost per token without any change to the model.

A practical consequence for buyers is that inference-optimized accelerators with HBM-plus-HBF configurations could serve models that are today considered “multi-node” on a single node, or serve current single-node models at substantially higher concurrency. The capacity headroom also extends the useful life of the hardware as models grow.

13. Workload Deep Dive: Mixture-of-Experts Models

Mixture-of-experts models are the clearest example of a workload where HBM capacity, rather than bandwidth or compute, is the binding constraint. An MoE model contains many expert sub-networks per layer—often dozens or more—and a routing mechanism selects a small number of experts for each token. The total parameter count can be enormous, but the compute per token is proportional only to the active experts. This is precisely why MoE has become the dominant frontier architecture: it delivers the quality of a very large model at the inference cost of a much smaller one.

The catch is memory. All experts must be resident somewhere fast enough that the selected ones can be loaded within the token latency budget. In an HBM-only system, this means all experts must be in HBM, which means the model is spread across enough accelerators to hold the full parameter set even though most of those accelerators are compute-idle most of the time. Expert parallelism across many accelerators also introduces all-to-all communication over interconnect for every layer, which is costly and complex.

HBF changes this calculus fundamentally. The complete expert set resides in HBF on the accelerator package. Routing decisions for a layer are computed before that layer executes, giving the runtime advance notice of which experts are needed. Those experts are streamed from HBF into HBM as large contiguous blocks—an access pattern ideally suited to NAND’s bandwidth-rich, latency-tolerant profile. Only the active experts occupy HBM at any moment. The number of accelerators required drops toward what compute and active cache demand, and the all-to-all interconnect traffic shrinks or disappears.

Expert access patterns also have favorable locality. In practice, expert selection is not uniform; some experts are chosen far more often than others, and within a batch the same experts tend to recur. A runtime can keep frequently used experts resident in HBM and stream rarer ones from HBF on demand, achieving high HBM hit rates and low staging traffic. Because expert weights are read-only during inference, endurance is not a concern for this use case.

For the industry, the combination of MoE and HBF is significant. MoE decouples model capacity from compute; HBF decouples memory capacity from HBM. Together they allow very large, very capable models to be served on modest hardware footprints. This is likely to be the first and most compelling application of HBF, and the one where the economic case is easiest to demonstrate.

14. Workload Deep Dive: Long Context and KV Cache

Key-value cache is the memory structure that stores, for every token in a sequence’s context, the attention keys and values computed at each layer. It allows the model to attend to prior tokens without recomputing them. Its size grows linearly with context length and concurrent sequences, and for large models at long context it can dwarf the weights themselves. KV cache is the reason long-context inference is expensive and the reason concurrency is capped.

Active KV cache—the cache for a sequence currently generating tokens—must be in HBM, because attention reads it at every layer for every token. But a large fraction of KV cache at any moment is not active. In conversational applications, sessions spend most of their wall-clock time waiting for the user’s next message. In agentic workflows, sessions pause while tools execute. In batch processing, sequences queue for their turn. All of this idle cache occupies HBM today, or is evicted and recomputed at resume.

HBF provides a destination for idle cache that is far better than either option. Cache for a paused session is written to HBF—a large sequential write, well suited to NAND—and restored to HBM when the session resumes, at HBF bandwidth rather than through prefill recomputation. Restore time for even a large context is a small fraction of prefill time and consumes no compute. Because HBF is persistent, long-lived sessions survive node restarts.

For very long contexts, HBF also enables tiering within an active sequence. Attention research has shown that for long inputs, most attention mass concentrates on a subset of positions—recent tokens, attention sinks, and semantically relevant blocks. Systems that identify and keep only the most relevant cache blocks in HBM while holding the remainder in HBF can serve contexts far longer than HBM alone would permit, with the full cache available on-package should the model need it.

The endurance question is most acute for KV cache because it is written continuously. The mitigation is architectural: the hottest cache stays in HBM and only spill and park traffic reaches HBF. Writes are buffered and coalesced in the HBF base die. Cache may be compressed before spilling. Vendors will need to specify HBF endurance under realistic cache write patterns, and serving software will need to enforce policies that keep write volume within rated limits. These are solvable engineering problems, but they are real, and buyers should ask about them directly.

15. Workload Deep Dive: Multi-Tenant Serving and RAG

Commercial inference is multi-tenant by nature. A platform serves many customers, each potentially with fine-tuned adapters, private retrieval corpora, custom system prompts, and distinct latency requirements. The economics depend on packing tenants densely onto shared accelerators while meeting each tenant’s service levels. HBM capacity is the primary limit on that density.

Parameter-efficient fine-tuning has made customization cheap to create but expensive to serve at scale. Each LoRA adapter is small, but a platform may hold thousands. Only those resident in HBM can be served without a cold start. With HBF, the entire adapter library for a node’s tenant population lives in near-package flash and is loaded into HBM on demand in the time it takes to schedule the request. Adapter cold starts, which today may cost hundreds of milliseconds from network storage, become negligible. Tenant density per node rises, and provisioning a new tenant means writing a few hundred megabytes to HBF once rather than distributing it at every restart.

The same applies to model pools. Platforms serving multiple base models can hold all of them in HBF and rotate them through HBM as demand shifts across the day. Low-traffic models no longer require dedicated accelerators; they share hardware with high-traffic models and are staged in seconds when needed.

Retrieval-augmented generation adds the vector index. Enterprise RAG systems index millions to billions of document chunks, producing indexes of hundreds of gigabytes to terabytes. These indexes live today in CPU DRAM or on SSDs, in a separate system from the accelerator. Every query incurs a network round trip to the retrieval service before generation can begin. With HBF, the warm partitions of the index can reside on the accelerator package alongside the model. Embedding, search, reranking, and generation execute within one package without leaving it. Time-to-first-token falls, the deployment topology simplifies, and for on-premises and sovereign deployments, sensitive data moves less.

Semantic caching closes the loop. A store of prior queries and responses, indexed by embedding, allows repeated or near-repeated questions to be answered without generation. At enterprise scale this cache is large, and holding it in HBF makes hits nearly free. For workloads such as customer support and internal knowledge assistants where query repetition is high, the cost per query falls substantially.

For system integrators and OEMs, this combination enables a new product category: a self-contained inference appliance in which model, adapters, index, and cache all reside on the accelerator package, deployable in an enterprise without a distributed retrieval and storage infrastructure. That is a meaningful commercial opportunity in markets that value simplicity and data locality.

16. Side-by-Side: HBM vs HBF vs Alternatives

The table below situates HBF relative to HBM and to the alternatives discussed earlier. Figures indicate technology class and relative positioning rather than vendor-specific commitments.

HBF is the only entry that combines on-package placement, HBM-class bandwidth ambition, multi-terabyte capacity, NAND cost, and persistence. Its disadvantages—microsecond latency and finite endurance—are exactly the properties that matter least for the capacity-dominated data classes that overflow HBM today.

17. Bandwidth, Latency, and Endurance: The Honest Physics

A credible analysis must be candid about what HBF cannot do. NAND flash and DRAM are different physical devices with different fundamental characteristics, and no amount of packaging ingenuity erases those differences.

Latency. Reading a NAND page involves sensing charge levels in cells and transferring data through page buffers, a process measured in microseconds—tens of microseconds for high-density cells. DRAM access is measured in tens of nanoseconds. HBF is therefore two to three orders of magnitude slower than HBM for a single random access. This is why HBF cannot hold activations or active KV cache, and why its value depends entirely on prefetching and streaming. For data accessed predictably or in large blocks, latency is hidden; for data accessed unpredictably at fine granularity, it is not.

Bandwidth. NAND’s per-die read throughput is far below DRAM’s. HBF reaches high aggregate bandwidth by exploiting parallelism—many planes per die, many dies per stack, a very wide interface—rather than by making individual accesses fast. Achieving HBM-class bandwidth at the stack level is an aggressive target that depends on the interface width, the number of concurrent NAND operations the base die can orchestrate, and the read characteristics of the NAND used. Shipping products may land below the target; buyers should evaluate measured sustained bandwidth under realistic mixed access, not headline figures.

Endurance. NAND cells degrade with program/erase cycles. High-density NAND tolerates a limited number of cycles before error rates rise unacceptably. HBF must therefore be used predominantly for read-mostly data, with writes managed through wear leveling, buffering, and workload policy. Model weights, experts, adapters, and indexes are read-mostly and pose no endurance concern. KV cache spill is the write-heaviest intended use and must be bounded. Vendors should publish endurance ratings under representative inference write patterns.

Tail behavior. NAND requires background management—garbage collection, wear leveling, read disturb handling—that can introduce latency variability. Inference SLOs are sensitive to tail latency. The HBF base die must schedule background work so that it does not intrude on serving-time accesses. Enterprise SSDs have solved this class of problem; HBF must solve it again in a new context.

Temperature. NAND performance, endurance, and data retention degrade at elevated temperatures. Sitting beside a compute die dissipating several hundred to over a thousand watts is a hostile environment. HBF-equipped packages will require careful thermal design and will likely be deployed in liquid-cooled systems.

None of these limitations undermines the core proposition, because the proposition was never that HBF behaves like DRAM. It is that for the large majority of inference memory by volume, DRAM behavior is unnecessary, and NAND behavior—with high bandwidth, on-package placement, and intelligent staging—is sufficient. The physics constrain how HBF is used; they do not prevent it from solving the capacity problem for the data classes it targets.

18. Economics: Cost per Gigabyte, TCO, and Supply

The economic argument for HBF operates at three levels: component cost, system total cost of ownership, and supply-chain structure.

At the component level, NAND cost per gigabyte is a small fraction of DRAM cost per gigabyte and a much smaller fraction of HBM cost per gigabyte. Even after accounting for the additional cost of TSV stacking, the base logic die, and HBM-compatible packaging, HBF should deliver capacity at a cost per gigabyte far below HBM. This is what makes multiple terabytes on-package economically thinkable where multiple terabytes of HBM would be absurd.

At the system level, the TCO case is built on the mechanisms described throughout this article. Fewer accelerators per model when capacity rather than compute was the constraint. Higher batch size and utilization from KV cache tiering. Reduced interconnect requirements. Elimination of adapter and model cold starts. Faster restarts and recovery from persistent provisioning. Co-located retrieval reducing external infrastructure. Less aggressive quantization preserving quality. Each of these reduces fully loaded cost per token. The magnitude depends on workload: MoE serving, long-context serving, and multi-tenant platforms see the largest effects; small single-model deployments see little.

At the supply-chain level, HBF diversifies AI memory sourcing. HBM comes from a few DRAM suppliers whose capacity is fully allocated, and the diversion of DRAM wafers to HBM has raised prices across the memory market. NAND comes from a broader base with different capacity dynamics. Shifting the capacity component of AI memory demand toward NAND reduces dependence on the most constrained part of the supply chain and may ease pressure on HBM by ensuring it is used only where its bandwidth is required. For buyers negotiating multi-year infrastructure commitments, this diversification is itself valuable.

19. Packaging, Thermal Design, and Footprint Compatibility

HBF’s footprint compatibility with HBM is one of its most strategically important attributes. If HBF required new stack positions, a new interposer design, or a new interface, adoption would depend on ground-up accelerator redesigns and would take years. By targeting the same physical and electrical footprint as an HBM stack, HBF allows accelerator designers to allocate existing stack positions between HBM and HBF, producing inference-optimized variants of designs already in production or development.

This flexibility enables product segmentation. A training-oriented part uses all HBM. An inference-oriented part might use half HBM and half HBF, or a quarter HBM and three-quarters HBF, depending on the target workload. The same compute die, interposer, and substrate serve both, reducing development cost and time to market. Buyers benefit from configuration choice matched to their memory profile.

Thermal design is the principal packaging challenge. HBM stacks are already difficult to cool because heat from the compute die conducts into them and heat generated within the stack must escape through its top. NAND has different thermal properties than DRAM and greater sensitivity to sustained high temperature in terms of endurance and retention. Designers integrating HBF must model thermal gradients across the package, potentially position HBF stacks in cooler locations, and rely on liquid cooling. The upside is that NAND generates little heat at idle since it requires no refresh, and the intended HBF workload—warm data accessed at moderate average intensity—produces less thermal load than continuously active HBM.

Packaging capacity is a shared constraint. HBF stacks consume the same advanced packaging and interposer capacity as HBM-equipped accelerators, which is already a bottleneck. HBF does not create new demand for stack positions, but it does compete for assembly throughput. Qualification of NAND stacks in packaging flows designed for DRAM will take engineering effort on the part of accelerator vendors and their packaging partners.

Interface standardization ties these considerations together. If the HBF interface is standardized in the manner of HBM, multiple NAND suppliers can produce interoperable stacks for multiple accelerator vendors, packaging partners can qualify a single interface, and software can target a common model. Sandisk’s announced collaboration with an HBM supplier on standardization is a meaningful step; buyers should watch for formal standards-body activity as a maturity signal.

20. Software: Making Capacity Usable

Capacity is only valuable if software can use it. HBF requires the inference stack to become tier-aware at several layers, and the maturity of that software will pace adoption as much as hardware availability.

At the driver and memory-management level, HBF must be exposed as a distinct address space with known characteristics, and the runtime must be able to issue asynchronous transfers between HBF and HBM without host involvement, ideally through dedicated DMA engines. Address translation, wear leveling, and error correction should be handled in the HBF base die so the runtime sees a flat, reliable space.

At the serving-engine level, the KV cache manager must support migration between HBM and HBF based on session state. The model loader must support placing weights in HBF and streaming layers or experts into HBM on schedule. The adapter manager must treat HBF as the library and HBM as the active set. The scheduler must account for staging latency and issue prefetches early enough. Several open-source serving engines already implement KV offload to host memory and disk; extending them to an on-package flash tier is an evolution rather than a rewrite, and the structured nature of inference makes prefetch decisions largely deterministic.

At the orchestration level, fleet controllers must track what each node holds in HBF and route traffic accordingly, provision new assets without disrupting serving, and treat HBF content as persistent state that survives restarts. Retrieval systems must support HBF as an index backend. Monitoring must expose HBF bandwidth utilization, hit rates, prefetch accuracy, wear, and thermal state so operators can tune policies.

The encouraging reality is that the software problem is well-understood. Tiered memory management, prefetching, and cache policies are mature disciplines. The inference community has already built much of the machinery for offloading to slower tiers. HBF makes that machinery far more effective by providing a tier that is fast enough for the offloaded data to remain useful. The work is integration and tuning, not invention.

21. Ecosystem, Standardization, and Timeline

HBF reaches the market only through accelerator vendors. Sandisk manufactures the stacks, but a GPU or custom accelerator designer must integrate them into a package, qualify them thermally and electrically, and expose them to software. The composition of Sandisk’s technical advisory board—including architects with deep GPU and memory system experience—and its public collaboration with an HBM supplier on interface standardization indicate engagement with the parties whose decisions matter.

Custom accelerator programs at hyperscalers are natural early adopters. They design for inference specifically, control their own software stacks, and have the largest absolute exposure to HBM capacity costs. Merchant GPU vendors may follow with inference-specific SKUs once the standard is settled and early deployments have validated the approach.

On the supply side, the concept is not inherently exclusive to Sandisk. Other NAND manufacturers possess the process and bonding technology to produce comparable stacks, and a standardized interface would enable multi-sourcing. For buyers, multi-sourcing is important; a tier central to inference economics should not depend on a single supplier.

The publicly communicated timeline places HBF samples in the second half of 2026 and first AI inference devices in early 2027. This positions HBF within the planning horizon for hardware decisions being made now for 2027 and later deployments. Procurement teams with long cycles should be engaging accelerator and server vendors on HBF roadmaps and configuration options today.

For server OEMs, ODMs, distributors, and system integrators, HBF represents a differentiation opportunity in a market that has commoditized around a small number of reference designs. Inference-optimized systems with tuned HBM-to-HBF ratios, validated tiering software, and integrated retrieval could command premium positioning in enterprise, sovereign, and regional cloud markets.

22. Risks and Limitations

Bandwidth shortfall. If shipping HBF delivers substantially less than HBM-class bandwidth, weight streaming and expert swapping for latency-sensitive serving become marginal, and HBF’s value narrows to adapter and model staging, session cache parking, and retrieval co-location. These remain valuable, but the headline accelerator-count reduction would shrink.

Tail latency. Background NAND management must not disrupt serving-time accesses. Failure here would make HBF unreliable for any latency-bounded staging.

Endurance under cache workloads. If KV cache spill volumes exceed rated endurance, device life shortens. Software policy and vendor specifications must align.

Thermal integration. NAND adjacent to a high-power compute die is a challenging environment. Early designs may be limited in HBF stack count or restricted to liquid-cooled systems.

Software lag. If serving engines and orchestration platforms are slow to add tier awareness, early HBF hardware will be underutilized.

Ecosystem hesitation. If major accelerator vendors conclude that larger HBM stacks, CXL, or algorithmic footprint reductions suffice, HBF could remain niche despite technical merit.

Competing trends. HBM capacity continues to grow; attention algorithms continue to shrink KV cache; model compression continues to improve. Each erodes part of the problem. HBF’s defense is that it addresses capacity, cost, persistence, and proximity together, which no single alternative does.

Security obligations. Persistent on-package data requires encryption at rest and secure erasure—capabilities accelerators have not historically needed. Vendors must add them; operators must incorporate HBF into data governance.

23. Strategic Implications for Buyers and Vendors

For datacenter operators and cloud providers: Profile the memory footprint of your inference workloads now—weights, active and idle KV cache, adapter and model pools, retrieval indexes. Quantify how much of your accelerator spend is driven by capacity rather than compute. Engage vendors on HBF roadmaps and HBM-to-HBF configuration options. Insist on workload-level benchmarks: tokens per second for models that exceed HBM, concurrency at SLO with cache tiering, session resume latency, adapter cold-start latency, and fully loaded cost per million tokens.

For enterprise infrastructure teams: HBF is most relevant if you run RAG at scale, serve multiple models or many fine-tunes, or need long-context capability. Evaluate whether a self-contained HBM-plus-HBF inference node could replace a distributed GPU, vector database, and storage architecture. Assess software readiness alongside hardware.

For server OEMs, ODMs, and integrators: Prepare inference-optimized product lines with configurable memory ratios and validated tiering software. Develop reference architectures for RAG appliances and multi-tenant serving nodes. Position on cost per token and deployment simplicity rather than raw accelerator count.

For memory and storage vendors: HBF elevates NAND from a commodity storage component to a differentiated AI infrastructure category. The value shifts toward stack design, base-die logic, and the software and services that make on-package flash usable. Engagement with standardization and with accelerator vendors is essential.

For investors: The HBM capacity constraint is one of the largest cost centers in AI infrastructure, and any technology that credibly relieves it addresses a very large market. Watch for accelerator design wins, standardization milestones, measured bandwidth and endurance data from samples, and serving-engine support as indicators of trajectory.

24. Outlook: Capacity as the New Competitive Axis

For the past several accelerator generations, the competitive axis has been bandwidth and compute. Product launches have headlined FLOPS and terabytes per second. As inference becomes the dominant workload and as models grow in total size faster than in active size, the competitive axis is shifting toward on-package capacity. The accelerator that can hold the largest model, the most concurrent long-context sessions, the largest adapter pool, and the largest retrieval index on a single package at acceptable bandwidth will win inference deployments on cost per token.

HBM will remain the bandwidth engine of every such accelerator. HBF, or something very like it, will become the capacity engine. Accelerators will be specified by both, and inference-optimized parts will be distinguished by their HBM-to-HBF ratios. The software stack will treat the two as a unified hierarchy, and the current performance cliff below HBM will be remembered as an artifact of a transitional era.

Further ahead, on-package flash may absorb logic functions—embedding lookup, expert selection, cache compression—performed within the stack, following the pattern by which HBM base dies have taken on more responsibility. The line between memory and storage inside the accelerator will continue to blur, and the concept of a model “fitting” on an accelerator will give way to the concept of an accelerator holding a working set of arbitrary size at graduated speeds.

The trajectory is clear even if the timeline and the specific winners are not. Capacity is the constraint that inference economics most needs relieved, and the industry is converging on the recognition that relief must come from a tier beside HBM rather than from HBM alone.

25. Conclusion: Preserving HBM’s Strength by Curing Its Weakness

HBM is indispensable. Its bandwidth and latency, achieved through stacked DRAM placed millimeters from the compute die, are the reason AI accelerators work, and no technology on any roadmap challenges that role. Training will remain HBM territory. The hottest inference data—activations, attention scratch, active KV cache, active weights—will remain HBM territory. Every accelerator of consequence will continue to integrate HBM in growing bandwidth and stack counts.

But HBM’s greatest weakness is capacity, and that weakness is structural. DRAM density scaling has slowed, stack height is thermally and mechanically bounded, yield compounds with die count, package area is finite, supply is concentrated, and cost per gigabyte is the highest of any volume memory. Meanwhile, AI memory demand has exploded across model size, MoE architectures, context length, concurrency, model and tenant multiplicity, and retrieval. The result is an industry that overprovisions accelerators to obtain memory, compresses models beyond preference, evicts and recomputes cache, tolerates cold starts, and carries interconnect and power costs that exist only because near-compute capacity is scarce.

Sandisk High-Bandwidth Flash is the first technology designed to address that weakness directly, in the same physical position as HBM, at HBM-class bandwidth ambition, with eight to sixteen times the capacity per stack, NAND cost per bit, and persistence. It does not ask HBM to change. It relieves HBM of the capacity burden it was never economically suited to carry, and in doing so it addresses accelerator count, batch efficiency, cold starts, recovery time, interconnect scale, and supply concentration simultane ously.

The honest caveats remain. NAND latency is microseconds, not nanoseconds, so HBF depends on prefetching and streaming rather than random access. Endurance is finite, so write-heavy data must be managed. Thermal integration beside a high-power compute die is hard. Bandwidth targets must be validated in shipping silicon. Software must become tier-aware. Accelerator vendors must integrate the technology, and the interface must be standardized for the ecosystem to scale. None of these caveats undermines the core proposition, because the proposition was never that flash behaves like DRAM. It is that most inference memory by volume does not need DRAM behavior, and that giving it a fast, large, persistent home on the package changes the economics of AI.

For buyers, vendors, and investors, the guidance is consistent: treat HBM and HBF as complementary, profile your workloads by access pattern and capacity, engage vendors on roadmaps and configuration ratios now, demand workload-level benchmarks, and evaluate software readiness alongside hardware. HBM will continue to make AI fast. HBF is positioned to make AI fit. Together they turn the most expensive constraint in the AI datacenter into a solved problem.

FAQ: HBM Capacity Limits and Sandisk HBF