SK hynix develops KV cache tiering for AI inference memory
The chipmaker said on its Q3 2026 earnings call that SALT-KV and CMM Hybrid split KV cache across HBM, DRAM and SSDs as agentic AI grows.
SK hynix said on its Q3 2026 earnings conference call on October 28 that it is developing memory tiering technologies for KV cache, as the spread of agentic AI and long context drives rapid growth in KV cache size, according to inews24.com.
The company said storing all KV cache in HBM and DRAM is limited by capacity and cost. It said a tiered memory structure that splits data across HBM, DRAM and SSD by access frequency is becoming more important.
SK hynix named two technologies under development. SALT-KV is described as software that judges the reuse value and storage cost of each KV cache and optimizes data placement among HBM, DRAM and SSD. CMM Hybrid is a CXL-based hybrid memory technology that stores the whole KV cache on SSD while preloading likely-needed data into DRAM.
The chipmaker said SSDs are already used for KV cache offload in some AI systems and expects NAND to gain new demand opportunities for storing and reusing context needed for AI inference. It added that it is specifying the application areas and required performance of enterprise SSDs in inference environments through system demonstrations based on real AI workloads.
In the mid-to-long term, SK hynix plans to address a new memory tier between HBM and SSD with High Bandwidth Flash (HBF), which combines NAND's capacity and cost efficiency with high bandwidth.
Quick answers
What are SALT-KV and CMM Hybrid?
SALT-KV is software that judges the reuse value and storage cost of each KV cache and optimizes data placement among HBM, DRAM and SSD. CMM Hybrid is a CXL-based hybrid memory technology that stores the whole KV cache on SSD while preloading likely-needed data into DRAM.
Why does SK hynix expect new NAND demand?
The company expects NAND to gain new demand opportunities for storing and reusing the context needed for AI inference, and said SSDs are already used for KV cache offload in some AI systems.
What is High Bandwidth Flash?
HBF is a planned memory tier between HBM and SSD that SK hynix says combines NAND's capacity and cost efficiency with high bandwidth, targeted for the mid-to-long term.