SK hynix, In Collaboration With SanDisk, Unveils The New High Bandwidth Flash (HBF) Standard, Helping To Resolve AI Inference Bottlenecks, Targeting Up To 3TB/s Bandwidth
As AI accelerators face severe performance disparity between High Bandwidth Memory (HBM) and SSDs, SK hynix and SanDisk have collectively introduced a solution that aims to resolve the performance gap with a new standard called High Bandwidth Flash (HBF). Both memory manufacturers aim to create an ecosystem where various enterprises can develop compatible products.
HBF specifications allow for up to 512GB in a single configuration that stacks NAND dies in eight or 16 layers
In a nutshell, HBF possesses attributes that allow it to be positioned between HBM and enterprise SSDs. Similar to HBM, HBF relies on multiple memory dies that have been stacked together, but instead of DRAM, it utilizes NAND flash to increase storage capacity. Its primary role will be to complement capacity limits rather than outright replace HBM. LLMs can be processed through HBM, which is immediately required for computing, with HBF storing data in a larger capacity.
SK hynix’s and SanDisk’s latest specifications allow for HBF to go as high as 512GB, with bandwidth ranging from 0.4TB/s to 3TB/s. To understand how the technology will benefit AI inference performance, assume that there’s an AI model with 500 billion parameters, and if each gigabyte can house a billion parameters, that’ll come to 500GB. Since it’s not cost-effective to store the entire model in HBM, companies can leverage HBF to store these AI models.
High Bandwidth Flash also supports the UCIe (Universal Chiplet Interconnect Express) specification, allowing HBF to connect to various CPUs and GPUs to introduce an entire flexibility layer. Kim Chun-sung, head of solution development at SK hynix, has said that the company “will expand the boundaries between memory and storage and contribute to building new architectures that enhance overall system efficiency.”
As SK hynix and SanDisk make efforts to popularize the HBF standard, Samsung is already establishing a healthy relationship with the likes of NVIDIA by developing and mass-producing CMX solutions. Through the manufacturing of its V10 and V11 NAND flash, which can reach up to 500 stacked layers, the Korean giant will target ultra-fast storage requirements for AI servers wanting to quickly access data to increase GPU offloading, leading to better AI inference.
Which standard will pick up the pace? It looks like we’ll find out in the coming months.
News Source: SK hynix
About the author: Omar Sohail is a reporter and analyst for Wccftech’s mobile section, specializing in the technology and business of the mobile industry. His expertise lies in the intricate hardware supply chain, covering developments in semiconductor manufacturing, chip lithography, and camera sensor technology.
Follow Wccftech on Google to get more of our news coverage in your feeds.
