Samsung adds an AI processor to its High-Bandwidth memory to ease bottlenecks

A few months back, Samsung and Xilinx co-introduced an SSD with a Xilinx FPGA processor on-board, making computational storage very real. The SSD meant data could be processed where it resided rather than moving it to and from memory.

Now they’ve introduced High Bandwidth Memory (HBM) integrated with an artificial intelligence (AI) processor, called the HBM-PIM. The new processing-in-memory (PIM) architecture brings AI processing capabilities inside the memory rather than moving contents in and out to the processor, to accelerate large-scale processing in data centers, high-performance computing (HPC) systems and AI-enabled mobile applications.

HBM is a different kind of memory than DRAM. It is stacked vertically rather than spread out in a 2D design, so data moves less distnace through memory. As a result, HBM chips are smaller than DRAM but also more expensive, around 30% more than comparable capacity DRAM.

HBM is found on processors, CPUs and GPUs primarily, and usually sits physically right next to the processor chip to minimize travel. It’s found a home in HPC and AI applications, where money is less of an object than it is for the mass market.

Samsung notes that most of today’s computing systems are based on the von Neumann architecture, which uses separate processor and memory units to carry out data-processing tasks. That means moving data back and forth between the CPU and memory, through both the memory and CPU buses.

This sequential processing approach requires data to constantly move back and forth, resulting in a system-slowing bottleneck, especially when handling ever-increasing volumes of data. CPU vendors for years have attempted to compensate by making L1 through L3 caches larger, to the point they take up more space than the CPU core.