Breaking the mould, Korean electronics giant Samsung has successfully developed the world’s first High Bandwidth Memory (HBM) with built-in AI processing hardware called HBM-PIM (for processing-in-memory). In contrast to the usual setup, this new RAM will be part of the CPU and GPU chip thus forming a single chip which is expected to be of great significance.
Here is a brief summary of how Samsung made it possible. The company took its HBM2 Aquabolt chips and added Programmable Computing Units (PCU) between the memory banks. This simpler setup with 16-bit floating-point values has a limited instruction set and can move data around while performing multiplications and additions.
This is the story of a single PCU and usually, there are many queued next to each other. Samsung succeeded in amping the speed of the PCUs to 300MHz resulting in 1.2 TFLOPS processing power per chip. Even though the per chip power usage is unchanged, this setup has dropped the overall system energy consumption by 71%. This is achieved by eliminating the need to move the data twice (read input and write output) and now the data doesn’t move places.
The new PIM arrangement has also managed to double up the system performance thanks to machine learning and inference tasks. In addition, the HBM-PIM design is compatible with HBM2 which means to upgrade the system to the new setup, all it requires is a bit of tweak on the software side and switch it from regular mode to in-memory processing mode keeping the hardware intact.
Some limitations caused by the new HB-PIM setup is that it brings down the total memory capacity to half as the PCUs occupy the space used by memory banks. But Samsung worked around this by splitting the difference and combining a 4 Gb PIM chips with 8 Gb regular HBM2 dies and used four of them to create 6 Gb stacks. At the moment Samsung has sent out samples to companies developing AI accelerators who would then approve the design sometime in July. It will be a while before we see them implemented on consumer-level hardware.
Read interesting news, reviews as well as tips & tricks on TechnoBugg website, and stay updated with the latest happenings of the tech world on the go with Technobugg App. Also follow on Google News and join our Telegram channel for the latest updates.
