\n| Table 1: Comparative Overview of DRAM-PIM vs. SRAM-CIM. This table provides a high-level comparison of the two main technological branches of PIM, establishing the fundamental trade-offs that define their respective roles in modern computing systems.[5, 13, 20]<\/span><\/td>\n | <\/td>\n | <\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n <\/p>\n DRAM-Based Processing-in-Memory Architectures<\/b><\/h2>\n <\/p>\n Architectural Principles: Leveraging DRAM’s Internal Bandwidth and Parallelism<\/b><\/h3>\n <\/p>\n Dynamic Random Access Memory (DRAM) has long been the cornerstone of main memory in computing systems. The primary motivation for developing DRAM-based PIM is to harness the massive internal bandwidth available within a DRAM chip. This internal bandwidth, accessible between the memory arrays and the chip’s periphery, can be an order of magnitude or more greater than the bandwidth of the external memory channel that connects the DRAM to the processor.<\/span>6<\/span><\/p>\nTo understand how PIM exploits this, it is essential to consider the hierarchical structure of a modern DRAM device, which consists of channels, ranks, banks, and subarrays.<\/span>14<\/span> By integrating small processing units at or near each memory bank, a PIM architecture can activate and process data from all banks in parallel. This bank-level parallelism allows the system to perform computations at a throughput that is dictated by the vast internal bus width, effectively bypassing the narrow off-chip interface for PIM-accelerated operations.<\/span>15<\/span> PIM units can be integrated at various levels of this hierarchy, but near-bank computing has emerged as the most commercially viable approach, offering a balance between performance gains and design feasibility.<\/span>14<\/span><\/p>\n <\/p>\n Commercial Implementations and Case Studies<\/b><\/h3>\n <\/p>\n Several major memory vendors have introduced commercial or near-commercial DRAM-PIM products, each tailored to a specific market segment and leveraging a different type of DRAM technology.<\/span><\/p>\n <\/p>\n Samsung HBM-PIM: Architecture of the Programmable Computing Unit (PCU)<\/b><\/h4>\n <\/p>\n Samsung’s “Aquabolt-XL” was the industry’s first commercially fabricated High Bandwidth Memory (HBM) device with integrated PIM capabilities.<\/span>15<\/span> This HBM2-PIM architecture embodies the PNM approach by integrating a <\/span>Programmable Computing Unit (PCU)<\/b> within each memory bank.<\/span>12<\/span> The PCU is an AI-focused engine, architecturally a 16-lane Single Instruction, Multiple Data (SIMD) array capable of performing 16-bit floating-point (FP16) operations, complete with its own lightweight control logic and register files.<\/span>15<\/span><\/p>\nA key aspect of Samsung’s strategy was to design the HBM-PIM as a “drop-in replacement” for conventional HBM2 modules. This was achieved by placing the PCUs at the bank boundary and preserving the standard JEDEC HBM2 interface and timing protocols.<\/span>12<\/span> This design choice significantly lowers the barrier to adoption for system integrators. In terms of performance, Samsung has reported a 2x performance improvement in applications like speech recognition and an energy reduction of over 70% compared to standard HBM.<\/span>12<\/span> When integrated into accelerator systems, such as the AMD MI-100 GPU or the Xilinx Alveo FPGA, HBM-PIM has demonstrated system-level performance gains of up to 2.5x and energy savings exceeding 60% for workloads dominated by General Matrix-Vector multiplication (GEMV) and Long Short-Term Memory (LSTM) operations.<\/span>4<\/span><\/p>\n <\/p>\n SK Hynix Accelerator-in-Memory (AiM): GDDR6 for High-Throughput Compute<\/b><\/h4>\n <\/p>\n SK Hynix has pursued a different path with its Accelerator-in-Memory (AiM) technology, which is based on high-speed GDDR6 memory.<\/span>24<\/span> AiM is explicitly designed as a “domain-specific memory” to accelerate memory-intensive machine learning workloads, particularly the GEMV operations that are fundamental to modern transformer models and Large Language Models (LLMs).<\/span>24<\/span><\/p>\nEach GDDR6-AiM chip integrates 32 Processing Units (PUs) and is capable of delivering 1 TFLOPS of compute throughput using Brain Floating Point 16 (BF16) precision.<\/span>25<\/span> The core architectural innovation is the concept of “all-bank parallelism,” which is enabled through an extended set of DRAM commands. These new commands, such as MACAB (MAC Across all Banks), allow a host controller to orchestrate simultaneous computation across all PUs in the chip.<\/span>24<\/span> This allows the architecture to fully leverage the massive internal memory bandwidth (rated at 0.5 TB\/s) for computation, which is approximately 8x greater than the chip’s external I\/O bandwidth (64 GB\/s).<\/span>27<\/span> SK Hynix’s AiMX accelerator card, which populates a board with multiple AiM chips, is designed to function as a co-processor to a GPU, offloading the memory-bound stages of LLM inference to improve overall system efficiency and throughput.<\/span>26<\/span><\/p>\n <\/p>\n DIMM-Based PIM: AXDIMM and the Path to Mainstream Adoption<\/b><\/h4>\n <\/p>\n While HBM and GDDR6 represent high-performance niches, PIM technology is also being integrated into the more ubiquitous Dual In-line Memory Module (DIMM) form factor for mainstream servers. Samsung’s Acceleration DIMM (AXDIMM) is a prime example, placing an AI engine on the buffer chip of a standard DIMM.<\/span>15<\/span> This allows the AXDIMM to perform parallel processing across the multiple memory ranks (sets of DRAM chips) on the module, a task not possible with conventional DIMMs.<\/span>23<\/span> While less tightly integrated than HBM-PIM, this approach provides a more straightforward upgrade path for existing server infrastructure. For AI-based recommendation applications, AXDIMM has demonstrated an approximate 2x performance gain with a 40% reduction in system-wide energy consumption.<\/span>23<\/span><\/p>\nSimilarly, the French company UPMEM has commercialized a DIMM-based PIM solution that integrates multiple general-purpose 64-bit in-order cores, which they call DRAM Processing Units (DPUs), onto their DRAM chips.<\/span>6<\/span> These DIMMs are compatible with standard commodity servers and can provide hundreds of gigabytes of compute-capable memory, targeting data-intensive applications like genomics, analytics, and search.<\/span>29<\/span><\/p>\nThe convergence of major vendors and academic research on the near-bank PNM architecture suggests that the industry has, for the present, settled on a pragmatic architectural template. This approach involves placing small, specialized compute units like SIMD or MAC arrays at the periphery of each memory bank. This strategy allows memory manufacturers to exploit the vast internal parallelism of DRAM to add computational value, all while preserving their core intellectual property and leveraging their highly optimized manufacturing processes without disruptive changes to the memory array itself.<\/span>17<\/span><\/p>\n\n\n\n| Specification<\/b><\/td>\n | Samsung HBM-PIM (Aquabolt-XL)<\/b><\/td>\n | SK Hynix GDDR6-AiM<\/b><\/td>\n<\/tr>\n\n| Base Memory Type<\/b><\/td>\n | HBM2<\/span><\/td>\n | GDDR6<\/span><\/td>\n<\/tr>\n\n| Compute Throughput<\/b><\/td>\n | 4.9 TFLOPS (per GPU with 4 cubes)<\/span><\/td>\n | 1 TFLOPS\/Chip<\/span><\/td>\n<\/tr>\n\n| Numeric Precision<\/b><\/td>\n | FP16<\/span><\/td>\n | BF16<\/span><\/td>\n<\/tr>\n\n| PIM Unit Architecture<\/b><\/td>\n | 16-lane SIMD Array (PCU) per 2 banks<\/span><\/td>\n | MAC-based Processing Unit (PU) per bank<\/span><\/td>\n<\/tr>\n\n| # of PIM Units<\/b><\/td>\n | 16 per stack<\/span><\/td>\n | 32 per chip<\/span><\/td>\n<\/tr>\n\n| Key Architectural Feature<\/b><\/td>\n | Drop-in replacement, JEDEC compliance<\/span><\/td>\n | All-bank parallelism via extended commands<\/span><\/td>\n<\/tr>\n\n| Target Workloads<\/b><\/td>\n | GEMV, Speech Recognition, LLMs<\/span><\/td>\n | GEMV, Transformers, LLMs<\/span><\/td>\n<\/tr>\n\n| Table 2: Technical Specifications of Commercial PIM Solutions. This table provides a direct comparison of the leading commercial DRAM-PIM products, highlighting their capabilities and target use cases based on published specifications.[4, 15, 24, 25]<\/span><\/td>\n | <\/td>\n | <\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n <\/p>\n Circuit-Level Modifications and Design Constraints<\/b><\/h3>\n <\/p>\n The integration of logic into DRAM is a non-trivial engineering challenge, governed by the unique and highly optimized DRAM manufacturing process. The chosen integration strategy has profound implications for design complexity, cost, and performance.<\/span><\/p>\nMost commercial PIM architectures have adopted the near-bank PNM approach precisely because it avoids modifications to the core DRAM subarray.<\/span>17<\/span> The subarray, which contains the 1-transistor-1-capacitor (1T1C) memory cells, is the densest part of the chip and is manufactured using a specialized process that is not optimized for high-performance logic. By placing compute units in the peripheral region alongside sense amplifiers and I\/O circuitry, manufacturers can implement a “dual-mode” functionality, where the chip can operate as either a standard memory or a PIM accelerator.<\/span>20<\/span> However, this peripheral area is extremely constrained in terms of physical space and power budget, which limits the complexity and performance of the integrated PIM logic.<\/span>17<\/span><\/p>\nMore experimental PUM approaches venture into the subarray itself, proposing circuit-level modifications to enable computation. One prominent technique involves leveraging the analog charge-sharing properties of DRAM. By activating multiple wordlines simultaneously (e.g., Triple-Row Activation or TRA), the collective charge from multiple cells is shared on the bitline, effectively performing a bitwise majority function, which can be used as a basis for other logic operations.<\/span>18<\/span> Other proposals, such as ReDRAM, suggest a “Dual-Row Activation” mechanism coupled with modest modifications to the sense amplifier circuitry to implement a complete set of bulk bitwise operations (AND, OR, XOR, NOT).<\/span>19<\/span> While these PUM techniques offer the highest degree of parallelism, they carry the significant risk of compromising the density, yield, and reliability of the commodity DRAM process, which has thus far limited their commercial adoption.<\/span>20<\/span><\/p>\n <\/p>\n Performance Analysis: Throughput, Latency, and Energy Efficiency Benchmarks<\/b><\/h3>\n <\/p>\n The performance of DRAM-PIM systems is highly dependent on the nature of the workload. These architectures excel in applications that are fundamentally memory-bound, where the ratio of data access to computation is high. For such workloads, PIM can provide substantial performance gains. For instance, an analysis of the UPMEM PIM system showed a 23x performance improvement over a high-end GPU for a GEMV kernel, but only when the GPU’s memory was oversubscribed, forcing it to access system memory and highlighting PIM’s strength in large-dataset scenarios.<\/span>31<\/span><\/p>\nHowever, PIM is not a panacea. For applications with significant computational requirements, the relatively simple processing units within PIM can become the new bottleneck, shifting the workload from memory-bound to compute-bound.<\/span>29<\/span> The effectiveness of PIM is also deeply tied to the software implementation, including how data is laid out across the memory banks, how parallelism is managed across the many PIM cores, and how synchronization is handled.<\/span>10<\/span><\/p>\nRoofline models, which plot computational performance against arithmetic intensity, clearly illustrate PIM’s value proposition: it dramatically raises the “memory roofline,” increasing the peak performance achievable for memory-bound applications.<\/span>14<\/span> For data analytics workloads, multi-level PIM architectures that exploit parallelism at the bank, chip, and rank levels have demonstrated throughput gains of up to 528x over baseline CPU systems.<\/span>8<\/span> Despite these impressive figures, a recurring challenge is load imbalance, where the work is not evenly distributed across all PIM cores. This is particularly problematic in applications with irregular data access patterns, such as graph mining, and can significantly degrade overall performance if not managed by sophisticated scheduling algorithms.<\/span>33<\/span><\/p>\n <\/p>\n SRAM-Based Compute-in-Memory for AI Acceleration<\/b><\/h2>\n <\/p>\n Architectural Principles: Optimizing for Low Latency and High Efficiency<\/b><\/h3>\n <\/p>\n While DRAM-PIM targets high-bandwidth computing, Static Random Access Memory (SRAM)-based CIM is engineered for a different set of goals: ultra-low latency and extreme energy efficiency. SRAM’s inherent advantages\u2014high speed, robustness, and compatibility with standard CMOS logic fabrication processes\u2014make it an ideal substrate for building specialized AI accelerators, particularly for inference tasks at the network edge.<\/span>5<\/span><\/p>\nThe architectural principle of SRAM-CIM is to perform the most fundamental and frequent operation in neural networks\u2014the Multiply-Accumulate (MAC) operation\u2014directly within the memory array.<\/span>5<\/span> By storing the neural network weights within the SRAM cells and streaming the input activations through the array, SRAM-CIM can perform thousands of parallel MAC operations in place, virtually eliminating the energy and latency costs associated with fetching weights and intermediate feature maps from off-chip memory.<\/span>3<\/span> This results in latency measured in single-digit nanoseconds and energy efficiency, often expressed in Tera-Operations per Second per Watt (TOPS\/W), that can be orders of magnitude better than conventional processors.<\/span>3<\/span><\/p>\n <\/p>\n Memory Cell and Peripheral Circuitry Modifications for Computation<\/b><\/h3>\n <\/p>\n Standard 6-transistor (6T) SRAM cells, which are optimized for density and stability in caches, are not well-suited for CIM. To enable in-memory computation, the bitcell and its surrounding peripheral circuits must be significantly modified.<\/span><\/p>\nA primary modification is the move from 6T to 8-transistor (8T), 9T, or even 10T bitcell designs.<\/span>5<\/span> The key innovation in these larger cells is the addition of a dedicated read port, which decouples the read and write data paths.<\/span>39<\/span> This separation is critical for CIM because it allows multiple wordlines to be activated simultaneously to perform a computation without causing the “read disturb” instability that would corrupt the stored data in a standard 6T cell.<\/span>39<\/span><\/p>\nThe peripheral circuitry surrounding the SRAM array is equally, if not more, critical. Since many SRAM-CIM designs operate in the analog domain to maximize efficiency, they require Digital-to-Analog Converters (DACs) to encode the digital input activations into analog signals (e.g., varying voltage levels or pulse widths on the wordlines).<\/span>41<\/span> After the computation is performed in the array\u2014typically as a summation of currents on the bitlines\u2014the resulting analog value must be converted back to the digital domain. This requires Analog-to-Digital Converters (ADCs) at the end of each column.<\/span>41<\/span> The design of these ADCs is a major challenge, as their power consumption and area can grow exponentially with the required precision, often becoming the dominant cost factor in the entire CIM macro.<\/span>41<\/span> Additional circuits, such as current mirrors for precise current control and sophisticated timing logic, are also necessary to ensure accurate MAC operations.<\/span>5<\/span><\/p>\n <\/p>\n SRAM-CIM in Practice: Macro Designs for CNN and Transformer Acceleration<\/b><\/h3>\n <\/p>\n These modified bitcells and complex peripherals are assembled into functional blocks known as CIM macros. A typical CIM macro consists of an array of thousands of modified SRAM cells that store a portion of a neural network’s weight matrix. When input activations are applied to the wordlines, the entire array performs a parallel matrix-vector multiplication in a single cycle.<\/span>3<\/span><\/p>\nA vast body of academic and industrial research is focused on designing and optimizing these macros for specific AI workloads. Numerous designs have been proposed to accelerate Convolutional Neural Networks (CNNs), Transformers, and Large Language Models (LLMs), with a strong emphasis on power-constrained edge devices.<\/span>46<\/span> Many advanced architectures are heterogeneous, combining SRAM-CIM with other memory technologies like Magnetoresistive RAM (MRAM) for non-volatility or Read-Only Memory (ROM) for fixed-function acceleration.<\/span>13<\/span> Others employ hybrid analog-digital techniques within the macro itself to strike a better balance between the raw efficiency of analog compute and the precision of digital logic.<\/span>46<\/span> The design of SRAM-CIM is not just about the memory itself; it is about creating a hyper-specialized accelerator where the memory cells act as programmable arithmetic elements, prioritizing computational function over pure storage density.<\/span><\/p>\n <\/p>\n Performance Analysis: Measuring Computational Density and TOPS\/W<\/b><\/h3>\n <\/p>\n The primary metric for evaluating SRAM-CIM performance is energy efficiency, measured in TOPS\/W. By performing computation in place, SRAM-CIM architectures dramatically reduce the data movement that dominates energy consumption in traditional systems. Published results for digital CIM architectures report efficiencies in the range of 1\u2013100 TOPS\/W, which represents a 100x to 1000x improvement over conventional CPUs.<\/span>3<\/span><\/p>\nSpecific research prototypes have demonstrated even more impressive figures, with some reporting efficiencies as high as 249.1 TOPS\/W or, in highly optimized cases, even 3707.84 TOPS\/W.<\/span>46<\/span> It is crucial to note that these figures are highly dependent on the fabrication technology node, the numerical precision of the data (e.g., 8-bit integer vs. 4-bit integer), and the specific operation being benchmarked.<\/span>5<\/span> While the energy efficiency is a major strength, the primary weakness of SRAM-CIM is its low storage density compared to DRAM.<\/span>20<\/span> This low density means that only small models or portions of larger models can fit within the CIM macro at one time. For large models, this can create a new bottleneck: the latency and energy required to frequently reload weights from off-chip DRAM into the SRAM array, potentially offsetting some of the gains from the in-memory computation itself.<\/span>48<\/span><\/p>\n <\/p>\n Comparative Analysis: Analog vs. Digital In-Memory Computing<\/b><\/h2>\n <\/p>\n Within the domain of SRAM-based CIM, the choice of computational paradigm\u2014analog or digital\u2014represents a fundamental design trade-off with profound implications for performance, efficiency, and accuracy. This choice dictates the architecture from the circuit level to the system level and is one of the most critical decisions for PIM designers.<\/span><\/p>\n <\/p>\n Principles of Analog CIM (AIMC): Leveraging Physics for Computation<\/b><\/h3>\n <\/p>\n Analog Compute-in-Memory (AIMC) performs calculations by directly harnessing the physical laws governing electrical circuits, most notably Kirchhoff’s current law and Ohm’s law.<\/span>3<\/span> In a typical AIMC macro, the digital values of an input vector are first converted into analog quantities, such as varying voltage levels or pulses of varying duration, by DACs. These analog signals are then applied to the wordlines of the memory array. Each memory cell, which stores a weight value as a variable conductance, modulates the current flowing from the wordline to the bitline. According to Kirchhoff’s law, all the currents from the cells in a single column are naturally summed together on the shared bitline.<\/span>51<\/span> | | | | | | | | | |
![\"\"](\"https:\/\/uplatz.com\/blog\/wp-content\/uploads\/2025\/11\/Processing-in-Memory-A-System-Level-Analysis-of-DRAM-and-SRAM-Architectures-for-Next-Generation-Computing-1024x576.jpg\")
<\/p>\n