03 Memory Tech

Memory Technology and Optimizations

Modern memory hierarchy design relies on matching implementation technologies to specific levels based on speed, cost, and capacity requirements,. Technologies closer to the processor prioritize access time and bandwidth, while lower levels prioritize density, nonvolatility, and cost-efficiency.

SRAM (Static Random Access Memory)

SRAM serves as the primary technology for integrated processor caches (L1, L2, L3) due to its high speed and seamless integration with CPU logic.

• Cell Structure: Utilizes six transistors per bit to retain charge statically. This design prevents data disturbance during read operations and requires minimal standby power for data retention,.
• Cache Integration:
  • SRAM arrays are designed with widths matching the cache block size.
  • Tag bits are stored and read in parallel with the data block, enabling entire blocks to be written or read in a single clock cycle.
• Performance Scaling: Access time scales proportionally with the number of blocks in the cache. Energy consumption scales dynamically based on the number of blocks accessed and statically based on the total number of bits (leakage).

DRAM (Dynamic Random Access Memory)

DRAM provides the foundation for high-capacity main memory.

• Cell Structure: Achieves high density by using only one transistor and one capacitor per bit.
• Dynamic Constraints:
  • Capacitors leak charge over time, requiring periodic refresh operations.
  • Memory controllers must access and write back every row within a strict time window (e.g., $64\text{ ms}$), typically consuming $<5\%$ of total operational time,.
  • Reading data destroys the stored charge; accessed rows must be buffered and subsequently written back to the cell,.
• Addressing: Address lines are multiplexed to reduce chip pin count. Addresses are split into two sequential signals: the Row Access Strobe (RAS) and the Column Access Strobe (CAS).

SDRAM and DDR Enhancements

To bridge the bandwidth gap between fast processors and slower memory, DRAM integrates advanced internal topologies and interfaces.

• Synchronous DRAM (SDRAM): Introduces a clock signal to the interface, eliminating synchronization overhead and enabling burst transfer modes,. Burst transfers stream multiple sequential data elements without requiring repeated column addresses.
• Double Data Rate (DDR): Transfers data on both the rising and falling edges of the clock signal, effectively doubling peak data throughput.
• Banking Architecture:
  • Internal memory is divided into independent banks (e.g., 16 banks in DDR4).
  • Banks allow interleaved operations. A precharge or row-activation delay in one bank can be hidden by concurrently reading from an open row in another bank,.
  • Complete addressing maps require: $\text{Bank Number}+\text{Row Address}+\text{Column Address}$.
• Power Management: SDRAMs include power-down modes that ignore the external clock to disable active circuitry, maintaining only internal automatic refresh logic,. Low-Power DDR (LPDDR) reduces supply voltage for battery-constrained devices but restricts the number of chips per memory channel.

High Bandwidth Memory (HBM)

HBM represents a packaging paradigm shift, moving DRAM physically closer to the processor to maximize bandwidth and minimize transmission power,.

• Die Stacking: Multiple DRAM dies are stacked vertically.
• Interposer Integration (2.5D): The DRAM stack is abutted directly next to the CPU/GPU using a silicon interposer substrate containing high-density microscopic connections,.
• Performance: A standard configuration featuring four stacks can deliver data transfer rates approaching $4\text{ TB/s}$,.
• Vertical Stacking (3D): Places memory dies directly on top of the processor logic. Due to severe thermal and electrical noise constraints, current 3D implementations are restricted to SRAM for L3 caches rather than DRAM,.

Nonvolatile Solid-State Storage

Solid-state technologies provide persistent secondary storage, entirely replacing magnetic hard disks in personal mobile devices and dominating active server storage,.

Flash Memory (NAND)

• Architecture: An Electronically Erasable Programmable Read-Only Memory (EEPROM) that maintains data without power.
• Access Characteristics:
  • Optimized for sequential page reads ($512\text{ B}$ to $4\text{ KiB}$).
  • High initial random-access latency ($\approx 25\text{ mus}$), but capable of supplying bulk sequential data rapidly,.
• Write and Erase Asymmetry:
  • Data cannot be overwritten in place; entire blocks must be erased before pages can be written.
  • Write latency is severe—approximately $1500\times$ slower than SDRAM.
• Endurance Limits: Flash blocks undergo physical wear and tear, typically failing after $100,000$ write cycles. Flash controllers rely on software-managed write leveling to uniformly distribute writes across all blocks, maximizing device lifespan,.

Phase-Change Memory (PCM)

• Mechanism: Stores data by heating a substrate to alternate between crystalline and amorphous physical phases, measuring the resulting differences in electrical resistance.
• Advantages over Flash: Does not utilize transistors, theoretically allowing higher density,. Eliminates the bulk-erase requirement, significantly improving random write performance and overall endurance.

Memory Dependability and Error Correction

As memory capacities and densities increase, systems are engineered to withstand inevitable failures.

• Fault Classifications:
  • Hard Errors (Permanent Faults): Result from manufacturing defects or hardware wear-out (e.g., degraded Flash cells). Masked by activating redundant spare rows embedded in SRAM, DRAM, and Flash chips,.
  • Soft Errors (Transient Faults): Dynamic, non-destructive bit flips caused by environmental factors like alpha particles.
• Error Correcting Codes (ECC):
  • Parity: Adds $1$ overhead bit per $8$ data bits to detect single-bit errors.
  • SECDED ECC: Adds $8$ bits per $64$ data bits to correct single-bit errors and detect double-bit errors.
  • Chipkill: A distributed parity system (analogous to RAID for memory) that scatters data and ECC information across multiple memory chips,. Enables complete reconstruction of data even if an entire DRAM chip suffers total failure, a strict requirement for warehouse-scale data centers,.