Groq plans stacked DRAM for its next inference chip to fix a memory problem

Groq built its reputation on speed. Its next chip may finally get the storage space to match. Industry sources revealed on October 20, 2026, that Groq plans to integrate 3D-stacked DRAM into its upcoming Language Processing Unit inference chips. The goal is to raise memory capacity and bandwidth for large AI models, without giving up the raw throughput that made the company’s hardware stand out. What Groq is reportedly building The plan centers on 3D-stacked DRAM, often shortened to 3DS, connected with through-silicon vias, or TSVs. According to the research findings, Groq’s version is meant to expand vertical DRAM capacity using standard interfaces. The emphasis is on scalability rather than chasing the most extreme bandwidth numbers possible. The SRAM trade-off The Groq 3 LPU carries 500 MB of on-chip SRAM. That memory delivers 150 TB/s of bandwidth per chip, which is approximately seven times the HBM bandwidth of comparable GPUs. Bandwidth is what matters most for token generation, the step where a model spits out words one after another. Groq’s design lets the Groq 3 LPX deliver up to approximately 3,400 tokens per second. The problem is capacity. A full 256-chip LPX rack offer...