Google DeepMind paper reveals recirculation method improves transformer context handling

Google DeepMind just published a paper that might quietly change how language models process text. The technique, called “recirculation,” takes activations from a transformer’s deeper layers and mixes them back into its shallower layers during inference. The paper, co-authored with researchers from the University of Texas at Austin, demonstrates that this lightweight recurrent connection delivers performance gains that rival, and in some cases surpass, full fine-tuning. No retraining required. No major architectural surgery. What recirculation actually does Recirculation breaks the one-way flow of transformer processing. A fraction of the activations computed in the model’s deeper layers gets fed back into the shallower layers during token generation. The model essentially gets a second pass at understanding its own internal representations, allowing it to refine what the paper calls its “belief states” across multiple steps. The key distinction from existing approaches like chain-of-thought prompting or looped transformer architectures is that recirculation doesn’t require extra reasoning tokens or added architectural depth. It’s a bolt-on modification to how inference runs, not a...