Harvard-linked study finds reasoning models get stuck on their first few words

Banning default opening tokens and adding an example lifted compliance by 34 to 41 percentage points across tested models AI reasoning models are supposed to think step by step. A new paper suggests they mostly start those steps the same way, and that habit makes them harder to steer. Researchers found that two cheap tricks, banning the usual opening words and adding an example, improved how well reasoning models followed instructions by 34 to 41 percentage points. For a field that spends heavily on bigger models, that is a large gain from tweaking a few words at the start. What the researchers found The paper is titled “The First Few Tokens Matter: Understanding Chain-of-Thought Controllability.” It was posted to arXiv on October 29, 2026, as arXiv:2610.40939, and runs 20 pages with 15 figures. The authors include Harvard student Hugh Van Deventer and colleagues. They are affiliated with Harvard and the Cambridge Boston Alignment Initiative. Their focus is chain-of-thought, or CoT. That is the visible “scratchpad” reasoning a model writes before giving a final answer. The study pays particular attention to large language models such as the OpenAI o1 series. The core finding is som...