Keep the prompt. Drop the middle.

Prefix Sliding caps the memory cost of long reasoning. The model keeps the prefix—system instructions, tools, and the task prompt—plus a sliding window of the latest few thousand tokens, and discards older intermediate reasoning as it continues. 1

On Qwen3-1.7B, the paper reports that Prefix Sliding can make existing models 3× faster without training while matching full-attention performance. A custom FlashAttention kernel on one 80GB H100 plateaus around 5,000 tokens per second once the window fills, while full attention keeps slowing. Prefix Sliding also beats last-k, summarization, and vanilla sliding-window baselines on AIME25 under a fixed local window. RL training with Prefix Sliding enables rollouts beyond 100,000 tokens. 1

The gains shrink on short tasks. HealthBench averages about 2,086 tokens, so a 2,048 window rarely slides. LiveCodeBench needs a window of at least 16,384 to match full attention when early code leaves the window during long comment-heavy reasoning. Large tool or file outputs can flood the window the same way. 1

A first build can start from the public repo at github.com/Muennighoff/prefix-sliding, try window 4096 on long-horizon tasks, raise the window for coding traces, and shadow-evaluate answer quality, p95 latency, peak memory, and flood failures beside a full-attention rollback. 23