Lost in the middle
A failure mode of long-context LLMs where retrieval accuracy is high for content at the start and end of the prompt but drops in the middle. Documented in a 2023 paper and still observable in 2026 frontier models at extreme context lengths.
Background
"Lost in the middle" names an empirically observed failure mode in which a language model uses information placed near the start or end of a long context far more reliably than information buried in the middle, even when all of it fits inside the model's context window. The effect shows up as a U-shaped accuracy curve: performance on a question is highest when the relevant passage sits at the beginning or the tail of the prompt and dips noticeably when the same passage is positioned in the middle. It arises from how attention and positional encoding behave over long sequences — the model's ability to attend to and retrieve a token is not uniform across positions, so relevant content competing against a lot of surrounding text at middle positions gets effectively down-weighted. A concrete example: give a model twenty retrieved documents and ask a question whose answer lives in document ten, and it may miss it, yet answer correctly when that same document is moved to position one or twenty. This matters directly when building AI software, especially RAG systems, because it means simply stuffing more retrieved chunks into the prompt can hurt rather than help. Practical responses include retrieving fewer but better chunks, re-ranking so the most relevant passages sit at the edges of the context, and not assuming a large context window guarantees the model will actually use everything inside it.