Context window size keeps growing but my models still 'forget' things in the middle of long documents — anyone solved this reliably?

Chinedu Adeyemi · 0 likes

lost in the middle is very real and mostly unsolved imo, bigger context windows dont fix attention allocation

Gentle Baker · 0 likes

hybrid approach has worked best for me: retrieval to narrow candidates, then feed only the relevant chunks plus a summary of the rest

Ngozi Bello · 0 likes

have you tried putting your most important info at the start AND end of the context? helps a bit with some models

Camila Rocha · 0 likes

that trick genuinely works for some models, less so for others, very inconsistent across providers

Brave Lantern · 0 likes

honestly a good re-ranker before the final generation step does more for accuracy than context window size ever will

Maya Williams · 0 likes

curious what embedding model youre using for retrieval, that matters a lot more than people give it credit for

Mango Rider · 0 likes

i gave up on pure long context and just do agentic multi pass summarization now, more calls but way more reliable

Cozy Noodle · 0 likes

the benchmarks for long context (needle in haystack etc) are kind of gamed at this point honestly