Paper Finds DeepSeek AI Retrieval Varies by Up to 40.2 Points by Position

Researchers from ByteDance, Princeton University, Stanford University and UC Berkeley have published a paper titled "Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression" on an online archive, reporting that chunked KV-cache compression used by DeepSeek causes position-dependent swings in information retrieval accuracy.

According to the paper, the same performance degradation phenomenon appeared in DeepSeek-V4, DeepSeek-V4.1 and every compression model the researchers pretrained themselves. In tests with 128,000-token inputs, retrieval accuracy of the DeepSeek V4 series base models varied by up to 40.2 percentage points depending on where the information sat in the input. The gap reached 40.2 percentage points for the DeepSeek-V4 Flash base model and 34.8 percentage points for the V4 Pro base model.

Additional training narrowed those gaps. The difference fell to 19.1 percentage points for V4 Flash, 14.8 percentage points for V4 Pro and 6.1 percentage points for V4.1 Flash. Comparative models using standard full attention did not show the same periodic variation in performance.

The researchers stated that high average accuracy can coexist with position-specific failures, and that performance should be measured by varying the position of information within compression chunks rather than relying on aggregate scores alone. They said further study is needed on how this phenomenon affects safety and reliability in real-world applications.