On the Theoretical Limitations of Embedding-Based Retrieval — New Research Paper
Paper calls for future research to develop methods that can resolve this fundamental limitation
Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, and more. These new benchmarks push embeddings to work for any query and any notion of relevance that could be given. While prior works have pointed out theoretical limitations of vector embeddings, there is a common assumption that these difficulties are exclusively due to unrealistic queries, and those that are not can be overcome with better training data and larger models. In this work, we demonstrate that we may encounter these theoretical limitations in realistic settings with extremely simple queries. We connect known results in learning theory, showing that the number of top-k subsets of documents capable of being returned as the result of some query is limited by the dimension of the embedding. We empirically show that this holds true even if we restrict to k=2, and directly optimize on the test set with free parameterized embeddings. We then create a realistic dataset called LIMIT that stress tests models based on these theoretical results, and observe that even state-of-the-art models fail on this dataset despite the simple nature of the task. Our work shows the limits of embedding models under the existing single vector paradigm and calls for future research to develop methods that can resolve this fundamental limitation.
The Unseen Wall: Google DeepMind’s “LIMIT” Reveals a Fundamental Flaw in Vector Embeddings
For years, vector embeddings have been a cornerstone of modern AI, powering everything from semantic search to complex information retrieval systems like Retrieval-Augmented Generation (RAG). The prevailing wisdom has been that with bigger models and more data, we can achieve ever-greater accuracy. However, a groundbreaking research paper from Google DeepMind, released at the end of August 2025, challenges this assumption, revealing a startling and inherent limitation to our current embedding-based methods.
The paper, titled “On the Theoretical Limitations of Embedding-based Retrieval,” introduces a new dataset and methodology called LIMIT (Linguistically Simple, Geometrically Impossible Task). This research demonstrates that even the most advanced large language models (LLMs) can fail at tasks that are linguistically trivial but geometrically complex, pointing to a fundamental bottleneck in how we represent information in vector spaces.
A Deceptively Simple Test
The researchers at Google DeepMind constructed a synthetic and perfectly controlled dataset to test the representational capacity of embedding models. This “LIMIT” dataset consists of:
- 46 simple documents: Each document is a single, straightforward sentence, such as “Ellis Smith likes apples.”
- 1,035 specific queries: These are simple questions like, “Who likes apples?”
- A predefined “ground truth”: The researchers created a matrix that explicitly defines which documents are relevant to which queries.
To make the task realistic, these 46 core documents were hidden within a much larger corpus of 50,000 documents. The challenge for the AI models was to retrieve the correct, predefined documents for each query.
State-of-the-Art Models Stumble
The results were nothing short of shocking. State-of-the-art embedding models from major players like Mistral, Llama, and even Google’s own Gemini, performed abysmally. On a “recall at 100” metric — meaning the correct documents should appear within the top 100 results — some top-tier models scored below 20%. Even more surprisingly, the decades-old keyword-based search algorithm, BM25, outperformed these sophisticated neural models by a massive margin, achieving close to perfect scores.
The researchers then simplified the experiment, removing the 50,000 distractor documents and testing the models on only the 46 core sentences. Even in this simplified scenario, the advanced embedding models struggled to consistently rank the two correct documents in the top two spots for a given query.
The Geometric Bottleneck
The Google DeepMind paper argues that this failure is not a result of linguistic misunderstanding or a search problem. Instead, it’s a fundamental issue rooted in the geometry of vector spaces.
Here’s a simplified explanation of the core problem:
- Embeddings as Points in Space: Vector embeddings work by representing words, sentences, and documents as points in a high-dimensional space. Proximity between these points is meant to indicate semantic similarity.
- Queries as Hyperplanes: A query can be thought of as creating a “decision boundary” or a hyperplane in this space. All the relevant documents are supposed to fall on one side of this plane.
- A Web of Constraints: The LIMIT dataset was specifically designed to create a dense and conflicting web of these hyperplanes. For a model to succeed, it needs to place all the document vectors and query vectors in a way that satisfies all these constraints simultaneously.
- An Impossible Task: The paper proves, both theoretically and empirically, that with a fixed number of embedding dimensions, it becomes mathematically impossible to satisfy the complex web of relationships present in the LIMIT dataset. The more interconnected the information, the more likely the vector space will “break,” making it impossible to retrieve the correct information.
This research connects the limitations of embedding-based retrieval to established concepts in communication complexity theory and the sign-rank of a relevance matrix, providing a firm theoretical foundation for the observed failures.
Rethinking Our Approach: Multi-Vector Representations
This research doesn’t spell the end for vector embeddings, but it does highlight the need for more advanced approaches. The paper points to multi-vector models like ColBERT as a promising direction. Instead of representing a document with a single vector, these models use multiple vectors, allowing for a more nuanced and granular representation of information. This added dimensionality can help to overcome the geometric constraints that single-vector models face.
Key Takeaways for the AI Community
The “LIMIT” paper from Google DeepMind offers several crucial insights:
- Single-vector embeddings are not a universal solution: There are inherent limitations to what can be represented in a fixed-dimensional vector space.
- Scaling up models won’t solve this problem: The issue is not with the size of the model, but with the underlying architecture of single-vector embeddings.
- Beware of overly simplistic benchmarks: The impressive performance of models on existing benchmarks may not reflect their ability to handle geometrically complex tasks.
- The future may be multi-vector: Architectures like ColBERT that use multiple vectors to represent documents offer a potential path forward for more robust and accurate information retrieval.
This research is a critical reminder that as we push AI to handle increasingly complex and interconnected information, we must also be prepared to fundamentally rethink the tools and architectures we use. The “unseen wall” that LIMIT has revealed will undoubtedly spur a new wave of innovation in the field of information retrieval.
New RAG Systems Are Continuously Being Developed
Retrieval-augmented systems are typically evaluated in settings where information required to answer the query can be found within a single source or the answer is short-form or factoid-based. However, many real-world applications demand the ability to integrate and summarize information scattered across multiple sources, where no single source is sufficient to respond to the user’s question. In such settings, the retrieval component of a RAG pipeline must recognize a variety of relevance signals, and the generation component must connect and synthesize information across multiple sources. We present a scalable framework for constructing evaluation benchmarks that challenge RAG systems to integrate information across distinct sources and generate long-form responses. Using our framework, we build two new benchmarks on Multi-Source Retrieval and Synthesis: MSRS-Story and MSRS-Meet, representing narrative synthesis and summarization tasks, respectively, that require retrieval from large collections. Our extensive experiments with various RAG pipelines — including sparse and dense retrievers combined with frontier LLMs — reveal that generation quality is highly dependent on retrieval effectiveness, which varies greatly by task. While multi-source synthesis proves challenging even in an oracle retrieval setting, we find that reasoning models significantly outperform standard LLMs at this distinct step.
