Search is solved. Embedding models reliably find what looks similar to a text, faster and more thoroughly than any human could. Similarity was simply never the question. A task asks what it needs in order to be solved correctly, and that is a judgement rather than a distance in vector space.
Similar is not important
Ask about a notice period and you get ten paragraphs that sound like termination, but not necessarily the one that governs this contract. All ten are similar and one is relevant. A system that cannot tell the difference fills the window with the plausible and crowds out the decisive.
Similarity is a hint, relevance is a judgement.
What an embedding can and cannot do
An embedding maps text onto a vector, and the proximity of two vectors stands for semantic similarity. That is a real achievement, but it is exactly one property. An embedding does not know which of two similar documents is the more recent one, which one governs this contract, which came from the responsible office and which was an abandoned draft. Those features are not in the vector space because they are not in the text. They live in metadata, in the process and in the task.
How you cut it matters
Before anything can be found, the text has to be divided. Cut too finely and a passage loses its reference, so a sentence like "this period does not apply to executive contracts" stands there without the contract it belongs to. Cut too coarsely and every hit drags along a lot of surrounding material that costs room in the window. Chunking is therefore not a preprocessing detail but one of the most effective adjustments in the whole system.
What helps in practice
Three things have proven themselves for us. First, combining keyword search with vector search, because numbers, reference codes and proper names are found more reliably by classical search than by an embedding. Second, a second scoring stage that reorders the candidates from the first round rather than trusting a single similarity score. Third, hard filters from metadata, meaning validity, responsibility and recency, applied before anything is ranked.
The price belongs in the same paragraph. Every extra stage costs latency, and money too when it calls a model of its own. It pays off when it substantially reduces the number of snippets you send on. It does not pay off when you send everything anyway, and that is often exactly how it gets built.
The wrong metric
Retrieval systems are usually measured by recall: how many of the matching documents appear among the first k hits? That number rewards generosity, because returning more finds more. For a system with a finite window it is therefore misleading. The more useful question is how much correctness you get per token, and it penalises every snippet that contributes nothing.
Anyone who treats retrieval as a by-product of their vector database is measuring the wrong quantity, and pays the difference on every single run.