Finding the right issuer, period and passage
An analyst asks: “What was ExampleCo’s operating margin in FY2024?” A dense retriever returns fluent passages about profitability. A sparse retriever finds many occurrences of “operating margin.” Neither result is useful if the passages belong to another issuer or a different period.
Retrieval has to combine semantic relevance with hard constraints. The client’s design used Qdrant with dense and sparse representations, followed by reranking. The interesting part is the contract between those stages: what gets filtered, what gets ranked and what evidence is still missing.
Parse the question into intent and constraints
A query has at least two components:
{
"semantic_query": "operating margin and supporting reported figures",
"constraints": {
"issuer_id": "example-co",
"period_end": "2024-12-31",
"document_types": ["10-K", "annual_report"]
},
"answer_shape": "reported_value_or_calculation_with_sources"
}
The constraints should come from validated entity resolution and user context. “Apple” as an issuer is not just a word to embed. A fiscal quarter should be resolved against the company’s reporting calendar, not guessed from a generic calendar quarter.
Keep uncertainty visible. If the user asks for “last quarter” and the intended fiscal period is ambiguous, ask or show the interpretation. A confident answer to an incorrectly resolved question is still a failure.
Apply access and scope before ranking
A retrieval request should carry tenant, entitlement, issuer and time constraints into every candidate channel. Filtering only after retrieval creates two problems: unauthorised evidence can reach downstream systems, and the candidate set may be exhausted by documents that will later be discarded.
The client’s metadata path included sector, industry, sub-industry and document type. A more complete financial research contract adds issuer identity, period, availability time and source access. These are extensions to the retrieval schema, not information that should be inferred from vector similarity.
Create payload indexes for the fields used in filters and benchmark realistic combinations. A global unfiltered search can look fast while a restrictive issuer-and-date query behaves quite differently.
Fuse ranks before making scores look comparable
Dense cosine scores and sparse retrieval scores do not have the same meaning or scale. A weighted sum can be dominated by whichever channel produces larger numbers.
Reciprocal rank fusion offers a simple baseline: each candidate receives a contribution based on its position in each result list. The following implementation uses one-based ranks and an explicit smoothing constant of 60:
from collections import defaultdict
def reciprocal_rank_fusion(rankings: list[list[str]], k: int = 60):
if k <= 0:
raise ValueError("k must be positive")
scores = defaultdict(float)
for ranking in rankings:
seen = set()
for rank, passage_id in enumerate(ranking, start=1):
if passage_id in seen:
raise ValueError("A ranking contains duplicate passage IDs")
seen.add(passage_id)
scores[passage_id] += 1.0 / (k + rank)
return sorted(scores.items(), key=lambda item: (-item[1], item[0]))
This constant is a configuration choice for the worked example. It is not Qdrant’s default. Current Qdrant documentation describes its own rank convention and default smoothing, and provides a Query API with prefetch and fusion for dense+sparse retrieval. Record the server version and fusion settings when comparing experiments. Qdrant hybrid-query documentation
For two lists A, B, C and B, D, A, candidate B receives 1/62 + 1/61, while A receives 1/61 + 1/63. B therefore ranks slightly higher. A result appearing strongly in both channels benefits without requiring their raw scores to be calibrated.
A hybrid request has several budgets
A practical experiment might retrieve 100 dense candidates and 100 sparse candidates, fuse them, retain 50 for reranking and pass a smaller evidence set to synthesis. These are starting budgets, not historical performance claims.
from qdrant_client import models
result = client.query_points(
collection_name="financial_passages_v2",
prefetch=[
models.Prefetch(query=dense_query, using="dense",
filter=scope_filter, limit=100),
models.Prefetch(query=sparse_query, using="sparse",
filter=scope_filter, limit=100),
],
query=models.FusionQuery(fusion=models.Fusion.RRF),
query_filter=scope_filter,
limit=50,
with_payload=True,
)
The variables represent validated query vectors and a server-built access/scope filter. Keep fusion at the intended stage of the query and verify its behaviour across shards. The conceptual pattern is supported by Qdrant’s Query API; pin the client and server versions in the runnable service.
Each budget limits what later stages can achieve. Reranking cannot recover a paragraph that neither candidate generator returned. Increasing reranker quality without measuring candidate recall can spend more money on the wrong bottleneck.
Measure recall at the evidence level
For a factual question, one passage may contain the answer. For a margin-change question, the required evidence can include revenue, operating income and commentary from two periods.
Define an evidence set for each evaluation question. Then inspect how many required evidence units appear within the retrieval budget. Also measure whether the candidate set contains all the evidence needed for the task.
A recall of two out of three passages can sound reasonable while the missing passage makes the question unanswerable. Task-complete recall exposes that distinction.
Use an ablation table with actual measured results when available:
| Retrieval configuration | Main question it tests |
|---|---|
| Dense only | Can semantic matching find the evidence? |
| Sparse only | Are exact and expanded terms sufficient? |
| Dense + sparse fusion | Do the channels recover complementary evidence? |
| Fusion + validated metadata | How much error came from wrong scope? |
| Fusion + parent expansion | Was the retrieved child missing necessary context? |
Do not fill this table with invented improvement percentages. The useful next step is a reproducible comparison on the same corpus snapshot and questions.
More candidates can create more noise
Increasing candidate depth helps until irrelevant near-duplicates dominate the reranking cost. Annual reports contain repeated boilerplate. Transcript excerpts may repeat a question in several overlapping chunks. One long filing can occupy most of the top fifty results.
Group near-duplicate passages by source occurrence and limit repeated chunks from the same local region. Preserve independent evidence from other periods when the task requires a comparison. Diversification should follow the question: a single-fact lookup and a multi-company comparison should not use the same per-document cap blindly.
Retrieval scores are not answer confidence
A score threshold such as 0.7 has no universal interpretation across models, indexes and fusion methods. It is especially unsafe to reuse a dense-score threshold after switching to rank fusion.
A release decision should rely on evidence adequacy: correct scope, sufficient source coverage, support for the requested claim and a resolvable citation. Score calibration can help choose candidates, but it cannot substitute for those checks.
The retriever’s job is to produce a well-scoped candidate set. The next article looks at turning that set into a compact body of evidence the answer generator can actually use.