Hands-on: building a RAG system that answers only from the policy currently in force

Imagine your client amends its refund policy: the window shrinks from 30 days to 14, effective 1 July 2026. Both versions sit in the vector store. A member of staff asks the chatbot “how long do customers have to get a refund?”, and the chatbot confidently answers 30 days, complete with a citation.

The language model is not at fault. Particula Tech points out that an amendment and the version it replaces are near-duplicates in embedding space, so the expired version can easily rank first. Semantic similarity has no notion of time.

On enterprise RAG projects this is a failure an FDE is almost certain to meet, because every policy, contract and procedure eventually gets a new version. This guide works from the payload, through the filter, to the test suite, so the system answers only from the version in force at the time being asked about.

What you will build, and what you need

The end result is a pipeline in three parts: ingestion that attaches a validity interval to each chunk, retrieval that filters on an as-of date, and a date-based test suite. You need a vector store (the main example uses Qdrant, with sections on moving to Pinecone and pgvector), a document with at least two versions, and Python for the ingestion code.

Why go to this trouble? On the VersionQA benchmark, the authors of VersionRAG, a published method distinct from the metadata-filtering pipeline described here, measured 90% accuracy, against 58% for naive RAG and 64% for GraphRAG. That gap shows how much RAG that ignores versions gets wrong on documents that change over time.

Step 1: effective dates must live on the chunk

The most common design mistake is to record effective dates only in a document registry. Particula Tech stresses that effective dates must be attached to every chunk, because retrieval filters chunks, not documents. Each chunk therefore needs to carry the full metadata of the version that produced it.

# Simplified: payload for one chunk
SENTINEL = "9999-12-31T23:59:59Z"

def make_payload(doc_id, version, text, effective_from, effective_to=None):
    return {
        "doc_id": doc_id,
        "version": version,
        "status": "effective",
        "effective_from": effective_from,           # RFC 3339
        "effective_to": effective_to or SENTINEL,   # never null
        "text": text,
    }

The line that matters is effective_to or SENTINEL. The current version has no expiry date yet, and the instinct is to leave the field empty. Particula Tech recommends writing an explicit far-future sentinel instead, because in most vector stores a missing key does not match a range condition. In other words, the newest chunk is the one that gets filtered out.

Check: count the chunks whose effective_to is empty or missing. The correct number is 0.

Step 2: when a new version arrives, close the old one

On 1 July 2026, v2 is ingested with effective_from = "2026-07-01T00:00:00Z". At the same time, every v1 chunk must have its effective_to updated to exactly that timestamp and its status switched to superseded. From then on v1 has a closed interval, while v2 runs to the sentinel.

Correct metadata is not yet grounds for comfort. Particula Tech warns that forgetting the filter on a single shared collection is enough for the system to return a superseded clause that still looks like a correct answer.

Their preferred fix is to move old versions into a separate archive collection. The main collection then holds only versions in force, while the archive keeps expired versions for use when questions concern the past.

Check: query the main collection with no filter at all. If any chunk still carries status = superseded, the move to the archive collection is broken.

Step 3: every query carries an as-of date

According to Particula Tech, every query should carry an as-of timestamp, defaulting to now, and keep only chunks whose validity interval contains it. A question such as “back in May, how long did customers have for a refund?” then differs only in the as-of date passed in.

Qdrant supports range conditions on datetime payloads in RFC 3339 format. A must clause is true only when every condition inside it holds, which makes it a natural fit for combining status with the validity interval. The filter below is simplified:

{
  "must": [
    { "key": "status", "match": { "value": "effective" } },
    { "key": "effective_from", "range": { "lte": "2026-10-26T00:00:00Z" } },
    { "key": "effective_to",   "range": { "gt":  "2026-10-26T00:00:00Z" } }
  ]
}

Note the pairing of lte and gt. The interval is closed at the start and open at the end, so at exactly 00:00 on 1 July only v2 matches; the two versions can never both get through.

For questions about the past, drop the status condition, keep the two interval conditions with the new as-of date, and search both the main and the archive collections. Search only the archive and a question about a date on which the current version was already in force returns nothing, because that version still lives in the main collection.

Check: set as-of to 15 June 2026 and then 15 July 2026, and ask the same question across both collections. The first must cite v1 (30 days), the second v2 (14 days).

What breaks when you switch to Pinecone or pgvector?

Each vector store has its own trap, and you need to know it before the client demo.

Pinecone offers $gte/$lte operators on metadata, but they work only on numeric values. Effective dates must therefore be stored as numbers, such as epoch seconds, and the sentinel must be a number too. The example below is simplified:

{
  "status": { "$eq": "effective" },
  "effective_from": { "$lte": 1792972800 },
  "effective_to":   { "$gt":  1792972800 }
}

Here 1792972800 is the epoch for 2026-10-26T00:00:00Z, and the matching sentinel is 253402300799 (9999-12-31T23:59:59Z). Store dates as strings by mistake and the range filter will not behave as you expect.

pgvector’s trap is subtler. With an approximate index, the WHERE clause is applied only after the index scan finishes. The result: you ask for the top 10, the index returns the 10 nearest chunks, most of them old versions that get filtered out, and you end up with fewer than 10.

The pgvector README suggests iterative scans, or a partial index when filtering on a small number of distinct values, which is exactly the case with status = 'effective'. The statement below is illustrative only; check it against the pgvector documentation for your version:

CREATE INDEX ON policy_chunks
USING hnsw (embedding vector_cosine_ops)
WHERE status = 'effective';

Check: request the top k and count the rows returned. If you regularly get fewer than k, enable iterative scans or use a partial index.

Step 4: a date-based test suite is your proof for the client

The filter solves only half the problem. In the same VersionRAG study, on questions about detecting implicit changes between versions, the baselines almost completely failed, at 0-10% accuracy, while VersionRAG reached 60%.

The lesson is that questions such as “what changed in the refund policy?” remain hard even with clean metadata, so put them in the test suite rather than making promises to the client.

A minimal test suite consists of triples: question, as-of date, expected version. Include questions before the amendment date, after it, exactly on the changeover, and a few about what changed.

Every time you re-ingest or change the embedding model, run the suite in CI and check the cited version, not just the answer text.

Which mistakes most often break the pipeline?

Before taking the system to a client, go back over the five mistakes covered in the steps above. Each has a quick check.

Mistake	Symptom	Fix
Effective dates only in the document registry	The retrieval filter has nothing to filter on	Write effective_from, effective_to and status on every chunk
effective_to left empty for the current version	The newest version vanishes from results	Backfill with a far-future sentinel
Old versions share a collection with new ones	One forgotten filter returns an old clause	Move old versions to an archive collection
Dates stored as strings in Pinecone	The range filter does not filter as expected	Store epoch values; the sentinel is a number too
WHERE applied after the approximate index in pgvector	Ask for top k, receive fewer than k	Enable iterative scans or use a partial index

If you have time to fix only one thing, start with the second row: counting chunks without effective_to takes minutes and blocks the hardest-to-spot failure.

What does this skill look like on site with a client?

The first job at the client is not code. Ask the people who own the documents: how versions are marked, where effective dates are stored, and who decides that a version has been superseded.

Often the answer lives in file names, in a spreadsheet, or only in the head of one person in the legal department. Step 1 fails if you cannot get it.

If you are applying for applied AI FDE roles, look for job descriptions that mention RAG over internal, compliance or legal documents.

On your CV, instead of writing “built RAG”, describe the exact problem you solved: stopping citations of expired versions, proven with a test suite that uses as-of dates. Recruiters will see that you understand real client data.

Every demo works when the store holds a single version. Your system is truly tested on the day the client issues its first amendment.