Advanced Retrieval for AI with Chroma: one hour, three techniques for the retrieval step of RAG

A RAG demo usually works well on questions written by the person who built it. The real test comes when users ask in their own words. DeepLearning.AI’s course Advanced Retrieval for AI with Chroma runs 1 hour 2 minutes and covers a single step: retrieving documents.

The instructor is Anton Troynikov, co-founder of Chroma. The course has 7 video lessons, 5 code examples and 1 graded quiz. The course page says it was last updated on 7 October 2025.

For an FDE, that narrow scope is a strength. When a client complains that the chatbot “gives wrong answers”, one hypothesis to check early is that the model is faithfully summarising irrelevant documents. This course teaches three ways to deal with that problem.

In what order should you learn them?

The three techniques worth keeping are query expansion, cross-encoder reranking and embedding adapters. The lessons are short. The reranking lesson is about 6 minutes and the adapter lesson about 8, so you can finish the course in one evening.

Go from cheap to expensive. Query expansion and reranking only add a step to an existing pipeline. An adapter needs labelled data to train on. At a client site, the first job is still to find where the pipeline is failing. Only then should you pick a technique.

One question, many phrasings

Query expansion adds related concepts and keywords to the user’s question. deepset’s Haystack documentation describes the same idea through its QueryExpander: from one query, generate several variants with similar meaning to improve recall.

Suppose you deploy an internal documentation assistant for an insurance company. An employee asks “can I get a refund if I cancel my policy?”, but the documents say “early termination of the contract”.

Generating a few variants such as “early termination” and “cancellation fee” can make it more likely that the search hits the right passage.

Educative explains why this works with probability. The more ways you phrase a question, the better the chance that at least one phrasing sits close to the relevant passage in embedding space.

That is an argument, not a measurement. You still have to measure it on the client’s data.

Retrieve broadly, then score carefully

The cost of query expansion is that you retrieve more. Say you have 1 original question plus 4 generated variants, and each retrieves 10 results. That gives you 50 documents, which might come down to about 30 after deduplication. You cannot fit all 30 into the context.

That is why the next lesson covers reranking: a cross-encoder reorders the retrieved results and picks those most relevant to the query. Because this step runs only on candidates you already have, it suits a few dozen documents rather than a whole collection. From 30 candidates, you keep 5 for the LLM.

Reshaping the embedding space for your application

The embedding adapter lesson has the course’s most useful idea. You add a small layer that reshapes the embeddings so the features that matter for your application carry more weight. The course page says only that this “can improve” retrieval. It makes no firm promise.

Chroma’s research report on embedding adapters gives a more specific number, but it needs careful reading. A linear adapter applied only to the query improved retrieval accuracy by “up to” 70% compared with the pre-trained embedding model on its own.

That result comes from the report’s own experimental setup. It was measured with NDCG on the CQADupstackEnglishRetrieval dataset, against all-MiniLM-L6-v2 as the baseline, and the adapter needs at least 1,500 labelled query–document pairs. Do not promise a client a 70% gain.

Even so, the detail that the adapter is “applied only to the query” matters in practice for FDEs. You do not have to re-embed millions of client documents, and you do not have to change models. If you log questions from day one, along with the documents users confirm as correct, those logs become your labelled pairs for trying an adapter later.

Who should take it, and what goes on your CV?

The course suits engineers who already have a working RAG system and keep asking “why is it fetching the wrong documents?” If you have never built RAG, build a simple version first. The techniques only make sense once you have run into the failures yourself.

To prove you fixed the right thing, you need a simple measurement. Hit rate asks one question: for each query, did the system retrieve at least one correct passage in the top k? Score 1 if yes and 0 if no. The average across the whole set is the overall hit rate.

Take 20 questions and k=5. Before reranking, 11 questions have the correct document in the top 5, so hit rate@5 is 11 ÷ 20 = 55%. After adding reranking, if 16 questions do, hit rate@5 is 80%. These numbers are only an illustration. Yours must come from your own labelled question set.

When reading job descriptions for FDE or applied AI roles, look for phrases such as “retrieval quality”, “RAG in production” and “evaluation”. On a CV, a line such as “added cross-encoder reranking, raising hit rate@5 on 20 real questions from 55% to 80%” counts for far more than “knows Chroma”.

Take the hour-long course, then spend your first session at the client labelling 20 real questions. Only trust a technique when you have before-and-after numbers.