# Semantic Caching for AI Agents: learn the measurement, distrust the savings figure > A short 1h24m course taught by Redis engineers shows how to cut repeated model calls. The skill worth taking to a client is knowing when the cache gives the wrong answer. Original: https://fdetimes.net/en/books-courses/semantic-caching-ai-agents-course-review/ Two customers type "How do I change my password?" and "I forgot my password, how do I reset it?", and your agent calls the model twice to produce the same answer. DeepLearning.AI's short course *Semantic Caching for AI Agents* teaches you how to skip the second call. For an FDE, though, the most valuable part lies elsewhere: knowing when that skipped call hands the customer a wrong answer. Picture a familiar situation: a client asks whether the model bill can come down and whether the agent can get faster. A cache is the most obvious answer. Without measurement, it is also the quickest way to lose the trust of end users. ## Who teaches it, and why listen selectively? This is an Intermediate-level Short Course, 1h24m long according to the [course page](https://www.deeplearning.ai/courses/semantic-caching-for-ai-agents), made with partner Redis. It is aimed at developers and ML engineers already comfortable with Python, embeddings and basic LLM applications; no prior knowledge of caching is needed. Both instructors are from Redis: Tyler Hutcherson, who manages the Applied AI Engineering team, and Iliya Zhechev, a Senior Research Engineer. There is nothing wrong with a vendor teaching its own tool, provided you can separate the general principles from the product promotion. ## Reusing answers by meaning, not by wording The mechanism is compact. A query is turned into an embedding and compared by cosine similarity with stored queries. According to Redis's blog, if the similarity exceeds a threshold, typically in the 0.85-0.95 range, the system returns the stored answer and does not call the LLM. The fixed cost of this approach is often overlooked. Each vector search adds 5-20ms; in return, a hit saves 1-5 seconds. On a miss, the user still waits the full LLM time plus that overhead, so a cache only pays off when enough queries repeat. ## Precision matters more than hit rate How the course allocates its time shows where the emphasis lies. Of its 7 videos, the three longest are "Measuring Cache Effectiveness" (13 minutes), "Enhancing Cache Effectiveness" (12 minutes) and "Fast AI Agent with Semantic Cache" (16 minutes): 41 minutes spent on measuring and improving, and only then on wiring the cache into an agent. The course teaches three metrics: hit rate, precision and latency. Hit rate tells you how much you save. Precision tells you how many of those hits returned the right answer; the rest returned the wrong one. Imagine 10,000 customer-support queries at a threshold of 0.90 with a 30% hit rate, meaning 3,000 model calls avoided. If 4% of those hits are false positives, 120 people received an answer to a question they did not ask. Now suppose you raise the threshold to 0.95, the hit rate drops to 20% and false positives fall to 1%. You are down to 2,000 savings, 1,000 fewer than before. In exchange, wrong answers fall from 120 to 20. **Key point:** A fast wrong answer is worse than a slow model call. The numbers in this example are hypothetical, but the trade-off is real, and only the client's logs will tell you where the balance lies. Redis recommends starting at a threshold of 0.90-0.95 and tracking false positives. If that rate goes above 3-5%, tuning the threshold is no longer enough; the architecture has to change. The first job at a client is therefore concrete: take a sample of real logs, label pairs of queries as same-meaning or different-meaning, measure both thresholds as above, then choose by precision. Do not pick a threshold by feel during a demo. ## When the data changes, the cache must change too Demos rarely touch the problem of changing data. For fast-moving data such as prices or inventory, Redis suggests a TTL of 5-15 minutes, or clearing the cache as soon as the source data changes. Otherwise the agent will confidently quote a price that is no longer valid. Semantic caching does not replace prompt caching either. Prompt caching is for large, fixed context; semantic caching is for repeated queries with the same intent. Redis calls combining the two layers "double caching", and it is worth considering when a client has both a long system prompt and many repeated questions. ## What to skip, and in what order to learn? The thing to skip is the headline number. Redis writes that teams using semantic caching typically cut LLM costs by "50% or more", depending on how often queries repeat. That is a figure the vendor states itself, not an independent measurement, so promise the client a measurement on their logs, not a percentage. As for order, you can watch the videos in sequence, but rewatch the two on measuring and improving effectiveness before doing the optional 10-minute project. Then read Redis's blog post comparing prompt caching with semantic caching to see the full picture. For your career, a CV line such as "built a semantic cache, chose a 0.92 threshold based on precision measured on a labelled set of query pairs, kept false positives below 3%" carries far more weight than "experienced with Redis". The numbers in that line must be ones you measured yourself. The course teaches you how to make an agent faster. When a client asks "how much will we save?", the answer can only come from their own logs. **Try this week:** - Take 200 real queries from the logs, label which pairs share a meaning, then measure the cache's hit rate and precision at thresholds of 0.90 and 0.95 - List which data in the client's application changes quickly (prices, inventory) and set a TTL or cache-invalidation mechanism for it - Write a CV line for a caching project that states the threshold you chose, the hit rate and the false-positive rate you measured ## Sources - [Semantic Caching for AI Agents (DeepLearning.AI)](https://www.deeplearning.ai/courses/semantic-caching-for-ai-agents) - [What is semantic caching? Guide to faster, smarter LLM apps](https://redis.io/blog/what-is-semantic-caching/) - [Prompt vs semantic caching: Complementary techniques for high-performance AI agents](https://redis.io/blog/prompt-caching-vs-semantic-caching/)