bundle-course-sap-ariba-sourcing-procurement-contract-management-administration By Uplatz<\/a><\/h3>\nII. Foundations: Prompt Embeddings and the Caching Imperative<\/b><\/h2>\nA. What are Prompt Embeddings?<\/b><\/h3>\n
Before implementing a cache, it is critical to understand <\/span>what<\/span><\/i> is being cached. When a user sends a prompt (e.g., “What is the capital of France?”) to an LLM, the model does not understand text directly. The input is first processed by an <\/span>embedding model<\/span><\/i>.<\/span>1<\/span><\/p>\nThis model converts the non-numeric text into a <\/span>numeric representation<\/span><\/i> called an embedding, which is a high-dimensional vector (a long list of numbers).<\/span>2<\/span> This vector captures the <\/span>semantic meaning<\/span><\/i> and <\/span>context<\/span><\/i> of the original text, allowing the LLM to mathematically determine relationships between concepts.<\/span>4<\/span> For example, the vectors for “king” and “queen” will be mathematically close to each other. These vectors are the fundamental unit that LLMs use to process, compare, and understand human language.<\/span>1<\/span><\/p>\n <\/p>\n
B. The Business Imperative: Why Cache LLM Calls?<\/b><\/h3>\n
<\/p>\n
LLM inference is computationally intensive and expensive. Every API call to a model like GPT-4o incurs a direct monetary cost (paid per token) and a time cost (latency).<\/span>6<\/span> Caching is the primary strategy to address both.<\/span><\/p>\n\n- Cost Reduction:<\/b> Many applications, particularly enterprise chatbots, receive a high volume of repetitive or semantically similar queries.<\/span>6<\/span> For example, “How do I reset my password?” and “I forgot my password, what do I do?” are different prompts with the same intent.<\/span>8<\/span> By caching the response to the first query and serving it for the second, the redundant LLM API call is eliminated. This can reduce input token costs by up to 90%.<\/span>9<\/span><\/li>\n
- Latency Reduction:<\/b> Caching moves the response from a slow, multi-second computation (the LLM) to a fast, millisecond retrieval (the cache).<\/span>11<\/span> For real-time applications like chatbots, this is critical for user experience. Benchmarks show that prompt caching can reduce latency by up to 80-85%.<\/span>6<\/span><\/li>\n
- Scalability:<\/b> Caching reduces the computational load on the core LLM, allowing the system to handle a higher volume of users without scaling up expensive GPU resources.<\/span>6<\/span><\/li>\n<\/ul>\n
<\/p>\n
C. The Platform Caching Model (e.g., OpenAI)<\/b><\/h3>\n
<\/p>\n
It is important to note that some LLM providers, like OpenAI, implement their own form of <\/span>prompt caching<\/span><\/i> automatically.<\/span>9<\/span> However, this caching is typically limited to <\/span>exact prefix matching<\/span><\/i>. The system routes requests with an identical prefix (e.g., the first 256 tokens) to a server that recently processed the same prompt. This is effective for static system prompts or instructions placed at the beginning of a prompt.<\/span>9<\/span><\/p>\nThis built-in caching is beneficial but insufficient. It cannot handle queries that are semantically identical but textually different (e.g., “password reset” vs. “forgot password”). To achieve this, a more sophisticated, application-layer caching strategy is required.<\/span><\/p>\n <\/p>\n
III. Redis as the Unified Platform for AI Caching<\/b><\/h2>\n
<\/p>\n
While many databases can store vectors, Redis presents a unique strategic advantage by being a <\/span>multi-model<\/span><\/i> database designed for real-time performance.<\/span><\/p>\n <\/p>\n
A. The Multi-Model Advantage<\/b><\/h3>\n
<\/p>\n
A “pure” vector database (e.g., Pinecone, Milvus) is specialized for one task: storing and searching vectors.<\/span>13<\/span> A production AI application, however, requires more. A typical Retrieval-Augmented Generation (RAG) application needs:<\/span><\/p>\n\n- A <\/span>Vector Database<\/b> for RAG document retrieval.<\/span><\/li>\n
- A <\/span>Semantic Cache<\/b> for LLM responses (also a vector database).<\/span><\/li>\n
- A <\/span>Key-Value Cache<\/b> for exact-match caching.<\/span><\/li>\n
- A <\/span>Session Store<\/b> for managing conversation history.<\/span><\/li>\n
- A <\/span>Message Broker<\/b> for asynchronous tasks or invalidation.<\/span><\/li>\n<\/ol>\n
Using specialized tools for each function results in a complex, fragmented architecture with multiple data silos. Redis, by contrast, can serve all these functions within a single, high-performance system.<\/span>15<\/span> An AI architect can use Redis to store RAG documents in RedisJSON, manage vectors with Redis Search, handle exact-match caching with standard keys, store conversation history in Redis Hashes, and manage cache invalidation with Redis Pub\/Sub.<\/span>15<\/span> This consolidation drastically simplifies the architecture, reduces data-transfer latency, and lowers the total cost of ownership.<\/span>16<\/span><\/p>\n <\/p>\n
B. Redis vs. Specialized Vector Databases<\/b><\/h3>\n
<\/p>\n
When evaluating Redis specifically for its vector search capabilities, it compares favorably against specialized, “pure” vector databases.<\/span><\/p>\n\n- Redis vs. Pinecone:<\/b> Pinecone is a proprietary, purpose-built vector database.<\/span>13<\/span> While effective for basic vector storage, Redis Enterprise is presented as a more comprehensive real-time data platform, offering capabilities beyond vector search, such as high availability, multi-tenancy, and durability, which specialized vendors are now building to catch up.<\/span>18<\/span> Benchmarking against Pinecone is legally restricted by their “DeWitt clause,” which forbids publishing performance evaluations.<\/span>20<\/span><\/li>\n
- Redis vs. Milvus:<\/b> Milvus is an open-source, purpose-built vector database designed for high-performance vector search, particularly at billion-scale.<\/span>14<\/span> However, Redis is a mature, in-memory key-value store that has added vector search as a module.<\/span>14<\/span> Some analyses suggest Milvus is finely tuned for high-demand vector workloads, while Redis offers broader versatility.<\/span>21<\/span> Conversely, benchmarks published by Redis claim superior querying throughput (up to 3.3x higher QPS than Milvus) and lower indexing times (up to 2.8x lower).<\/span>20<\/span><\/li>\n<\/ul>\n
For many AI applications, the combination of Redis’s “good enough” or superior vector performance with its dominant, industry-standard in-memory caching and multi-model features makes it the most practical and architecturally simple choice.<\/span>22<\/span><\/p>\n <\/p>\n
IV. Part 1: Implementing Exact-Match Caching in Redis<\/b><\/h2>\n
<\/p>\n
This is the most basic caching strategy, which targets identical, repeated prompts.<\/span><\/p>\n <\/p>\n
A. Concept and Pattern<\/b><\/h3>\n
<\/p>\n
The pattern, often called “query caching,” is a simple implementation of the cache-aside pattern.<\/span>23<\/span><\/p>\n\n- Prompt Identification:<\/b> The application receives a user prompt. A unique, deterministic key is generated from the exact prompt string, typically using a fast hashing algorithm like MD5 or SHA-256.<\/span>23<\/span><\/li>\n
- Cache Lookup:<\/b> The application checks Redis for the existence of this key.<\/span>23<\/span><\/li>\n
- Cache Hit:<\/b> If the key exists, the stored response is retrieved from Redis and returned to the user. The LLM is never called.<\/span>23<\/span><\/li>\n
- Cache Miss:<\/b> If the key does not exist, the application forwards the prompt to the LLM. The LLM’s response is then stored in Redis using the hash key\u2014critically, with a Time-to-Live (TTL) set\u2014and returned to the user.<\/span>23<\/span><\/li>\n<\/ol>\n
This pattern is incredibly fast but completely rigid; even a single-character change in the prompt (e.g., an extra space) will result in a cache miss.<\/span>8<\/span><\/p>\n <\/p>\n
B. Implementation with redis-py<\/b><\/h3>\n
<\/p>\n
Using the standard redis-py library, the implementation is straightforward.<\/span><\/p>\n <\/p>\n
Python<\/span><\/p>\n <\/p>\n
# [26, 59, 60]<\/span>
\n<\/span>import<\/span> redis<\/span>
\n<\/span>import<\/span> hashlib<\/span>
\n<\/span>import<\/span> os<\/span>
\n<\/span>
\n<\/span># Assumes llm.invoke() is defined elsewhere<\/span>
\n<\/span># from langchain_openai import OpenAI<\/span>
\n<\/span># llm = OpenAI()<\/span>
\n<\/span>
\n<\/span>try<\/span>:<\/span>
\n<\/span>\u00a0 \u00a0 r = redis.Redis.from_url(os.getenv(<\/span>“REDIS_URL”<\/span>, <\/span>“redis:\/\/localhost:6379”<\/span>))<\/span>
\n<\/span>\u00a0 \u00a0 r.ping()<\/span>
\n<\/span>\u00a0 \u00a0 print(<\/span>“Connected to Redis”<\/span>)<\/span>
\n<\/span>except<\/span> redis.exceptions.ConnectionError <\/span>as<\/span> e:<\/span>
\n<\/span>\u00a0 \u00a0 print(<\/span>f”Redis connection error: <\/span>{e}<\/span>“<\/span>)<\/span>
\n<\/span>\u00a0 \u00a0 <\/span># Handle connection failure<\/span>
\n<\/span>
\n<\/span>def<\/span> get_response_exact_match<\/span>(prompt: <\/span>str<\/span>):<\/span>
\n<\/span>\u00a0 \u00a0 <\/span># 1. Create a deterministic key<\/span>
\n<\/span>\u00a0 \u00a0 cache_key = <\/span>f”llm:exact:<\/span>{hashlib.md5(prompt.encode()).hexdigest()}<\/span>“<\/span>
\n<\/span>
\n<\/span>\u00a0 \u00a0 <\/span>try<\/span>:<\/span>
\n<\/span>\u00a0 \u00a0 \u00a0 \u00a0 <\/span># 2. Cache Lookup<\/span>
\n<\/span>\u00a0 \u00a0 \u00a0 \u00a0 cached_response = r.get(cache_key)<\/span>
\n<\/span>\u00a0 \u00a0 \u00a0 \u00a0 <\/span>if<\/span> cached_response:<\/span>
\n<\/span>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 print(<\/span>“Cache Hit! (Exact)”<\/span>)<\/span>
\n<\/span>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <\/span>
![\"\"](\"https:\/\/uplatz.com\/blog\/wp-content\/uploads\/2025\/11\/A-Solutions-Architects-Guide-to-Caching-LLM-Prompt-Embeddings-with-Redis-1024x576.jpg\")
<\/p>\n