Use embeddings to semantically search past conversations. Retrieves relevant context based on meaning, not recency.
1. Each turn is embedded and stored in a vector database
2. When new input arrives, embed it and search for similar past turns
3. Retrieve top-K most relevant past turns
4. Inject them into the context window
User: "What did we decide about the pricing model?"
→ Embed query → Search vectors → Retrieve relevant past discussion
→ Inject: "Last session you decided on tiered pricing: $10/$50/$200"
import numpy as np
class VectorMemory:
def __init__(self, embed_fn, top_k=3):
self.embed_fn = embed_fn # Callable: str → np.array
self.top_k = top_k
self.entries = [] # List of {text, embedding, metadata}
def add(self, text, metadata=None):
embedding = self.embed_fn(text)
self.entries.append({
"text": text,
"embedding": embedding,
"metadata": metadata or {},
})
def search(self, query):
q_emb = self.embed_fn(query)
scores = []
for e in self.entries:
score = np.dot(q_emb, e["embedding"]) / (
np.linalg.norm(q_emb) * np.linalg.norm(e["embedding"])
)
scores.append(score)
indices = np.argsort(scores)[-self.top_k:][::-1]
return [self.entries[i] for i in indices]
def get_context(self, query):
results = self.search(query)
if not results:
return ""
parts = ["[Relevant past context]"]
for r in results:
parts.append(f"- {r['text']}")
return "\n".join(parts)| Store | Best For |
|---|---|
| ChromaDB | Local, lightweight, Python-native |
| Pinecone | Production, high-scale, managed |
| pgvector | PostgreSQL-native, relational + vector |
| FAISS | Fastest search, in-memory, large scale |
## Retrieved Relevant History
{vector search results}
Use the above if relevant to the current question.
If nothing seems relevant, proceed without it.Pros: Semantic understanding, scales to millions of turns, cross-session recall.
Cons: Requires embeddings (cost/latency), cold start problem, embedding quality matters.