LeumaStudio · AI Engineering demo
Semantic cache.
Before a prompt reaches the model, it's embedded and matched against past prompts in a shared Postgres + pgvector store. A close enough match is a cache hit — the stored answer is served instantly, no model call, zero tokens. Because the cache is persistent and shared, you can hit a prompt someone else asked.
Demonstrates pgvector cosine ANN (HNSW) · shared persistent cache · threshold tuning · latency & cost savings
0/0
hits this session
~0
tokens saved
—
shared cache (pgvector)
Hit threshold0.82
Tip: ask the first sample, then the second (a reworded version) — it should hit the cache. Lower the threshold and more paraphrases match; raise it and only near-identical prompts hit.