Enterprise chat history is where decisions actually live — and where they go to die. This is a walkthrough of the retrieval system behind Auralis, our sovereign AI platform: hybrid search, reciprocal-rank fusion, exact message provenance, and a system that would rather say "I don't know" than make something up. All of it runs on hardware the company owns.
Why chat history is a knowledge goldmine
Release decisions, delivery risks, client feedback, technical trade-offs — in most companies these are made and recorded in workplace chat, then buried within weeks. The questions people actually ask are simple: What was decided about the July 15 release? What did the team work on in June? Which risks were discussed for this project? Simple questions, but nearly impossible to answer by scrolling.
Why plain semantic search isn't enough
The obvious approach — embed everything and do vector search — fails on chat data in three ways. Embeddings blur exact identifiers, dates and names, which is precisely what enterprise questions hinge on. Chat is fragmented: a decision spans a reply chain and a time window, not a single message. And pure similarity search always returns something, even when the honest answer is "we never discussed that."
So before anything is indexed, Auralis reconstructs conversations: messages are grouped by reply relationships and time windows into coherent documents, then split into chunks that preserve links back to every underlying message.
Hybrid retrieval: full-text plus vectors, fused
Retrieval runs two engines in parallel inside PostgreSQL. Full-text search catches exact terms — names, ticket numbers, dates. Vector search over pgvector embeddings catches meaning — "delivery risks" matching a thread that never uses the word "risk." The two ranked lists are merged with reciprocal-rank fusion, which rewards results both engines agree on without requiring fragile score calibration. Query-time fast paths handle common shapes — a person plus a time period, an exact identifier — without paying full retrieval cost.
Every answer ships with receipts
Each chunk keeps a mapping to the exact source messages it was built from. When the system answers, every material claim must be supported by retrieved evidence, and the answer carries its sources with it. A skeptical reader can walk any sentence back to the original conversation.
Teaching the system to say "I don't know"
The most important behavior in enterprise retrieval is refusing to answer. When retrieval turns up nothing relevant, Auralis returns an explicit insufficient-evidence response instead of an eloquent guess. One fabricated answer costs more trust than a hundred correct ones earn.
The numbers
We froze a 50-case retrieval benchmark — 44 answerable questions and 6 deliberate trick questions with no supporting evidence — as our regression baseline:
- Answer-status accuracy: 100%
- Correct source message retrieved at rank 1: 95.45%; within top 5: 100%
- Exact-identifier recall at rank 1: 90.91%
- Abstention on no-evidence questions: 100% — with a 0% false-grounded rate
- Median retrieval latency: 4.2 seconds on local hardware
And all of it stays private
Embeddings and answer generation both run on local models served by Ollama. No message, question or embedding ever leaves the company's infrastructure — which is what makes it safe to point this system at the most candid data a company has: how its people actually talk to each other.
Key takeaways
- Chat history holds the highest-value, least-accessible knowledge in most companies.
- Reconstruct conversations before indexing; raw messages are the wrong retrieval unit.
- Hybrid full-text + vector retrieval with reciprocal-rank fusion beats either engine alone.
- Provenance and abstention — receipts and honest refusals — are what make enterprise RAG trustworthy.
- A fully private RAG system is practical today on hardware a company can own.
Frequently asked questions
What is reciprocal-rank fusion?
Reciprocal-rank fusion (RRF) merges ranked result lists from different search engines by scoring each result on its rank positions rather than its raw scores. Results that rank well in both keyword and vector search rise to the top, without needing to calibrate incompatible scoring systems.
Why does abstention matter in enterprise RAG?
Because a retrieval system that always answers will eventually present fabrications with confidence. In enterprise settings a single fabricated answer can destroy trust in the whole platform. An explicit insufficient-evidence response keeps the system honest and auditable.
Can RAG really run fully on-premises?
Yes. Auralis runs embeddings, retrieval and answer generation on local open-weight models served by Ollama, with PostgreSQL handling both full-text and vector search. No cloud AI service is involved at any step.
Originally published by Prestanda Consulting. Auralis is Prestanda's sovereign AI platform — explore the platform.