Engineering notes · chatzora

What Chatzora does when a knowledge search finds no answer

How Chatzora uses retrieval thresholds and unanswered conversations, and why a retrieval failure needs separate handling.

By Muhammad Usama

A search result is not automatically a useful source

Chatzora stores document passages with embeddings in PostgreSQL. For an incoming question, it searches the current agent’s knowledge with pgvector. The search returns candidate passages, but the closest result can still be a poor match.

The chat route filters those candidates using a configurable similarity threshold. That is a retrieval score, not a percentage probability that an answer is correct. It needs to be evaluated against real questions and the actual source material.

Make the unanswered path visible

When retrieval succeeds but no passage passes the threshold, and the assistant has no active tools, Chatzora can return the configured fallback message. It records the answer as a fallback and marks the conversation as unanswered.

That creates a review path for the owner. A missing answer might mean the business needs to add a document, clarify an existing one, or change how the assistant is configured. It should be possible to find that question again.

An outage is a different condition

A failed embedding request and a successful search with no match are different events. The current implementation has a separate direct-prompt fallback when embeddings or vector search fail. That answer should not be described as grounded in retrieved documents.

Tools add another branch: a question may be answered by a connected tool even when no document matches. The behavior needs to be tested for the agent’s settings and integrations. A retrieval threshold is one control in the system, not a guarantee that every response is accurate.

Have something in mind?

Tell me what you’re building, where things stand, and what you need help with.

Let’s talk