A convincing demo proves very little

Most RAG demonstrations use clean documents and questions the builder already knows the system can answer. Production users behave differently. They use shorthand, combine several questions, ask about outdated policies, and expect the assistant to understand context that exists only inside the organisation.

The useful question is not simply, 'Does the chatbot work?' It is, 'Under what conditions can people rely on it, and what happens outside those conditions?' That changes evaluation from a presentation into an engineering and operating discipline.

Start with the decision and its risk

Define what someone will do with the answer. An assistant that helps locate an internal template can tolerate a different failure profile from one used to interpret workplace policy or prepare a consequential customer response. The greater the consequence, the stronger the evidence, review, and escalation requirements should be.

Before choosing metrics, document the operating boundary.

  • Who will use the system
  • Which source collections it may access
  • What decisions its answers may influence
  • Which mistakes are merely inconvenient
  • Which mistakes require human review
  • What the assistant must never reveal or claim
  • Which release thresholds match the risk of the workflow

Build an evaluation set from real questions

A useful evaluation set resembles production traffic rather than a product demo. Start with questions collected from the people who currently answer them. Record the expected source and, when practical, the essential facts a good answer should contain.

Keep this set after launch. It becomes a regression suite whenever documents, retrieval settings, prompts, or models change.

  • Routine questions with one clear source
  • Questions requiring information from multiple passages
  • Ambiguous questions that need clarification
  • Questions whose answers changed between document versions
  • Questions involving conflicting sources
  • Questions that the approved collection cannot answer
  • Questions that attempt to access another user's restricted material
  • Misspellings, shorthand, and natural follow-up questions

Test retrieval before judging the prose

Inspect the passages returned before the model writes an answer. Check whether the expected source was retrieved, whether irrelevant material outranked it, whether the passage includes enough surrounding context, whether the current version beat obsolete material, and whether access restrictions were applied before retrieval.

If the evidence is weak, changing the answer prompt may only make the system sound more confident. Fix source ownership, document parsing, chunking, metadata, filtering, or ranking first.

Amazon Bedrock's RAG evaluation guidance makes the same useful separation: retrieve-only evaluation measures context relevance and coverage, while retrieve-and-generate evaluation assesses the final response. That separation helps a team identify which part of the pipeline is actually failing.

Evaluate the answer and its evidence together

Once retrieval is credible, evaluate the generated response. These dimensions should not collapse into one unexplained average. A system can be helpful and fluent while still producing unsupported claims.

Automated graders can accelerate repeated testing, but subject-matter experts should review a representative sample—especially failures and consequential use cases.

  • Correctness: does the answer reach the right conclusion?
  • Completeness: does it cover the important parts of the question?
  • Faithfulness: can each factual claim be supported by retrieved material?
  • Citation precision: does each citation support the associated claim?
  • Citation coverage: are important claims supported rather than presented without evidence?
  • Usefulness: is the response clear enough for the intended user to act appropriately?

Test whether the system knows when to stop

A trustworthy assistant needs an honest no-answer behaviour. Test what happens when no relevant source exists, the available source is outdated, two approved documents disagree, the question lacks necessary details, the user requests restricted information, or retrieved passages mention the topic without answering the question.

The right response may be to ask a clarifying question, state that the approved sources do not support an answer, show the conflicting evidence, or escalate to an owner. NIST identifies confident false content—including invented supporting logic or citations—as a generative-AI risk. Refusal and escalation are therefore product capabilities, not signs that the system is unfinished.

Verify the operating system around the model

Answer quality is only part of production readiness. A RAG assistant is connected to a living knowledge system, and its quality can drift when sources and evaluation are left unattended.

The launch review should cover the system around the response as carefully as the response itself.

  • New and revised documents become searchable predictably
  • Deleted material disappears from retrieval
  • Permission changes take effect promptly
  • Retrieval and answer traces can be inspected without exposing sensitive content
  • Latency and usage cost fit the workflow
  • Users can report a poor answer and identify its source
  • An owner is responsible for reviewing failures
  • Model, prompt, and retrieval changes trigger regression testing

Ask an outsourced RAG partner for evidence

Before accepting a RAG project, ask the provider to demonstrate the evaluation set, retrieval results separately from final answers, failed and unanswerable examples, citation and permission tests, behaviour when documents conflict or change, the update and deletion workflow, the regression process, and the limitations that remain at launch.

The strongest handoff is not a perfect-looking chat screen. It is a system whose boundaries, evidence, failure modes, and maintenance process are visible.

The practical release gate

Do not ask whether the assistant is intelligent enough. Ask whether its behaviour is sufficiently predictable for the decision it supports.

A credible first release has a defined source and permission boundary, a versioned set of realistic questions, separate retrieval and answer evaluation, visible citations, tested no-answer and escalation behaviour, a controlled pilot with intended users, and an owner for monitoring and document quality. That is what turns a RAG prototype into a knowledge product people can use responsibly.

Primary sources