Find and repair gaps in a Chat Agent’s knowledge
Knowledge diagnostics and the evaluation suite help you understand whether a Chat Agent has enough approved information to answer the questions it receives reliably. A knowledge gap may occur when: \- The required information has not been added to your workspace. \- The relevant object has not been assigned to the agent. \- A source record is incomplete or outdated. \- Recently changed content has not yet been prepared for retrieval. \- The question is outside the agent's intended purpose. ## How the evaluation suite works The evaluation suite runs your configured questions against the agent's current draft and approved knowledge. Questions are processed in batches of up to eight. Each answer is checkpointed individually as it completes, so progress is saved even if a batch is interrupted. Key behaviours to understand: \- **Up to 20 questions** are evaluated per run. Duplicate or blank questions are deduplicated automatically. \- **Batches of eight** are processed at a time. After one batch finishes, select **Continue** to process the next. \- **Resuming after failure** picks up from where the run left off. Only remaining unanswered questions are processed; completed answers are not repeated. \- **Source coverage** measures whether a search found a relevant published source for each question. It does not verify that the answer is factually correct — always review individual answers alongside the sources cited. \- **Partial and failed evaluations** do not count as launch-ready. An evaluation must complete all questions successfully before it can satisfy the launch readiness check. ## Evaluation results and the editor Evaluation results appear in the agent editor and in Activity history. Both show: \- How many questions have been evaluated out of the total. \- How many are still pending. \- The source coverage percentage among evaluated questions. \- Whether the result is for the current draft or an older one. \- Expanded individual answers and their cited sources. A result labelled **Older draft** means the agent draft was edited or saved since the evaluation ran. The result no longer reflects the current draft. Navigating away from the editor and returning also marks results as stale. Run a new evaluation to get a current result. ## Stale results and draft changes When you edit and save the agent, any displayed evaluation result is marked as an older-draft result. Unsaved changes in the editor also mark the displayed result as stale. A stale result cannot be used to satisfy the launch readiness check, even if its source coverage was high. ## Run the evaluation suite 1\. Open **Chat & Agents**. 2\. Select **Agents**. 3\. Open the agent you want to evaluate. 4\. Go to the agent's **Operations** or **Health** section. 5\. Confirm that the agent has at least three evaluation questions configured. 6\. Select **Run evaluation**. Hub runs the first batch of up to eight questions. When the batch completes, select **Continue** to run the next batch. Repeat until all questions are evaluated. If a batch fails, the progress up to that point is saved. Select **Continue** or **Retry** to resume from the last checkpointed answer. ## Review evaluation results After each batch completes: \- Check the source coverage percentage. A low percentage suggests that approved knowledge is missing or not assigned to this agent. \- Expand individual question results to read the full answer and the sources cited. \- Look for **Content gap** labels — these indicate questions where no relevant published source was found. \- Remember that **Source found** confirms a search result was available. It does not confirm the answer is complete or correct for a real customer question. ## Resolve a missing-content problem If the answer does not exist in your workspace: 1\. Open the object that should contain the information. 2\. Create a new record or improve the relevant existing record. 3\. Make the answer clear, specific and suitable for customers. 4\. Return to the agent. 5\. Select **Rebuild assigned knowledge**. 6\. Save the agent to advance its draft revision. 7\. Run a new evaluation. ## Resolve an assignment problem If the information exists but is not assigned to the agent: 1\. Open the agent's knowledge settings. 2\. Confirm that the correct object is selected. 3\. Check that the required records are published and not archived. 4\. Save the configuration. 5\. Rebuild the assigned knowledge. 6\. Run a new evaluation. ## Improve unclear answers When the agent finds relevant content but produces an incomplete answer, review the source record itself. Good source content should: \- Use a clear title. \- Answer one subject directly. \- Include important conditions or exceptions. \- Avoid contradictory instructions. \- Use the terminology customers are likely to recognise. ## Evaluation and launch readiness The agent editor shows a launch readiness check for evaluation. This check passes only when: \- At least three evaluation questions are configured. \- The latest evaluation covers the same draft revision that is currently saved. \- No questions are pending (the run is complete, not partial). \- The evaluation run status is successful — partial or failed runs do not count. If the check shows a pending count, select **Continue** to finish the remaining batches. If the check shows an older-draft warning, save the agent and run a fresh evaluation. ## When to rebuild assigned knowledge Rebuild the agent's knowledge after: \- Assigning a new object. \- Adding important records. \- Making substantial changes to existing content. \- Correcting inaccurate information. \- Seeing a diagnostic gap for content that already exists. Rebuilding refreshes the assigned knowledge. It does not rewrite or delete the original object records. After rebuilding, save the agent and run a new evaluation to confirm the gaps are resolved. ## Ongoing maintenance Review diagnostics regularly, especially after launching an agent or changing important business information. Repeated content-gap results are useful signals that your knowledge library needs another article, product detail or process record.