How to test an AI knowledge base before your team uses it

What does testing an AI knowledge base involve?

Testing an AI knowledge base means checking, before people rely on it, that the assistant answers from the right sources, that every answer can be traced back to those sources, and that it says so when the sources do not support an answer. A clear answer in a demo proves none of this.

An AI knowledge base is a controlled set of source material that an assistant draws on when it answers: reports, policies, transcripts, submissions, evidence tables, spreadsheets and project files. Testing it covers five layers:

• Sources: the right material is in, the wrong material is out, and every file is readable.
• Retrieval: the system finds the right document and the right section.
• Answers: the text is accurate, specific and honest about its limits.
• Citations: each cited source exists and supports the claim attached to it.
• People: users know what the tool is for, how to check it, and who fixes it.

Testing assumes the documents have already been prepared. If they have not, start with how to get your documents ready for AI retrieval and come back to this plan once the source set is in place.

What should the knowledge base be allowed to do?

Write down the approved and unapproved uses before you test anything, because a test needs a target. A tool for finding documents needs different tests from one that drafts report text.

The unapproved list matters as much as the approved one. It becomes part of the test set: you will ask the assistant to do each unapproved thing and check that it declines or flags the request.

How do you check the source set before testing answers?

Check the inputs first, because an assistant cannot retrieve a document that is missing, unreadable or indistinguishable from an old draft. Many failures that look like AI errors start here.

A folder upload is not a source base. A controlled source base has a register of what is in it, with names, IDs, dates, versions, permissions and sensitivity flags. The fields that matter most for retrieval and filtering are set out in which metadata fields matter for AI retrieval.

How do you test retrieval separately from the writing?

Look at what the system retrieved before you judge the answer, because a polished answer can sit on top of the wrong sources. Many assistants can show the passages or files they used; where yours can, read those first.

Retrieval tests ask whether the system:

• finds the right document and the right section;
• misses an important source;
• pulls in irrelevant material;
• copes with acronyms, programme names and local terms;
• tells draft and final versions apart;
• recognises when the answer is not in the source set.

A simple test is to ask the same question three ways:

• "What does the strategy say about district-level coordination?"
• "Find the section on coordination between districts."
• "Where is district coordination discussed?"

If each phrasing returns different or weaker sources, the knowledge base is not ready. The cause may be document preparation, metadata, how documents were split into chunks, naming, or the assistant's instructions. When the pattern of failures is unclear, why AI gives weak answers when the source material is messy sets out how to trace a weak answer to its cause.

How do you score answer quality?

Score each answer against a short, fixed rubric so that two reviewers reach the same verdict. "Sounds right" is not a score.

Check, too, whether the answer keeps fact, summary and interpretation apart, and whether it states uncertainty where the sources are thin. A subject-matter reviewer, not the person who built the tool, should do the scoring.

How do you check citations and traceability?

Open every cited source in the test set and confirm that the passage supports the sentence it is attached to. A citation is not proof on its own.

NIST's Generative AI Profile lists, as a suggested action, reviewing and verifying sources and citations in generative AI outputs during pre-deployment testing and ongoing monitoring (NIST AI 600-1, 2024). It also notes that these systems can produce confabulated citations that make a wrong answer look justified.

Common citation failures:

• the cited document exists but does not say what the answer claims;
• the answer blends several sources and cites only one;
• the citation points to the right document but the wrong section or an old version;
• the citation is decoration: it is present, but no reviewer could use it to check the claim.

The test is whether a reviewer can move from the answer to the cited source, to the exact section, to the original document. That chain is the same one a report needs, set out in how to keep every claim in a report traceable to its source. If the chain breaks, the answer is not ready for serious use.

How should the assistant behave when the sources do not answer the question?

It should say so. An assistant that answers every question is not safe for evidence work, and the no-answer test is one of the quickest ways to find out.

Include questions that:

• fall outside the source material;
• ask about a document you know is missing;
• ask for a judgement or recommendation the evidence does not support;
• ask for confidential information;
• invite the assistant to guess.

Good behaviour sounds like "The available sources do not answer this", "The sources only cover part of this question" or "This needs a reviewer". Bad behaviour is a confident answer anyway, an invented source, general knowledge presented as project evidence, or no flag on uncertainty.

How do you test sensitive-data boundaries?

Test whether a user can reach restricted material, directly or through indirect questions, and whether the assistant summarises sensitive material more freely than your rules allow. The safer pattern is to keep sensitive material out unless it is needed, approved and access-controlled.

In South Africa, the Protection of Personal Information Act 4 of 2013 (POPIA) sets stricter conditions for special personal information, such as health information, and for the personal information of children (sections 26 to 35). Interview transcripts, case files and safeguarding records often contain both.

Test questions should check:

• whether users without access can retrieve restricted files through rephrased prompts;
• whether the assistant will give identifying details when asked;
• whether redacted versions should replace the originals;
• whether the user guide says what must not be pasted into the tool;
• whether the platform's data handling meets your organisation's and your funders' rules.

This is a governance question as much as a technical one, and the decisions belong to the data owner.

What goes in a test question bank?

A test question bank is a fixed, reusable set of questions with expected answers, run before launch and again after every meaningful change. It is the most useful asset the testing produces.

There is no standard number of questions that proves readiness. As a starting point: 20 to 30 for a small, low-risk source set; 40 to 75 for a medium one; 100 or more for a large or high-risk one, grouped by use case and risk level. Coverage of edge cases matters more than the count.

How do you log and fix failures?

Record every failed test in an issue log, so that fixes go to the right layer and every fix is retested. Without a log, failures become anecdotes.

AI KNOWLEDGE BASE ISSUE LOG
Test ID:
Test question:
Expected answer (and source):
Actual answer:
Sources retrieved:
Issue type: retrieval / citation / accuracy / sensitivity / format / instruction / missing source
Severity: low / medium / high / launch blocker
Fix needed: source change / metadata change / prompt change / user guidance / exclusion
Owner:
Retest status: not retested / passed / failed again

Not every problem is a prompt problem. Some failures come from missing documents, some from weak metadata, some from old files and some from unclear user instructions. The log shows which, and it becomes a handover record for whoever maintains the knowledge base.

What should users be told before launch?

Users need a short guide covering what the tool is for, what it is not for, which sources it holds, known gaps, how to check a citation, when to escalate to a reviewer, what not to paste in, how to report a bad answer, and who maintains it.

Keep it short enough to be read, with example questions and a "how to check an answer" section. Make clear that the tool supports search, summary and drafting, and that review and sign-off stay with people.

When is an AI knowledge base ready to launch?

It is ready when the tests have run, the launch blockers are fixed, the limits are documented and someone owns it, not when it feels useful. Most teams should launch to a limited pilot first.

Copy this checklist into your launch review:

AI KNOWLEDGE BASE LAUNCH CHECKLIST
[ ] Approved and unapproved uses written down
[ ] Approved source set and source register in place
[ ] Source IDs, dates and versions clear; old versions removed or marked
[ ] Retrieval tested with rephrased questions
[ ] Answers scored by a subject-matter reviewer
[ ] Every cited source in the test set opened and checked
[ ] No-answer behaviour tested
[ ] Sensitive-data boundaries tested
[ ] Launch blockers in the issue log fixed and retested
[ ] Known gaps documented
[ ] User guide written
[ ] Maintenance owner named
[ ] Human review rules agreed
[ ] Retesting schedule set

In our work on the White Paper on Local Government submissions, the evidence assistant worked only on approved, coded source material, and decisions stayed with the drafting team. That boundary is what the checklist protects. If you would rather not run this alone, we can deliver a source-linked evidence system, built, tested and handed over, with the test bank and issue log included.

What happens after launch?

Testing continues. Rerun the question bank when documents are added or removed, when instructions or user roles change, when the platform updates its model or retrieval settings, and when users report bad answers.

A knowledge base without an owner drifts: old documents stay in circulation, new material goes in unreviewed, and failures go unlogged. Name one person to update sources, review the issue log, retest after changes and keep the user guide current.

AI can help with the testing itself: drafting test questions, comparing expected and actual answers, grouping failures by type and summarising the issue log. It should not mark its own answers as accurate, decide whether sensitive material is safe, or decide launch readiness. People make those calls.

What tools do you need to test an AI knowledge base?

Most teams need a spreadsheet before they need specialist software. One workbook can hold the source register, the question bank, the scoring sheet, the issue log and the launch checklist.

Specialist evaluation tools earn their place when there are many users, complex permissions or a custom retrieval layer. The structure of the tests matters more than the software.

Common mistakes

• Uploading a folder and calling it a knowledge base. Sort, label and approve the source set first.
• Testing only demo questions. Include ambiguous, sensitive and out-of-scope questions.
• Not opening the citations. A citation that looks right may not support the claim.
• Keeping drafts beside final versions. The assistant may cite the draft.
• Judging retrieval by the answer. Fluent text can sit on weak sources.
• Skipping the no-answer test. An assistant that always answers is not safe for evidence work.
• No issue log. Failures are forgotten and fixes go to the wrong layer.
• No owner and no retesting. Every source change can change the answers.
• Pasting AI output straight into a report. The assistant supports drafting; it does not replace evidence review.