A Private AI Pilot Needs a Scorecard Before It Needs a Winner

The demonstration finishes. The answer reads well, the source links look convincing, and somebody asks when the team can roll it out. One question is still unanswered: what evidence would justify that decision?

A private AI pilot needs an acceptance scorecard agreed before its strongest examples become the unofficial test. Otherwise, every awkward answer becomes an exception and every impressive answer becomes proof. The team can spend weeks improving the demonstration without learning whether the service is ready.

For a fictional internal knowledge assistant, here is a practical way to build that scorecard. These are proposed evaluation choices, not results from a customer deployment.

Turn the task into something a reviewer can judge

Start with a narrow job: help an authorised employee find and explain a current operating procedure. A useful response must identify the right procedure, answer the question from that document, cite the relevant passage, and stay within the employee's access. Fluent wording is only one part of that job.

Write a rubric with three quality levels. A score of two means the answer is usable without a material correction. One means a reviewer must repair it before use. Zero means it is wrong, unsupported, or fails to address the task. Give each level a concrete example and name the reviewer responsible for applying it.

Record separate fields for factual support, task completion, citation accuracy, and appropriate abstention. A single overall mark hides why an answer failed. An accurate answer with a fabricated citation and an incomplete answer with a correct citation need different fixes.

Give failures consequences before seeing the results

Some errors should block release for the intended scope. In this proposed assistant, disclosure of a restricted document is one. Presenting an invented policy as an approved instruction is another. Those events should remain visible even when most other answers are excellent.

Other failures may justify a narrower pilot. If the system handles current procedures but cannot reliably compare superseded versions, exclude version comparison, explain that boundary to users, and test whether it is respected. The release decision should describe exactly which work is supported.

NIST's Generative AI Profile calls for evaluating capability claims empirically and cautions against generalising from narrow, anecdotal assessments. Its suggested actions also cover checking generated sources and citations. The scorecard here is an application of that principle, rather than a NIST-prescribed scoring scheme. [1]

Include questions that deserve no answer

An evaluation set built entirely from answerable questions rewards confident completion. Real users will ask about missing documents, contradictory instructions, expired procedures, and material they are not allowed to see. Include those cases deliberately.

For a missing policy, a good response might explain that the available sources do not establish an answer and identify the appropriate escalation route. It should not fill the gap with a plausible rule. For conflicting versions, the expected behaviour might be to flag the conflict rather than silently choose the more convenient passage.

Pair allowed and denied cases around the same fictional document. Then change the user's access and repeat the test. Assess the application that enforces permissions, including retrieval and cached answers. Asking the language model to be careful does not test an access-control implementation.

Protect the test from becoming a rehearsal

Use a development set to improve prompts and retrieval. Keep a separate evaluation set for the actual release decision. Record who can see each set and what changed between runs. If the team edits a failing evaluation question until it passes, preserve the original failure and explain why the question was revised.

Synthetic examples are useful for checking whether the scoring process works. They do not establish how frequently a failure occurs in ordinary use. Eight carefully constructed tasks can expose a broken rubric; eight successful answers cannot establish dependable performance across an organisation.

For subjective judgements, have two reviewers score a sample independently. Keep their disagreement and resolution. If they cannot agree on what counts as usable, the rubric needs work before a small difference between models becomes meaningful.

Keep quality, responsiveness, and capacity visible

A correct answer can arrive too late to help. A fast answer can be useless. Report quality and service behaviour separately, then check them together in the application under the intended demand.

vLLM measures benchmark latency at the client. Its documentation distinguishes the wait for the first streamed output from request-level generation time and the gaps between streamed outputs. Those definitions matter when a team compares measurements from different tools. [2]

Choose responsiveness objectives with the people doing the work. Record the input lengths, expected response lengths, request arrival pattern, timeout, and measurement boundary. A local serving test omits parts of the experience if the deployed application also performs authentication, retrieval, document filtering, and network transport.

Retain failures in the service report. Show the number attempted, completed, timed out, and rejected, together with the latency distribution for successful requests. If retries are allowed, link them to the original request. An unsuccessful first attempt does not disappear because the second one worked.

Leave a decision someone else can reconstruct

The acceptance record should identify the model and software revisions, evaluation-set version, scoring rules, run conditions, raw evidence, and unresolved failures. It should also state the tested scope and the person accountable for the decision.

Proceed when the agreed criteria are met within that scope. Revise when a specific change has a credible path to resolving a failure. Stop when the remaining gap makes the service unsuitable for the intended work. Keep the original criteria alongside the outcome, including any changes made after testing.

The companion local AI evaluation design study translates this scorecard into proposed workload profiles and evidence files. The research protocol now includes a synthetic preparation pack. Both are starting points for an accountable evaluation. A winner still needs evidence.