Choose a task with an observable finish line
A first pilot should answer one operational question: can this workflow produce an acceptable result under realistic conditions, with manageable review and operating effort? Avoid a proposal such as “add AI across the company.” Define an input, a desired output, an owner and a point at which a human can determine whether the task succeeded.
A reviewed first draft of an internal support reply is easier to assess than an autonomous system that promises to resolve every customer issue. The narrower pilot can still expose important requirements: source quality, permissions, failure handling and review time. Its small scope is a way to collect evidence before expanding responsibility.
Screen candidate workflows before building
| Question | Favorable condition | Pause condition |
|---|---|---|
| Can correctness be checked? | A reviewer can compare output with approved evidence | Errors are hard to detect until harm occurs |
| Is information ready? | Current, authorized sources with clear ownership | Conflicting documents or unknown permissions |
| Can failure be contained? | Draft-only output and a manual fallback | Immediate irreversible external actions |
| Is there enough repetition? | A recurring task with comparable examples | A rare task with no usable evaluation cases |
Do not average away a critical risk. A task with a large potential time saving can still be unsuitable if the team cannot detect a serious error. Resolve that limitation or choose a different task. The NIST AI Risk Management Framework offers broader background for organizing AI risk work; the screening table here is an illustrative project method, not a certification.
Worked example: drafting answers from approved guidance
Suppose staff repeatedly answer questions about a company's software onboarding process. A reasonable pilot could retrieve the current onboarding guide and draft an answer for a staff member to approve. Exclude account changes, billing commitments and messages sent automatically to customers. Provide an explicit “insufficient information” path when the guidance does not answer the question.
Build a test set containing ordinary questions, ambiguous requests, outdated terminology and questions outside the approved scope. Record the expected source and acceptable answer for each case. Evaluate retrieval, factual accuracy, reviewer corrections and total handling time separately. A fast response with an incorrect prerequisite is not a successful result.
Compare against the existing manual process using similar cases. Include setup, review and correction effort. If staff spend longer fixing plausible errors than they previously spent writing answers, the pilot has not yet demonstrated useful savings.
Agree expansion, revision and stop conditions
Before starting, specify what evidence would justify broader use. Examples include no unauthorized data exposure in the test set, reliable escalation on unanswered questions and a measurable improvement in total handling effort. Select thresholds with the process owner; an example percentage from another business is not automatically appropriate.
Keep a change log for source updates, prompts and model versions. Re-run important cases after changes. Assign someone to own the tool after the pilot, including access reviews, cost monitoring and fallback operation. If no team can support it, production rollout is premature even when the demonstration looks convincing.
Next, use pilot acceptance criteria, value after review and the pilot exit decision to make the proposal reviewable.
A practical starting point
Work through these checks in order. Use the results to decide what needs to change in your own workflow.
- Map the current task, its inputs and the person responsible for the outcome.
- Identify repetitive judgment or language work with a verifiable result.
- Choose a pilot that can run alongside the existing process.
Drafting internal ticket summaries may be easier to evaluate than autonomously resolving complex disputes.
A mistake to avoid
Do not select a use case solely because a demonstration looks impressive.
How to check progress
Measure accepted outputs, review time and exceptions against the existing workflow.
Your action checklist
Use this checklist while reviewing your own setup. Checkmarks are temporary and stay in this open page.
Further reading
Official background resources for this topic. The workflow above is general guidance, not a guarantee of a particular result.
AI-assisted editorial content. Examples are illustrative, not client case studies. Read our editorial policy or report a correction.