Recovering a pilot whose users are leaving

This worked application uses explicit planning assumptions to demonstrate FIELD. Quantities below are inputs, calculated implications or proposed targets; they are not reported customer results. The engagement decision remains conditional on the evidence described.

Planning basis and engagement boundary

F — Frame the outcome

Decide whether to repair, reposition or stop the assistant; declining use and expensive verification are outcome evidence, despite 90/100 offline accuracy.

Value measure: Human minutes and cost per completed task; eligible-to-used funnel with abandonment.

I — Investigate the workflow

Observe abandoned and completed tasks. Decompose extra work into verification, context switching, corrections, latency and inappropriate task selection.

First evidence task: Time baseline versus assistance including verification and exception correction.

E — Engineer the solution

Instrument the full funnel and human effort. Improve evidence presentation or constrain task scope before increasing model or architecture complexity.

The implementation assessment below makes the authority, workload, resource and failure assumptions concrete. Its numbers are calculations from the stated scenario, not a vendor performance guarantee.

Interpretation

Ninety percent accuracy on 100 selected examples is limited offline evidence. It does not explain why adoption fell or whether verification erases value. First inspect sample selection, task mix and whether accuracy included source checks, exception handling and user effort.

Diagnosis

Instrument the workflow funnel: eligible task→opened assistant→submitted→received useful result→verified→used in actual work→returned later. talk with active and abandoned users, and observe representative tasks. Separate avoidable UI/context switching from missing evidence, poor task fit, latency and genuine quality failure. Check whether the workflow imposes double entry or forces users to re-open every source.

Worked value model

Suppose baseline handling is eight minutes. New interaction takes two, verification five and exception correction averages two: total nine minutes, a one-minute loss despite attractive response accuracy. This is an illustrative hypothesis to measure. Track distributions and error severity, not just average time among completed happy paths.

Design response

Improve citations/side-by-side evidence, integrate into the existing task surface, prefill structured inputs and narrow to tasks with reliable evidence. Retain a search/rules baseline. If verification remains inherently expensive, stop or reposition the product rather than expanding the pilot.

Experiment

Define a representative task set, blinded correctness review where possible and measured handling time including failures. Compare revised workflow with baseline using suitable assignment/paired tasks while avoiding learning effects. Measure adoption over an agreed window, not one demo session. Keep quality and usage denominators visible.

Executive pressure

Report the 90/100 development result as such, alongside declining usage and net-time evidence. Explain what it establishes and what it does not. Recommend a bounded repair with a decision date and stop criteria.

L — Launch with evidence

Compare revised assistance with the current workflow on representative tasks. Require positive net effort improvement and acceptable correctness, not a chosen accuracy headline.

Critical decision condition: Do not declare business success from the offline score while net work increases.

Attach raw artifacts, the tested configuration, population, denominator, date and limitations to the release record. Missing or stale evidence holds the dependent decision. A favorable value forecast cannot offset a failed critical condition.

D — Develop ownership and reuse

The workflow owner keeps the adoption/value review after the pilot. Stop funding if a bounded repair fails its predeclared test.

Candidate reusable asset: Full-workflow timing and adoption diagnostic.

Commercial decision and next commitment

Accountable decision owner: Business sponsor with actual workflow owner.

The initial commitment is the bounded journey above, with the stated exclusions. Confirm access, customer staffing and acceptance ownership during discovery; price and schedule unresolved dependencies explicitly. A new source, tenant class, language, action or operating envelope requires an impact review rather than silent scope growth.

Approve only the next exposure level supported by the evidence. The next useful investment is: time baseline versus assistance including verification and exception correction.

Method and references

Apply the FIELD method, economic model and evidence rules. Methodological and technical influences are listed in the source register.