Evaluate a marketplace change without misleading attribution

This worked application uses explicit planning assumptions to demonstrate FIELD. Quantities below are inputs, calculated implications or proposed targets; they are not reported customer results. The engagement decision remains conditional on the evidence described.

Planning basis and engagement boundary

F — Frame the outcome

Decide whether a marketplace intervention improves the market, not merely the directly treated user's metric.

Value measure: Market-level outcome and participant guardrails with design-appropriate uncertainty.

I — Investigate the workflow

Map supply/demand interference, carryover, promotions and actual assignment/exposure. Establish independent experimental units before promising precision.

First evidence task: Run a small assignment/carryover diagnostic before committing to a user-level A/B test.

E — Engineer the solution

Use cluster or switchback design where justified, serving assignment integrity and capacity parity between conditions.

The implementation assessment below makes the authority, workload, resource and failure assumptions concrete. Its numbers are calculations from the stated scenario, not a vendor performance guarantee.

Causal question

Define the intervention and whose outcomes matter: users, merchants/drivers, supply availability, cancellations and longer-term behavior. If one treatment user's allocation removes supply from another user, independent user-level treatment assumptions are questionable.

Implementation plan

Map the likely interference network. Use sufficiently independent geographic/market clusters when possible, or switchback time blocks when the market-level treatment can alternate with manageable carryover. Choose block length using response/carryover dynamics and balance time-of-day/day-of-week. Neither design automatically eliminates spillovers.

Analysis

Randomize at the chosen unit, log assignment and actual exposure, predeclare primary metric, guardrails, effect size and stopping policy. Cluster-level inference or the appropriate randomized design analysis respects dependence. Thousands of transactions in a handful of markets do not create thousands of independent units. Power planning must use cluster/time variation.

Resource/operations

Keep serving assignment stable and observable. Ensure treatment capacity does not introduce latency/timeouts that masquerade as model-quality changes. Retain enough logging to reconstruct supply/availability at decision time, with a clear retention/privacy policy.

Failure scenarios

A promotion can confound effects if uneven across arms/blocks; plan exclusion/stratification or rerun rather than select a favorable window afterward. Cross-region demand movement requires spillover analysis and perhaps larger clusters. If conversion rises while cancellations worsen, the guardrail decides whether to stop or narrow the treatment.

Outcome

Report effect size, uncertainty, practical trade-offs and limits on generalizing to other markets/seasons. If interference cannot be adequately controlled, present observational evidence as such and propose a better experiment rather than claiming causal lift.

L — Launch with evidence

Predeclare outcomes, guardrails and stopping policy; analyze at the randomization unit and report spillovers/uncertainty rather than counting transactions as independent samples.

Critical decision condition: A biased assignment or unaddressed interference invalidates the causal rollout claim.

Attach raw artifacts, the tested configuration, population, denominator, date and limitations to the release record. Missing or stale evidence holds the dependent decision. A favorable value forecast cannot offset a failed critical condition.

D — Develop ownership and reuse

An experiment owner maintains assignment records and the product owner accepts the business trade-offs and follow-up window.

Candidate reusable asset: Interference-aware experiment contract.

Commercial decision and next commitment

Accountable decision owner: Marketplace product owner with experimentation lead.

The initial commitment is the bounded journey above, with the stated exclusions. Confirm access, customer staffing and acceptance ownership during discovery; price and schedule unresolved dependencies explicitly. A new source, tenant class, language, action or operating envelope requires an impact review rather than silent scope growth.

Approve only the next exposure level supported by the evidence. The next useful investment is: run a small assignment/carryover diagnostic before committing to a user-level A/B test.

Method and references

Apply the FIELD method, economic model and evidence rules. Methodological and technical influences are listed in the source register.