Share GPU capacity without breaking interactive work

This worked application uses explicit planning assumptions to demonstrate FIELD. Quantities below are inputs, calculated implications or proposed targets; they are not reported customer results. The engagement decision remains conditional on the evidence described.

Planning basis and engagement boundary

F — Frame the outcome

Share scarce GPUs only when interactive deadlines, background progress and tenant fairness remain useful to their owners.

Value measure: Interactive goodput, batch deadline attainment and cost/fairness by tenant.

I — Investigate the workflow

Classify workloads, residency, preemption costs, batch deadlines and the economic value of reserve capacity.

First evidence task: Fail a device during mixed load and measure useful surviving capacity.

E — Engineer the solution

Use measured isolation/scheduling and token/KV admission; include warm model availability and device-failure headroom.

The implementation assessment below makes the authority, workload, resource and failure assumptions concrete. Its numbers are calculations from the stated scenario, not a vendor performance guarantee.

Contract

Interactive generation has latency deadlines; embedding and evaluation jobs have throughput/completion targets. Specify tenant shares, batch-job deadlines, model compatibility and allowed preemption. High GPU utilization is not the sole success metric.

Implementation plan

A control plane admits workload classes and budgets; serving pools enforce token/cache limits and fairness. Begin with separate interactive and batch pools if workload interference is hard to bound, sharing overflow only under measured rules. Keep a queue/checkpoint path for deferrable batch work and warm reserve for critical interactive demand.

Placement

Consider model residency, device memory, KV growth and host pipeline rather than abstract “GPU fractions.” Co-resident models can fragment memory or compete for bandwidth. Hardware partitioning/time sharing, where supported, changes isolation and usable resources and must be validated for the specific device/runtime. A batch task may not be cheaply preemptible mid-kernel or without a checkpoint.

Sizing

For an illustrative four-GPU fleet, dedicating two to interactive, one to batch and one as reserve is only a starting policy. It is valid only if the surviving warm/ready capacity meets interactive demand after the selected failure and required models fit. Cold-loading the reserve can violate SLOs; test the actual readiness delay. Measure batch progress so “low priority” does not become starvation.

Changed conditions

A long-context tenant is constrained by token/KV admission and per-tenant active state, not just request count. Eviction/recompute may hurt latency; reject or queue oversized work with truthful deadlines. On device failure, route to ready capacity, stop new low-priority admission and reconcile in-flight jobs.

Evaluation

Compare mixed and isolated runs using the same jobs: interactive goodput/tails, batch deadline completion, per-tenant fairness, OOMs, cold-start recovery and total cost. Profile host CPU/I/O to avoid blaming the scheduler for a starved GPU.

L — Launch with evidence

Compare mixed versus isolated runs at identical workload. Check batch starvation, interactive tails, OOMs and recovery before increasing sharing.

Critical decision condition: High utilization cannot compensate for missed critical deadlines or unusable failover capacity.

Attach raw artifacts, the tested configuration, population, denominator, date and limitations to the release record. Missing or stale evidence holds the dependent decision. A favorable value forecast cannot offset a failed critical condition.

D — Develop ownership and reuse

Platform operators own scheduler policy; workload owners agree priority and degradation rules rather than fighting through retries.

Candidate reusable asset: Workload-class admission contract.

Commercial decision and next commitment

Accountable decision owner: GPU platform owner with workload owners.

The initial commitment is the bounded journey above, with the stated exclusions. Confirm access, customer staffing and acceptance ownership during discovery; price and schedule unresolved dependencies explicitly. A new source, tenant class, language, action or operating envelope requires an impact review rather than silent scope growth.

Approve only the next exposure level supported by the evidence. The next useful investment is: fail a device during mixed load and measure useful surviving capacity.

Method and references

Apply the FIELD method, economic model and evidence rules. Methodological and technical influences are listed in the source register.