Enterprise search that respects changing access

This worked application uses explicit planning assumptions to demonstrate FIELD. Quantities below are inputs, calculated implications or proposed targets; they are not reported customer results. The engagement decision remains conditional on the evidence described.

Planning basis and engagement boundary

F — Frame the outcome

Give staff useful enterprise search while the data owner retains the 60-second revocation contract; retrieval usefulness cannot compensate for unauthorized disclosure.

Value measure: Time to a verified authorized answer and zero observed leakage in defined tests.

I — Investigate the workflow

Probe how each source updates permissions and group membership. Identify whether the promised revocation latency is technically supported before estimating answer quality.

First evidence task: Revoke access mid-response and query an old cached result.

E — Engineer the solution

Separate content and authorization freshness; link access enforcement and cache invalidation to tests while retaining the vector, token and filter sizing analysis.

The implementation assessment below makes the authority, workload, resource and failure assumptions concrete. Its numbers are calculations from the stated scenario, not a vendor performance guarantee.

Contract

Content updates may be daily, but revoked permissions must stop access within 60s. Define whether an in-flight response is stopped at revocation and whether group membership comes from a source with adequate change-notification capability. If the identity/source systems cannot support the deadline, expose that feasibility gap before promising it.

Implementation plan

Store immutable document/chunk versions, source ACL references and active manifests. Maintain a separately fresh permission service/cache with bounded expiry and revocation signals. Authenticate the user, retrieve only eligible candidates, revalidate selected evidence, rerank and generate. Recheck sensitive delivery if policy requires it. Caches include tenant, permission scope/version, corpus/model/prompt versions; expired authority does not become valid through a cache hit.

Sizing

One million documents is not one million chunks. Assume five chunks/document: 5M×768×4=15.36GB raw vectors, before ANN metadata, ACLs, text and replicas. If only one vector/document, raw storage is 3.072GB. At 100 queries/s and 2,000 input/300 output tokens, offered generation demand is 200k input/30k output tokens/s. Parsing and embedding refresh require separate throughput calculations based on changed pages/chunks and source quotas. Large group sets make authorization joins/filter selectivity part of the latency benchmark.

Revocation failure path

Invalidate permission-dependent answers, reject stale ACL snapshots after their allowed age and prevent pending jobs from using old scope. During identity outage, either use a still-valid bounded snapshot or deny requests whose authority cannot be established. Keep the security decision outside model output. Content deletion also propagates to vectors and summaries, but authoritative denial should work before cleanup finishes.

Verification and rollout

Seed two departments with distinct synthetic secrets. Test retrieval, reranking, cache hits, streaming, queue resumption and post-restore behavior. Measure recall within the authorized corpus, grounding, permission freshness and tails under highly selective ACLs. Start with a small set of source systems whose permission semantics are understood.

Alternative

Separate per-tenant indexes simplify some boundaries but increase index/operations cost and do not remove per-document permissions within a tenant.

L — Launch with evidence

Pilot departments with distinct permissions. Measure authorized recall and human verification effort alongside end-to-end revocation tests through caches, streams and restores.

Critical decision condition: Failed revocation or cross-tenant access tests block the affected rollout.

Attach raw artifacts, the tested configuration, population, denominator, date and limitations to the release record. Missing or stale evidence holds the dependent decision. A favorable value forecast cannot offset a failed critical condition.

D — Develop ownership and reuse

The identity/data owner accepts permission semantics and the service owner accepts monitoring. Reuse the ACL contract only with sources that meet its assumptions.

Candidate reusable asset: Source permission adapter plus revocation fixtures.

Commercial decision and next commitment

Accountable decision owner: Data owner and accountable service release owner.

The initial commitment is the bounded journey above, with the stated exclusions. Confirm access, customer staffing and acceptance ownership during discovery; price and schedule unresolved dependencies explicitly. A new source, tenant class, language, action or operating envelope requires an impact review rather than silent scope growth.

Approve only the next exposure level supported by the evidence. The next useful investment is: revoke access mid-response and query an old cached result.

A deeper FIELD walkthrough

F — Define the business value

In this worked engagement, assume 2,000 employees generate 12,000 eligible policy lookups/month. The existing workflow takes six human minutes. Plan for 50% attempted use, 0.5 minutes interaction, 1.5 minutes review and 10% fallback requiring six additional minutes. Assisted effort is 2.6 minutes; 6,000 attempts free 340 hours/month. At SGD 60/hour and 50% realization, capacity value is SGD 10,200. With SGD 0.20/attempt, SGD 4,000 fixed monthly cost and SGD 36,000 build cost allocated over 12 months, net after allocation is SGD 2,000/month. This is a conditional value case, not a measured saving.

I — Resolve the decisive uncertainty

Observe lookup and verification separately. Sample current policies, superseded versions, attachments and documents with different access groups. Ask the data owner whether a denied document may influence a summary at all. Validate deletions and permission changes through indexing, result caches and generated-answer caches. The million-document corpus is a storage scope; 12,000 monthly tasks is the business-volume assumption. The 100-query/s peak is a separately tested burst envelope.

E — Bound the release

Start with one department, one source and read-only answers. Carry identity to retrieval, filter before context assembly, and recheck sensitive access where the freshness contract requires it. Store source revision and citation spans with each answer. If revocation propagation cannot meet the proposed 60-second bound, shorten caches, add an enforcing read check or narrow the promise. Do not hide the gap behind a confident answer.

For capacity, five million 768-dimensional float32 vectors require 15.36 GB decimal raw. This excludes graph/index structures, metadata, replicas, caches and source text. A candidate pilot topology might use two 4-vCPU/16-GiB application instances and a managed index with a measured memory/storage envelope. That is a test configuration, not a claim that these machines can serve the specified peak. At 100 queries/s, 2,000 input and 300 output tokens/query imply 200,000 input and 30,000 output tokens/s if every query uses that path; model quota must match the burst or admission must bound it.

L — Make the evidence decisive

Propose 240 acceptance tasks: 100 routine lookups, 40 multi-source questions, 30 stale/amended-policy cases, 30 denied/revoked-access cases, 20 missing-evidence cases and 20 injection attempts. Require every critical access test to block forbidden evidence. Propose 95% supported answers among answerable routine tasks and p95 first useful response within three seconds on the stated pilot workload. Record target misses separately from security failures. A source unavailable during evaluation is a limitation, not an automatic pass.

D — Transfer the operation

The knowledge owner maintains source quality and stale-policy handling. The service owner monitors index lag, revocation lag, citation failures, fallback and review time. An operator performs a revocation, reindex and recovery exercise using their own access. Expand corpus and departments in separately evaluated slices.

Decision under these assumptions

Fund the bounded pilot. The monthly margin after build allocation is only SGD 2,000, so verification burden is a sensitive input. Increasing review from 1.5 to two minutes removes 50 freed hours and SGD 1,500 of realized capacity value, leaving SGD 500. A modest improvement in answer quality that increases review beyond this may weaken the business case. The next experiment measures verification effort before adding another source.

Method and references

Apply the FIELD method, economic model and evidence rules. Methodological and technical influences are listed in the source register.