Prompt assembly is an engineering pipeline

When an agent gives a poor answer, the instruction “improve the prompt” is often too vague to be useful. Which prompt? The base persona, the booking procedure, the host’s property rule, the retrieved knowledge or the channel’s formatting instructions?

The client platform assembled the model input from named components. This made each turn inspectable and made changes more local. It also exposed a practical truth: the quality of the answer depends on the context pipeline as much as on the wording of instructions.

Separate instructions, facts and conversation state

The base persona defines the assistant’s role and general behaviour. A skill fragment defines a procedure. Property knowledge provides authored facts. Domain hydration supplies current reservation or listing data. Conversation state records what has already been offered, selected or reported.

These sources should not be flattened conceptually even if a provider ultimately receives text. Each has a different owner and freshness requirement.

If a retrieved manual contains “ignore the previous instructions,” it remains document content. It does not acquire the authority of a system instruction because it appears inside a prompt block. Scope and authorization still have to be enforced outside the model.

Hydrate independently, assemble deterministically

The context builder uses pluggable hydrators for different evidence sources. Independent reads can run concurrently. Their results are then rendered in a predictable order.

For example, property knowledge, current domain facts and host rules can often be fetched in parallel. Stay resolution may create a dependency: a property-specific retrieval cannot be safely scoped until the relevant stay is known. Parallelising everything blindly would turn a latency optimisation into a correctness bug.

import asyncio

async def hydrate_independent(loaders):
    names = list(loaders)
    results = await asyncio.gather(
        *(loaders[name]() for name in names),
        return_exceptions=True,
    )
    return {
        name: {"value": None, "degraded": True}
        if isinstance(value, Exception)
        else {"value": value, "degraded": False}
        for name, value in zip(names, results)
    }

This small example illustrates independent degradation, not an unlimited permission to continue. A missing optional brand-voice fragment can have a safe fallback. Missing authorization context cannot. Missing availability means the agent cannot claim a property is available.

The runtime records degradation so the operator can distinguish “no knowledge matched” from “the knowledge service could not be reached.” Both can produce an empty block; operationally they are different.

Keep the stable prefix stable

The system prompt is divided into a relatively stable base and a per-turn suffix. A provider supporting prefix caching can reuse the stable part, while current facts and instructions are assembled for the request.

Caching depends on the provider’s rules, including matching prefixes, supported models, minimum sizes and cache lifetime. A cache=True flag in an internal object is an instruction to the adapter, not proof of a cache hit. Hits must be observed in usage records. Anthropic prompt caching

A useful layout is:

Stable:
  persona → domain vocabulary → general operating rules

Per turn:
  date and channel → authenticated context → scoped knowledge
  → live facts → host rules → selected skill → conversation state
  → locale and outbound contract

The ordering should follow dependencies: establish identity before fetching scoped evidence, resolve ambiguous references before requesting resource-specific facts, and separate instructions from data. Record the resolved component versions so the assembled input can be reconstructed without exposing live prompt contents in routine telemetry.

Putting a timestamp at the very beginning of an otherwise cacheable prefix can invalidate reuse. Putting a volatile price inside the stable persona can produce a stale answer. The cache split is therefore both a cost decision and a data-lifetime decision.

Render variables in two passes

Prompt authors can supply defaults for declared variables. The platform then supplies reserved values such as the current date, channel, locale and scoped domain IDs for each turn.

The two-pass design avoids asking every prompt author to reproduce platform context. It also means author-provided defaults must not override authenticated scope. A variable called tenant_id is not an acceptable route for a prompt editor or model to choose another account.

Validate unresolved placeholders before publication. A required variable needs a declared provider and a failure policy; an optional variable needs an explicit fallback. Missing authorization context must stop the action, while a missing style preference can safely use the default voice. This prevents a prompt that renders successfully from being mistaken for one with complete evidence.

Contradictions have a location

Suppose the base says “never claim a booking is confirmed before payment,” while a skill says “confirm the booking after selecting a property.” The model has been given incompatible instructions. A longer prompt makes the conflict harder to inspect.

The correction is to establish ownership: selection returns a checkout opportunity; confirmation comes from a verified booking state. The skill’s language should follow that contract. The operational distinction belongs in the tool result and flow state, with the prompt reinforcing it.

This is why fragment review should include relationships, not just individual text. A perfectly reasonable fragment can become wrong when combined with another one.

Build a context budget around evidence

Context is finite even when the model’s nominal window is large. More material increases prefill time, cost and the chance that important facts are buried.

A useful allocation process reserves output and tool-result space first, then budgets the persona, current state, retrieved evidence and history. Consider an illustrative 16,000-token input envelope with 3,000 tokens of stable instructions, 2,000 of live state, 4,000 of relevant evidence and 5,000 of history. That leaves 2,000 tokens of input headroom for the next tool result. The numbers are a worked allocation, not a measured configuration of the client deployment.

Trimming must preserve evidence relationships. A price without currency, dates or a property reference is smaller but less useful. A tool result without its error status is actively misleading. A summary of previous offers must preserve which option was shown in which search.

The assembly view is a debugging tool

The runtime emits an assembly event describing the model, skills, tools and prompt-fragment breakdown. The admin playground renders that information alongside the conversation.

When a property answer is wrong, inspection can proceed in order: was the stay resolved correctly, was retrieval scoped to that property, did the expected fragment appear, and did the model receive the relevant fact? This is much faster than repeatedly editing the final instruction and hoping the next answer improves.

Prompt text can contain guest information and access details. Routine metrics should carry sizes, hashes and source references. Full prompt capture needs deliberate access and retention controls. Making a system inspectable does not require putting every conversation into every telemetry backend.

Test the assembly, not just the prose

Useful assembly tests compare the selected fragments and tool surface for the same scenario under code-backed and database-backed configuration. They verify scope, ordering, missing-context behaviour and cache-block construction.

A good regression fixture also records the reason a fragment is absent. “No reservation” and “reservation fetch failed” must not accidentally activate the same optimistic booking behaviour.

The result is a prompt pipeline that can be operated like software: inputs are explicit, composition is reproducible and failures can be localised. The next chapter examines the most easily confused of those inputs—memory.