Context, memory and live updates

An assistant needs to remember enough to continue the task, while the application needs to know what has actually changed. Those are related requirements, but they belong to different kinds of state.

We developed three layers: short-lived conversation context, optional longer-term memory, and event infrastructure for changes in the application. Understanding their boundaries makes the product easier to explain and easier to finish.

Working context should have a lifetime

The session manager stores conversation state in Redis. A save refreshes a 30-minute expiry, and the normal history policy retains 50 messages, preserving system messages and recent conversation. Session context includes the current patient, current screen, recent actions and user role.

Pending actions have their own storage and a shorter default lifetime. That distinction is sensible: a conversation may remain useful after a time-sensitive proposal needs to be reconsidered.

The domain records live elsewhere. A patient or journey does not disappear because the conversation expires. I want the assistant to reconstruct the task from authoritative records when necessary, rather than treating a long transcript as the source of truth.

Longer-term memory is an optional dependency

The orchestration layer includes an optional connection to our broader agent platform for memory, agent configuration and tracing. It can request contextual memory, store messages and obtain an agent's configured system prompt while retaining local tool execution.

The integration is controlled by configuration flags and has fallback paths when the remote service is unavailable. That is a useful separation between the application performing an operation and the platform helping configure or contextualize it.

The important check is whether retrieved information actually reaches the model making the decision. In this snapshot, the orchestrator fetches recent conversation history and passes it to the agent, but the shared agent execution path does not consume that argument. Memory-shaped interfaces therefore exist ahead of complete conversational continuity.

I would validate continuity with a small multi-turn example: select a patient, ask a follow-up, correct an earlier detail, then switch tasks. Inspect the actual model input and resulting operation. A memory retrieval log alone is insufficient.

“Realtime” describes several different things

The application contains distinct mechanisms that can make it feel responsive:

These distinctions keep the performance story honest. Sending a completed answer in chunks changes its presentation; it does not make the model's first token arrive sooner. Likewise, a WebSocket implementation elsewhere does not establish that the dashboard is receiving live updates.

To complete the experience, I would connect a real change to a visible update: modify an appointment, publish the event, refresh the relevant screen and handle reconnect without losing or duplicating the change.

Observe the operation, not just the model call

The optional tracing layer records intent classification, agent execution, tool timing and model usage. Those hooks are a useful start, but their meaning needs checking. A displayed card count, for example, is not necessarily the number of tools executed.

The trace I want follows the user's task: request, selected patient, proposed plan, approval, service result and final record. Sensitive content should be minimized in that trace, and access to patient context must be established independently of the model's output.

Give each kind of memory an owner

For the worked booking scenario, I would distinguish four state stores. Conversation history helps interpret “that one.” A workflow checkpoint records which step is waiting. A proposal records what can be approved. The appointment service records what happened. A message summary should never overwrite the latter three.

The following is a reference context envelope assembled by the server:

{
  "context_version": 12,
  "selected_patient": {"id": "p-001", "version": 7},
  "active_workflow": {"id": "wf-001", "state": "awaiting_slot"},
  "recent_actions": [{"operation": "patient.resolve", "status": "succeeded"}],
  "policy_refs": [{"id": "scheduling-policy", "version": 3}],
  "user_text": "Use the afternoon slot"
}

The current patient is a verified reference, not a copied clinical profile. A policy reference carries a version so a later answer can identify which rule it used. Free text remains user input, not permission metadata.

I would assemble this envelope after authorization and before model invocation. A browser tab switching patients should increment the context version and invalidate an interpretation based on the old selection. Otherwise a fluent “book that” can act on a patient the user is no longer viewing.

Budget context by relevance and sensitivity

For an initial implementation profile, I would reserve 8,000 input tokens: 1,200 for operating instructions and tools, 1,500 for recent conversation, 1,000 for workflow state, 2,500 for retrieved policy excerpts and 1,800 for headroom. These are allocation defaults, not measurements of the original model client.

The useful limit is also semantic. The appointment agent does not need a full history of unrelated laboratory results to reschedule a visit. Retrieval should select the scheduling policy, verified identifiers and the appointment facts required for this operation.

When summarizing old conversation, preserve unresolved questions and explicit corrections. “Patient selected: p-001; preferred location unconfirmed” is better than “We discussed a follow-up.” Store the summary's source message range and generation version. If a summary conflicts with a current record, the current record wins.

Publish progress events that correspond to work

The original SSE wrapper waits for the completed answer, then emits five-word groups. To provide useful execution progress, I would emit typed workflow events instead:

id: wf-001:17
event: proposal.ready
data: {"workflow_id":"wf-001","revision":2,"proposal_id":"proposal-001"}

id: wf-001:18
event: action.started
data: {"workflow_id":"wf-001","operation":"appointment.book"}

id: wf-001:19
event: action.succeeded
data: {"workflow_id":"wf-001","appointment_id":"appt-001"}

These are proposed application events, not a transcript of the current service. Each event needs an authorized stream subscription, a monotonically increasing sequence within the workflow and a replay policy. On reconnect, the client submits its last sequence; the server replays later events or tells it to reload authoritative state if history has expired.

A transport disconnect must not cancel a booking already committed. Conversely, displaying action.started does not justify a success message. The final event needs the verified operation result. Token streaming can coexist with these events, but it is a separate presentation channel.

Measure the wait the user experiences

I would capture request receipt, interpretation completion, proposal availability, confirmation receipt, commit and destination readiness as separate timestamps. A worked latency budget might allocate 100 ms to authentication and context, 1,200 ms to interpretation, 250 ms to availability lookup and 150 ms to proposal persistence and response: 1.7 seconds to a reviewable proposal before queueing and network variability.

That sum is a planning example, not a p95 derived from component percentiles. End-to-end percentiles must be measured from complete traces. The dashboard should show a slow dependency separately from time waiting for a human confirmation; otherwise the model gets blamed for a coordinator's lunch break.

I would also measure stale-context rejection, reconnect recovery and the rate at which the client shows a result unsupported by a persisted record. Those metrics are more revealing than counting how many tokens reached the browser.

What I would prove before expanding further

My next milestone would be one complete administrative workflow with demo fallbacks disabled: find a patient, resolve ambiguity, prepare a booking, confirm it, persist it and open the resulting record. Then I would repeat it with an expired proposal, a repeated click, a service failure and a reconnect.

That is an acceptance plan, not a claim that the repository already passed it. The implemented components give us a concrete foundation; those tests establish whether their connections are reliable.

The central lesson from the build is that an agentic product earns trust across boundaries. Interpretation, review, execution, memory and navigation each need a clear account of what they know and what they have completed.

The implementation kit includes the articles, diagrams and an offline reference lab. Code identified as a reference design is a new teaching implementation; it is not a claim that the original project already shipped that behavior.