The system that keeps its shape

01 / The frame

Frame the job before choosing the agent

Start with the business outcome and the unacceptable outcomes. “Resolve a support case” is a goal. “Never issue a refund without an approved policy match” is a boundary. The architecture needs both.

Use a fixed workflow when the path is known and an agent when model-driven decisions are genuinely useful. Anthropic’s field guidance makes the same distinction: workflows favor predictability, while agents earn their complexity when the path cannot be prescribed in advance.

02 / The split

Decompose by contracts, not personalities

“Research agent,” “writer agent,” and “manager agent” are memorable labels, but they are not boundaries. A stronger split is behavioral: retrieve evidence, normalize records, apply a policy, draft a response, request approval.

Give each step a narrow input and a typed output. The orchestrator should know what a step promises and what it can reject; it should not need to know the model’s private chain of thought. OpenAI’s Agents SDK describes handoffs as delegation between specialized agents, but the same principle works with ordinary services and queues.

03 / The state

Keep workers stateless; keep progress durable

A worker should be safe to replace halfway through a run. Put durable progress in a workflow record: task ID, version, current step, input references, attempt count, and the last accepted output. The worker receives a slice, does work, writes an idempotent result, and disappears.

This is the difference between “the process is still alive” and “the work can continue.” AWS Step Functions documents the trade-off explicitly: Standard Workflows persist execution state between transitions, while Express modes trade that persistence for higher-volume processing models. Choose the semantic you need before choosing the product.

04 / The queue

Make retries boring

Every network boundary eventually times out. A retry policy should distinguish transient failures from invalid input, use bounded exponential backoff with jitter, and stop when the next attempt is more dangerous than the original failure. AWS Step Functions exposes retry and catch policies as first-class workflow behavior; that is a useful mental model even when your queue is homegrown.

Idempotency is the companion discipline. Put a stable key on every side effect, persist the result, and return the original receipt when the same key arrives again. Queue delivery guarantees do not make downstream APIs idempotent. Even SQS FIFO’s deduplication protection is bounded by a five-minute deduplication interval, so the application still needs a durable replay key.

05 / The edge

Treat tools as capability boundaries

A tool is not a clever prompt with an API attached. It is a permissioned boundary around a real effect. Give it the smallest useful surface, validate its arguments at the boundary, constrain the data it can read, and return structured errors the orchestrator can classify.

Keep “read,” “propose,” and “commit” separate. A model may be allowed to draft a SQL query without being allowed to run an arbitrary one, or to prepare an email without sending it. OWASP’s Excessive Agency guidance names excessive functionality, permissions, and autonomy as root causes; tool design is where those risks become concrete.

06 / The signals

Observe the whole loop

One final answer is not enough to operate an agent. Record a trace for the run, spans for model turns and tool calls, and metrics for queue age, step latency, token use, retries, policy failures, and human takeovers. OpenTelemetry’s primer frames traces, metrics, and logs as complementary signals; each answers a different operational question.

Evaluation belongs beside observability, not after it. Build a small set of representative tasks, adversarial inputs, and known failure cases. OpenAI’s evals documentation treats graders, data sources, and runs as first-class objects; use that same separation so you can tell whether a prompt change improved the behavior you care about or merely changed its style.

07 / The handoff

Escalate on purpose

Human review should not be a vague “if unsure.” Define triggers: irreversible effects, policy exceptions, low evidence coverage, repeated retries, or a risk score above a threshold. Preserve the run context so a reviewer sees the request, evidence, proposed action, and reason for the pause in one place.

Make the human decision an explicit state transition—approved, rejected, edited, or expired—and resume from that state. OpenAI’s agent primitives include guardrails and human-in-the-loop mechanisms; the broader design lesson is to keep the pause legible and reversible rather than hiding it in a timeout.

“

Autonomy is a budget you allocate per action, not a personality you assign to the model.

08 / The budget

Control cost, latency, and blast radius together

Cost and latency are architecture constraints. Set a maximum number of model turns, a tool-call deadline, and a per-run spend ceiling. Cache stable context, use smaller models for classification and routing, and reserve expensive reasoning for the steps where it changes the decision.

Fault isolation keeps one bad run from becoming an outage. Put concurrency limits around expensive tools, separate tenants or risk classes where necessary, and give every run a cancellation path. A queue that can grow without a drain plan is not elasticity; it is deferred failure.

09 / The release

Roll out in slices

Start with a read-only shadow mode. Let the agent produce plans and evidence while the existing path remains authoritative. Compare outcomes, inspect traces, and collect the cases where the agent would have made a different decision.

Promote one reversible action at a time. Use a small cohort, a hard kill switch, and a rollback that does not depend on the agent. Expand only when quality, latency, cost, and escalation rates are within the envelope you wrote down at the beginning.