Back to libraryContentsContents

Essay

Engineering Handbook: Executive Summary

A concise management introduction to production engineering for agent workflows.

About 5 min read

Contents

Production Agent-Centric Workflows

Based on: Engineering Handbook: Production Agent-Centric Workflows

For the wider organizational perspective, see the shared executive summary. This document retains the handbook’s deeper management introduction to production engineering.

Core idea

A production system should not rely on a “magical AI agent” that understands everything, makes decisions by itself, and performs actions on its own.

The design unit is a production workflow, where traditional software, agents, tools, APIs, the data plane, and humans work together in a controlled process.

The core architectural formula is:

The workflow owns the process.
The agent owns only a bounded task.

An agent is not a model and not a prompt. An agent is a software component with a task, a contract, inputs, outputs, allowed tools, constraints, evals, traces, a version, and an owner.


Three core principles

1. Prompt explains policy; system enforces policy

A prompt can explain rules to the agent, but a prompt is not a security boundary.

Real constraints must be enforced by:

  • application logic;
  • tool executor;
  • runtime sandbox;
  • network policy;
  • database authorization;
  • policy engine;
  • enforced human approval gates.

Audit and incident response support detection, investigation and recovery; they do not themselves block an unauthorized action.

If the model makes a mistake, receives prompt injection, or confidently proposes an unsafe action, the system must still technically block unauthorized access or side effects.

2. Memory is governed data

Agent memory cannot be built as just “a summary in the prompt.”

Production memory must preserve:

  • scope;
  • authorization;
  • source;
  • provenance;
  • trust class;
  • valid time;
  • current vs historical truth;
  • correction and supersession;
  • deletion/forgetting;
  • retrieval audit.

Key rules:

Chat summary is not source of truth.
Vector search is not authorization.
Model inference is not authoritative fact.
User claim is not verified business data.

3. Tools are capabilities, not access

An agent does not receive “database access,” “email access,” or “CRM access.”

It receives narrow tools:

  • read approved document;
  • get customer summary;
  • create draft;
  • create review task;
  • propose intent;
  • execute approved action.

Every tool needs an owner, schema, permissions, risk level, audit, timeout, approval requirements, and a kill switch.


Two production-hardening layers

The handbook develops these principles through two complementary layers.

Secure Autonomous Execution

If an agent can run code, modify a workspace, use browser/computer use, install packages, or work with MCP/tools, security must be built on technical boundaries:

  • runtime isolation;
  • private clone or ephemeral workspace;
  • no host Docker socket;
  • deny-by-default network;
  • scoped secrets;
  • egress controls;
  • MCP registry;
  • tool authorization;
  • supply-chain controls.

Important rule:

Agent sandboxed ≠ tools sandboxed.
Agent sandboxed ≠ MCP server sandboxed.
Agent sandboxed ≠ downstream API permissions scoped.

Durable Memory & Data-Plane Authorization

If an agent works with private data or durable memory, access and trust must be enforced by the data plane.

A strong practical formula:

“Viewer agent” with root database credential is not a viewer.
It is a root process politely asked to behave like a viewer.

The system should use least-privilege identity, explicit scope and visibility boundaries, RLS or other backend enforcement where appropriate, field-level protection where needed, relationship-based access where needed, retrieval traces, and denied-access audit.


What a mature workflow looks like

A proper production workflow usually works like this:

  1. System receives request.
  2. Deterministic code normalizes input.
  3. Agent performs a bounded classification or drafting task.
  4. System retrieves memory/knowledge with authorization and provenance.
  5. Deterministic code checks risk.
  6. Workflow routes the case.
  7. Human review is created if needed.
  8. Side effect is executed only through intent, authorization, approval/policy envelope, execution, readback, and audit.
  9. Trace, retrieval trace, tool audit, and eval feedback are stored.

This is a design for controlled automation, not proof that an implementation is production-ready. Approval must bind to the action being executed, with current authorization enforced at the effect boundary. An idempotency key only helps when deduplication and provider retry semantics are implemented. An uncertain external outcome calls for reconciliation, not a blind retry. Rollback of software does not undo every completed action.

Its operations must distinguish runtime health, outcome health, and trajectory health. A completed run with valid tool calls is not sufficient evidence that a workflow achieved its business goal safely. Production traces reveal recurring patterns; confirmed cases become regression evidence and controlled experiments; authorized owners, not the agent, promote changes.


What leadership should control

The key question should not be only:

“Do we already have an agent?”

Better questions are:

  • Which workflow are we automating?
  • What is the business outcome?
  • What is the risk tier?
  • What can the agent do?
  • What is the agent forbidden to do?
  • Which data sources are trusted and which are untrusted?
  • Which memory records are authoritative and which are claims?
  • Where is authorization enforced?
  • Is runtime isolation in place?
  • Are tools scoped?
  • Is human review in place?
  • Are there evals and a regression dataset?
  • Are traces and retrieval traces captured?
  • Are denied-access events logged?
  • How are changes rolled out and rolled back?
  • How does an incident become a regression case?
  • Is the workflow worth maintaining from a value perspective?

Minimum production bar

A serious production agentic workflow needs:

  • agent contract;
  • workflow design;
  • typed inputs/outputs;
  • scoped tools;
  • least-privilege identity;
  • trusted-input/provenance model;
  • data-plane authorization for private data;
  • runtime isolation for autonomous execution;
  • evals;
  • traces;
  • retrieval traces;
  • human review for risky actions;
  • side-effect lifecycle;
  • rollback/canary process;
  • incident runbooks.

Most dangerous anti-patterns

  • One large autonomous agent instead of a workflow.
  • Prompt policy instead of enforced policy.
  • Viewer agent with root database credential.
  • Vector metadata filter as the only tenant isolation.
  • Model summary as authoritative fact.
  • Direct mount workspace for autonomous coding agent.
  • Host Docker socket inside agent runtime.
  • Local privileged MCP server without tool-level authorization.
  • Side effect without approval binding, readback, and idempotency.
  • Production failure without regression case.

Product-learning workflows need the same discipline. Preserve whether evidence is a human report, observed event, executed test, simulation, or hypothesis; synthetic feedback must not become confirmed demand. Protect metric definitions and distinguish human activity from automated and test events. Include negative tests where a counter improves without customer benefit. Keep initiative approval separate from task execution: a completed task does not authorize a new investment, and an issue label does not enforce permission. After release, verify the intended business effect rather than inferring it from passing technical checks.

Bottom line

Engineers are not working on “AI magic.” They are working on a new type of production automation.

They will:

  • describe processes as workflows;
  • define bounded agent steps;
  • give agents scoped tools;
  • isolate autonomous execution;
  • build durable memory as governed data;
  • enforce authorization in the data plane;
  • build human review;
  • create evals;
  • enable traces and retrieval traces;
  • roll out changes through baseline/candidate/canary;
  • turn failures into regression tests.

The main leadership focus:

Judge production readiness by demonstrated controls and useful outcomes, not by a convincing demo.

Sources and further reading

This summary presents the handbook’s engineering synthesis, not an external standard or certification. Its proposed conventions and examples require implementation-specific validation. The handbook’s chapter 23 provides the broader source map; principal references include Anthropic’s workflow patterns, OWASP’s AI Agent Security guidance, and W3C PROV for provenance concepts. Those sources do not endorse or certify this summary.