How to Deploy AI Agents to Production: From Demo to Recoverable, Observable, Rollback-Ready Deployment

Launch Roadmap: Define Boundaries, Then Validate Each Requirement

The key to deploying an AI agent to production is making task failures diagnosable and recoverable, and changes verifiable and reversible. Adding models or orchestration complexity is secondary. Follow this sequence: define tasks and completion criteria, choose a deployment model, set up the runtime and state storage, enforce security boundaries, evaluate representative tasks, then roll out to a small share of traffic with on-call coverage and rollback plans in place.

First, define triggers, inputs, permitted tools, final business outcomes, timeout handling, and human approval checkpoints. If a fixed workflow already meets your needs, there is no reason to introduce multiple agents just to make it “agentic.” For guidance on choosing between workflows and agents, see Anthropic's discussion of approaches to building agents. For details on connecting system interfaces and managing tool permissions, see How to Integrate AI Agents with Existing Enterprise Systems.

Choose a Deployment Model: Weigh Your Constraints

The table below compares architectural constraints; it does not establish performance or pricing conclusions. Verify each candidate platform's capabilities, quotas, and operational responsibilities against its current documentation.

When using inference through a model API, deployment typically covers the application runtime, controlled tools, and state services. Whether to self-host inference should be evaluated separately based on latency, data boundaries, model availability, and operational requirements. The Kubernetes Deployment documentation provides a reference for rolling updates and revision rollbacks, but this does not mean an agent must be deployed on Kubernetes.

Minimal Production Architecture: Connecting Requests, Execution, State, and Observability

调用方 → 鉴权入口/请求校验 → 任务执行器（同步短任务或异步任务 ID）
      → Agent 运行逻辑 → 模型端点/受控工具 → 状态存储与结果核验
      → 日志、指标、追踪与告警

For short tasks, the caller can wait for the result. For tasks that exceed the caller's wait window or require recovery across processes, return a task ID so the client can check status or receive a completion notification. Define task states, cancellation semantics, timeout boundaries, and permissions for accessing results.

System integration is covered in the earlier article How to Integrate AI Agents with Existing Enterprise Systems and is not repeated here. For on-call responsibilities and engineering collaboration, see FDE Engineering Delivery Responsibilities and Collaboration Boundaries. Specific responsibilities must be defined within the project.

Preparing Release Artifacts: Making Changes Reproducible

Include code, dependency lockfiles, runtime images, environment configuration, prompt templates, and model and tool configuration versions in the change record. Validate dependencies during the build; keep development, testing, and production environments separate; and inject credentials through controlled configuration mechanisms rather than storing them in code repositories or ordinary logs. Manage model API keys, tool service credentials, and business identities separately.

The release checklist should confirm at least the following:

• The application entry point, health checks, resource limits, and shutdown signal handling have been verified;
• Dependency versions, image digests, and configuration changes are traceable;
• Prompts, tool contracts, model identifiers, and key parameters are versioned;
• Regression test cases and evaluation results can be linked to the release version;
• The target rollback version can start and either read the current state format or explicitly refuse to process it.

Recovering Long-Running Tasks: Checkpoints Do Not Guarantee Exactly-Once Business Operations

Recoverability spans three layers: task records, execution state, and business side effects. These capture whether a task is queued, running, or waiting for human input; which execution steps have been reached; and whether writes to external systems succeeded. Saving conversation text alone may not be enough to restore the execution context, and checkpoints alone cannot establish the outcome of external write operations.

For example, the LangGraph documentation distinguishes between a checkpointer, which saves execution state by thread, and a store, which shares data across threads. Recovery after a process restart requires a suitable persistent backend and verification of writes, reads, isolation, retention, and failure recovery; an in-memory saver cannot serve as the basis for persistence across restarts. See the LangGraph Persistence documentation.

Retry policies should distinguish between per-attempt timeouts, retryable errors, and handling after retries are exhausted. Tools with side effects should use idempotency keys or deduplication mechanisms, and check the business outcome before retrying: a call timing out does not mean a write did not occur. If the outcome cannot be confirmed, escalate for manual verification or run a compensating workflow; checkpoints, queues, and retries do not, by themselves, provide an end-to-end exactly-once guarantee. See the LangGraph fault tolerance documentation.

Security Controls: Define Conditions That Block High-Risk Actions

Before release, verify at least the following:

• External documents, web pages, and user input cannot expand tool permissions on their own;
• Tool services validate identity, resource scope, and parameters on every call, and high-impact actions require explicit human confirmation;
• Credentials are isolated by environment and can be rotated, while logging is minimized according to data sensitivity, sensitive data is redacted, and retention periods are defined;
• Approval records link the request, the operator, the proposed action, and the final business outcome;
• Tests for unauthorized access, prompt injection, duplicate submissions, and sensitive data exfiltration all produce the expected rejection outcomes.

The OWASP Agentic AI Threats and Mitigations resource can help build a threat inventory, but does not constitute proof that this system is secure or compliant.

Evaluation Gates: Assess Both Outcomes and Execution

Do not accept an agent's claim that a task is “complete” as proof of success. Define success criteria for each task type that can be verified in the environment, such as the state of target records, output formats, and prohibited actions; retain execution traces to pinpoint failures in planning, tool calls, or result validation. See Anthropic's guide to evaluating AI agents.

Test cases should cover normal inputs, missing fields, conflicting information, tool timeouts, permission denials, duplicate requests, and attempts to steer the agent through external content. Important test cases can be run multiple times to record variation in outcomes. Release gates can cover task outcomes, prohibited actions, exception handling, recovery drills, and human takeover paths; set thresholds based on business risk, existing baselines, and test results.

Monitoring: Quality, Latency, Recovery, and Cost

For each metric, define how it is measured, where it is collected, alert conditions, and who is responsible:

Traces help pinpoint deviations during multistep execution, while the final business state confirms task completion. Determine which log fields require redaction, who can access logs, and how long logs are retained for each project. Observe baseline behavior during the initial rollout, then adjust alert thresholds based on business risk.

Staged Rollout, Rollback, and In-Flight Task Handling

Before release, run regression tests on the combination of code, prompts, model configurations, and tool contracts. Validate with limited traffic or low-risk tasks first, then gradually increase traffic. Set stop conditions, such as unexpected business outcomes, failed authorization boundary tests, or errors and latency exceeding team-defined limits.

During an incident, pause new task intake or disable high-risk tools. Take stock of running, waiting, and completed tasks, verify external side effects, then roll back to a validated version. Verify that the previous version is compatible with the current state format. Route tasks that cannot recover automatically to a human, or perform approved compensating actions. Kubernetes Deployment can serve as a release reference when using Kubernetes; rolling back the application does not automatically undo writes already made to business systems.

Training Exercise: Validate Recovery Paths with Fault Injection

The following is a training exercise design, not a customer case study or measured results. Define the task as “generate a report, then write the results after human confirmation.” For each run, record the expected outcome, actual outcome, and evidence:

The exercise should verify that the business state and execution records match expectations, and clearly identify the owner and handling procedure for tasks that encounter exceptions.

Production Readiness Checklist

• Task scope, completion criteria, prohibited actions, and human confirmation points are documented.
• The deployment approach matches task duration, recovery requirements, and the team's operational capabilities.
• Dependencies, images, prompts, model configurations, and tool contracts are traceable.
• Task state, persistence, and business side effects can each be verified independently, and retries have idempotency protection.
• Permissions, prompt injection defenses, sensitive log handling, and human approvals have passed testing.
• Evaluations cover representative tasks, exceptional cases, outcome verification, and recovery scenarios.
• Metric definitions, alert owners, staged rollout stop conditions, and on-call procedures are confirmed.
• Rollback versions, in-flight task handling, state compatibility, and business compensation procedures have been rehearsed.

Common Deployment Questions

Q: Is Kubernetes required? A: No. If your team already has a platform that meets requirements for task duration, state recovery, network isolation, and operations, there is no need to add cluster complexity just for agents.

Q: Does deploying an agent require self-hosting a model? A: No. When using a model API for inference, you generally only need to deploy the application runtime, tools with controlled access, and state management components. Evaluate self-hosted inference separately based on data boundaries, latency, availability, and operational requirements.

Q: Will tasks be lost after a service restart? A: That depends on task records, execution state persistence, and validation of the recovery process. Process memory alone cannot support recovery across restarts. You also need to verify whether external writes succeeded to avoid duplicate execution.

Q: How can retries be prevented from causing duplicate writes? A: Design idempotency keys or deduplication records for business operations, and check the external system's state before retrying. If the outcome cannot be confirmed, pause automatic retries and route the task for manual verification or through a compensation workflow.

Q: What success-rate threshold should be required for launch? A: There is no universal value. Set release gates and alerts based on task risk, business baselines, and test results. Acceptance testing should also cover prohibited actions, human takeover, recovery, and business outcomes.