I gave an agent a research task at 11 PM. Checked it at 7 AM. It had used 40,000 tokens. The output was three paragraphs — all technically correct, all useless.
What happened: the agent called the same search tool 23 times because the results kept not quite satisfying its termination criteria. It didn’t fail. It just optimized the wrong thing, indefinitely, until I killed it.
That’s the gap nobody tells you about. Between demos and production. Between “it works” and “it works at scale, under load, on bad days, with real data.”
Start with what an agent actually is, because the word has been stretched past useful.
An agent is an AI system that takes actions based on its own reasoning, receives feedback from those actions, and uses that feedback to decide what to do next. The loop is what makes it an agent. A single model call isn’t an agent. A model that calls a tool, reads the result, decides whether to call another tool, and continues until a task is done — that’s an agent.
Three properties matter: autonomy (decisions without human input per step), tool use (can interact with external systems), and multi-step reasoning (can break goals into steps and adjust based on results). All three are required. Two out of three is something else.
The core of any agent system is the same regardless of framework or model.
The model reasons. It sees a context window containing the task, available tools, history, and prior results. It generates either a response (task complete) or a tool call (more work needed). Frontier model choice — Claude Opus 4.6, GPT-5, Gemini 2.5 Pro — mostly affects latency and cost for most production tasks, not capability ceiling.
The tools act. Tools are functions the model calls by generating structured output — tool name plus validated parameters — that the runtime intercepts and executes in the application layer, not inside the model. Results come back as additional context.
Tool design is the highest-leverage work in agent systems. Not model selection. Tool design. A model with average reasoning and excellent tools beats a model with excellent reasoning and poorly designed tools. The description a tool shows the model — what it does, when to call it, what it returns, what can go wrong — determines how reliably it gets used correctly. Most teams spend their time on model selection and ignore this.
The memory layer has four forms. In-context: whatever fits in the current window, fast and temporary. External: databases the agent queries via tool calls, for knowledge larger than any context window. Episodic: records of past sessions — this is what Claude Projects uses, carrying facts about you across conversations via the system prompt. Semantic: distilled facts extracted from experience, more research than production today. Most production systems use in-context plus external. Episodic and semantic are where the interesting architecture evolution is happening.
The orchestration layer runs the loop: receive task → generate action → execute action → receive result → decide next action → repeat until done. In simple agents, five lines of code. In multi-agent systems, it handles task delegation, result aggregation, error recovery, timeout management, and parallel execution coordination.
Most teams underinvest in orchestration and overinvest in model selection. This is backwards.
The architectures that matter in production:
ReAct (Reason + Act)
The foundation most production agents build on. The model alternates between reasoning steps and action steps — writes out its thinking before deciding what to do. Transparent, debuggable, reliable. The limitation: sequential. Each tool call waits for the previous one. For tasks requiring many independent calls, this stacks up.
Plan-and-Execute
Separates planning from execution. A planner model generates the full step sequence first; executor models work through it. Advantage: the planner reasons about the whole task before any action begins. Disadvantage: if the plan is wrong, execution faithfully completes all the wrong steps.
The systems that actually work add a feedback loop — executor reports back after each step, planner revises. Without that, Plan-and-Execute is just confident mistakes at scale.
Multi-agent
An orchestrator delegates sub-tasks to specialized subagents, each optimized for a domain. Benefits: parallelism, specialization, scale. Challenges: coordination overhead, error propagation, debugging complexity. A failing subagent in a multi-agent system is harder to trace than a failing single agent.
I’ll be direct: most teams shouldn’t start with multi-agent. It’s the right architecture for genuinely distributed problems. For most agent work, a well-built single ReAct agent with good tools outperforms a poorly coordinated multi-agent system.
Here’s what actually breaks in production, as opposed to what people expect to break.
Context window overflow. Long-running agents generate more tool results and history than any context window holds. Systems that don’t deliberately manage context — compressing old history, summarizing completed sub-tasks, retrieving rather than retaining large data — fail at scale. The agent starts losing critical context mid-task. My 40,000-token agent wasn’t broken. It just didn’t know when to stop retaining.
Tool error accumulation. Individual tools fail. APIs return errors, rate limits hit, databases have connection issues. Agents without explicit error handling — retry logic, graceful degradation, fallback tools — enter error loops: call the failing tool again, get the same error, call it again. This is the most common production failure mode and the easiest to prevent. Handle tool errors explicitly. Don’t let the model figure out what to do on its own.
Reasoning drift. On very long tasks, models lose track of the original goal. They start optimizing for intermediate sub-goals rather than the actual objective. Periodic re-anchoring — explicitly restating the original task every N steps — reduces this significantly. Most frameworks don’t do this automatically.
Underspecified termination. Agents without explicit, testable criteria for “done” either terminate too early or loop indefinitely. The termination condition is part of the task specification, not an afterthought. If you’re vague about what done means, the agent will be vague about it too. That’s what happened at 11 PM.
The 88% failure rate on production agent deployments isn’t because the models aren’t good enough. It’s because teams treat the model as the product and treat everything else as implementation detail.
The tooling has consolidated. Frontier model providers: Anthropic (Claude Opus 4.6 / Sonnet 4.6), OpenAI (GPT-5 / GPT-4o mini), Google (Gemini 2.5 Pro). All support tool calling with JSON Schema validation.
Tool integration: MCP is the standard. An agent framework that supports MCP can use any of the 6,400+ public MCP servers without custom integration code. If you’re writing custom API wrappers for tools in 2026, check whether an MCP server already exists before you start.
Memory infrastructure: vector databases (Pinecone, Weaviate, Chroma) for semantic search over large knowledge bases. Redis for fast key-value retrieval. PostgreSQL for structured episodic memory. Most production systems combine at least two.
Orchestration frameworks: LangGraph, CrewAI, AutoGen in Python. The Anthropic Agent SDK (released Q4 2025) for Claude-native builds. Frameworks have largely converged — pick based on team familiarity and community size, not technical differentiators.
Observability: LangSmith, Langfuse, Helicone. Non-negotiable. Without trace-level visibility into agent runs, debugging production failures is guesswork. Build observability before you build capabilities. This is the rule most teams learn the hard way.
The technology is ready. Agents are in production, at scale, at companies that don’t announce it. Code review, research synthesis, document processing, customer support triage — these aren’t demos.
What’s not ready, at most companies, is the operational discipline. Agents with broad mandates and loose guardrails produce confident mistakes at scale. The combination of autonomy and tool access means a failing agent can do damage — send wrong emails, delete the wrong files, make API calls that cost money.
The teams building effective systems aren’t the ones with the most ambitious architectures. They’re the ones who constrain scope deliberately, handle errors explicitly, and instrument everything before deploying.
The honest version: most teams need better orchestration, better tool design, and better observability. Not a bigger model.
If you’re building agents in production — or trying to — Deep Stack covers the architectural decisions that separate systems that work from systems that look like they work. The spec, the failure modes, the tradeoffs that show up in week three when the simple version breaks.
Also on EchoNerve: MCP vs Traditional APIs — Frontier Model Comparison 2026 — Prompt Engineering Playbook 2026