AI Agents in Production: What Breaks and What Doesn’t

The demo problem

AI agents are extraordinary in demos. You give them a goal, they call tools, retrieve information, make decisions, and complete the task. It looks like magic. Then you put them in production and discover they’re fragile in ways that are hard to anticipate and expensive to debug.

The core problem: demos are run on happy paths with cooperative environments. Production has unreliable APIs, ambiguous inputs, edge cases the prompt didn’t anticipate, and a long tail of situations the agent doesn’t handle gracefully.

What actually breaks

Tool call failures cascade

An agent calls Tool A, gets an error, and instead of stopping or retrying gracefully, it hallucinates a response as if the tool had succeeded and continues. Every subsequent step is now built on a false premise. By step 7, the agent is confidently doing something completely wrong, and the error is extremely hard to trace back to step 2.

def safe_tool_call(tool_fn, *args, max_retries=2, **kwargs):
    """Wrap tool calls with error handling and retry logic."""
    for attempt in range(max_retries + 1):
        try:
            result = tool_fn(*args, **kwargs)
            return {"success": True, "result": result}
        except Exception as e:
            if attempt == max_retries:
                # Return structured error — never let the agent guess
                return {
                    "success": False, 
                    "error": str(e),
                    "tool": tool_fn.__name__,
                    "instruction": "Do not proceed. Report this error to the user."
                }
            time.sleep(2 ** attempt)  # exponential backoff

Context window exhaustion

Long-running agents accumulate tool outputs, intermediate reasoning, and conversation history until they hit the context limit. At that point, early context falls out of the window and the agent “forgets” decisions it made earlier — leading to inconsistent behaviour or outright contradictions.

The fix: external state management. Keep a structured task state object that summarises what’s been decided, what’s been done, and what remains. Pass a compact summary, not the full history.

class AgentTaskState:
    def __init__(self, goal: str):
        self.goal = goal
        self.completed_steps = []
        self.decisions = {}  # key decisions made
        self.pending_steps = []
        self.errors = []
    
    def to_context_summary(self) -> str:
        """Compact summary for LLM context — not the full history."""
        return f"""Goal: {self.goal}
Completed: {'; '.join(self.completed_steps[-5:])}
Key decisions: {json.dumps(self.decisions)}
Next: {self.pending_steps[0] if self.pending_steps else 'determine next step'}
Errors encountered: {len(self.errors)}"""

Ambiguous task decomposition

Give an agent a vague goal and it will decompose it differently every time — sometimes well, sometimes badly, and you won’t be able to predict which. The solution is explicit task decomposition as a first step, with the decomposition shown to a human for approval before execution begins.

Human-in-the-loop: where and why

The instinct is to make agents fully autonomous. The right approach — at least for production systems where mistakes are costly — is strategic human checkpoints:

• Before irreversible actions — anything that writes to a database, sends a message, makes an external API call with side effects
• After task decomposition — show the plan before executing it
• On uncertainty — when the agent’s confidence falls below a threshold, escalate rather than guess
• After N steps — periodic check-in for long-running tasks to confirm direction is still correct

What actually works in production

The agent patterns that hold up in production are narrower in scope than the demos suggest:

• Single-domain agents with 3–5 well-defined tools, not general-purpose agents with 20+ tools
• Structured outputs at every step — never free-form text as intermediate state
• Idempotent tools — tools that can be called twice safely, to enable retry without side effects
• Observable pipelines — every tool call logged with inputs, outputs, and timing
• Graceful degradation — when something goes wrong, the agent reports clearly what it got done and what it didn’t, rather than pretending to have succeeded

The hardest part of production AI agents is not the LLM. It’s building the surrounding system — error handling, state management, observability, and the human interfaces — that makes the LLM’s output trustworthy.