Production agent reliability: an AI Engineer guided collection

Running agents in production requires more than checking one generated change: someone must own state, retries, recovery, and the consequences of failure. This editorial selection of six talks from AI Engineer's Agent Reliability playlist approaches those decisions from different angles, from when automation is worth its cost to whether it improves human control.

Where to start

1. Is this agent worth running when failures and supervision have real costs? Harrison Chase(opens in a new tab) of LangChain/LangGraph frames the choice through success probability, successful-outcome value, failure cost, and operating cost 01:58–02:47(opens in a new tab). Reversible actions and human approval can bound consequential failures, while background execution may increase useful concurrency without implying full autonomy 09:48–17:47(opens in a new tab). The equation is a decision frame, not a supplied measurement method; correlated failures and review capacity still need local evidence.
2. Which decisions belong to models, and which belong to deterministic control? Sam Bhagwat(opens in a new tab) of Mastra.ai treats agents and workflows as composable primitives: agents provide iterative flexibility, while workflows add branching, parallelism, suspension, resumption, and replay 07:16–09:21(opens in a new tab). He recommends adding narrower calls or explicit structure where broad model behavior becomes unreliable 09:21–11:20(opens in a new tab). The talk provides architecture guidance, not comparative reliability measurements.
3. Who owns long-running state, retries, and recovery? Preeti Somal(opens in a new tab) of Temporal presents agentic applications as distributed systems spanning unreliable calls, parallel work, human interaction, and long-lived state 00:55–03:53(opens in a new tab). Her example separates code-defined business workflows from platform-managed execution state, retries, signals, queries, and history 09:54–13:42(opens in a new tab). This is a vendor presentation, and the design introduces a platform and programming-model dependency.
4. How can plans and tools remain bounded and interpretable? Yogendra Miraje(opens in a new tab) of FactSet separates blueprint generation, planning, execution, joining, and bounded replanning 05:18–06:18(opens in a new tab). Natural-language blueprints restrict tool context and reduce planner load, while capability-oriented contracts, validation brakes, and component-level evaluation make failures easier to locate 06:18–11:27(opens in a new tab). The blueprint is interpretable, but the talk does not establish who approves changes before execution.
5. How should reliability span the software lifecycle? Itamar Friedman(opens in a new tab), CEO and co-founder of Qodo, connects context, generation, testing, review, organizational practices, and explicit success criteria into specialist stages 05:23–15:39(opens in a new tab). He demonstrates sequential CLI composition and distinguishes it from future-facing A2A coordination 16:42–20:39(opens in a new tab). Much of the evidence comes from Qodo's product architecture, and Friedman reports that A2A adoption among practitioners he asked was effectively absent.
6. Does the system increase human agency, or only agent autonomy? Danielle Perszyk(opens in a new tab), a cognitive scientist at Amazon AGI, contrasts autonomous-system goals with tools that increase human leverage and warns that automation can reduce agency 04:16–07:28(opens in a new tab). She connects controllable atomic interactions and a shared environment to broader outcomes such as creativity, productivity, strategic thinking, and flow 10:30–19:36(opens in a new tab). Those outcomes are difficult to operationalize and raise unresolved questions about privacy, consent, representation, and metric gaming.

Use the collection as an operating-design review

Editorially, the six talks suggest a compact sequence: justify the expected value, mark deterministic and model-controlled boundaries, assign durable state and recovery ownership, bound plans and tools, evaluate the composed lifecycle, and check whether the result improves human control. Component reliability does not automatically transfer to the system around it.

For the narrower proof required on each generated change, see Make the Agent Prove It. Watch the full Agent Reliability playlist(opens in a new tab) for the two additional talks and the source context behind this route.