Rethinking Agent Harnesses, Live Software, and Token Economics

Geoff Huntley joins Armin Ronacher and Ben Vinegar to question assumptions inherited from early coding agents: a single conversation loop, file edits followed by deployment, and tool calls whose purpose a user can easily inspect. Their examples include durable agent execution, an LLM-connected Lisp runtime, and experiments whose apparent affordability depends on subscription pricing.

This summary uses English auto-generated captions, not an audio review. The demonstrations and engineering experiences are the speakers' accounts; their labor-market and provider-economics predictions are not established findings.

Key points

• Learning agent internals is a career recommendation, not a guarantee against displacement. Geoff urges developers to understand inference, prompt caching, and how to build a coding agent, rather than identify primarily with a language or editor. Ben emphasizes that family responsibilities and limited time make rapid retraining difficult; Armin questions whether software engineers' historically scarce position will persist. Their disagreement matters: curiosity is actionable advice, but the conversation does not establish that job loss is simply an individual's failure to adapt. 04:40–09:38(opens in a new tab), 11:29–17:08(opens in a new tab)
• Concurrent work strains the single-transcript model of an agent session. Armin describes spending four months on roughly 10,000 lines of Pi Durable, despite agents producing plausible solutions quickly. The intended behavior is to stop and restart a harness without losing its place, including network interactions and communicating subagents. Background compaction and several people talking to one agent introduce simultaneous work that fits poorly into the original request/tool-call/tool-result loop. At recording time, he says the new structure was not yet used in the Pi coding agent because it would break extensions. 34:55–40:04(opens in a new tab)
• Revisit instructions and harness assumptions as models change. Geoff provocatively proposes deleting and rebuilding AGENTS.md instructions and skills after model releases or every few months. His underlying concern is that yesterday's workarounds become unquestioned best practices. He distinguishes interface improvements such as steering messages from deeper changes to how the execution loop works. This is a proposal to reconsider accumulated assumptions, not evidence that every existing instruction is obsolete. 40:04–42:58(opens in a new tab)
• A live runtime offers a different place for an agent to change software. Geoff describes a Lisp demonstration in which the model inspects defined functions, defines new ones, executes expressions, and rolls back to an earlier generation of the running environment. Instead of editing files and then compiling or deploying them, the agent modifies the program through the runtime itself. Armin connects this to Lisp's ability to represent functions similarly to data and serialize more of its state than typical languages. They also discuss an experimental web app that exposes modification through its own interface. These are demonstrations of an alternative development model, not proof that deployment, verification, or safe multi-user operation have been solved. 44:43–53:42(opens in a new tab), 55:35–58:34(opens in a new tab)
• Ad hoc code makes agent activity harder to explain and control. Armin reports that newer models often write Python or JavaScript instead of using a supplied edit tool. He distinguishes shell execution near the project files from code mode running inside the harness loop. A known edit tool lets the interface show a recognizable operation; arbitrary, compact code is harder to label accurately. He worries that hiding this output weakens users' ability to verify critical work along the way. A second model that classifies or narrates actions is discussed as a possible response, with additional cost, rather than a complete solution. 59:37–1:04:54(opens in a new tab), 1:06:49–1:11:51(opens in a new tab)
• Subscription prices can conceal the cost of an experiment. Armin contrasts a roughly $20 CarPlay adapter with what he describes as $15,000 worth of tokens used to modify it, while stressing that subscriptions made the project affordable to him. Ben makes a similar comparison for agent-assisted 3D modeling: a cheap printed part may require expensive inference and repeated corrections. The point is the difference between the user's subscription bill and equivalent token pricing, not that they paid those API amounts or measured the providers' actual costs. 1:17:30–1:19:26(opens in a new tab), 1:23:27–1:26:00(opens in a new tab)
• The ability to customize software does not establish demand for constant customization. Ben challenges the assumption that ordinary users want to keep rebuilding their applications. Geoff argues that people in businesses want changes without switching tools or consulting someone else; Ben counters that the desired change may be occasional. His example is spending about an hour customizing a system for his family and then being satisfied. This limits the product claim behind the live-programming demo: reducing friction can be useful without making software modification everyone's continuous activity. 1:31:38–1:34:23(opens in a new tab)

Practical interpretation

Editorial synthesis: evaluate a new agent architecture by the behavior it must preserve, such as resumable work, concurrent conversations, and inspectable actions. Treat live modification as a promising interaction model that still needs verification, and distinguish subscription affordability from sustainable operating cost before turning an experiment into a product.

Full video: State of Agentic Coding #11 with Geoff, Armin, and Ben(opens in a new tab).