with-agents

Tobi Lütke – Building Shopify and the Future of AI | Ep. 50

AI adoption metrics, small teams, customer feedback, and experienced judgment

Tobi Lütke describes Shopify's AI mandate, why a token-usage leaderboard produced bad incentives, and how agent-summarized customer feedback supports smaller teams. He argues that experienced engineers still steer coding agents through problem understanding. Shopify's adoption, cost, and workflow outcomes are speaker-reported, not independently measured.

Uncapped with Jack Altman

Shopify CEO Tobi Lütke discusses what changed after the company made AI use an explicit expectation. The strongest material concerns organizational design: measure impact rather than a usage proxy, route customer evidence into small teams, and retain experienced judgment even when agents perform more of the implementation.

Evidence basis: This summary uses the saved English auto-generated captions, captured September 19, 2026, without an original-audio check. Shopify policies, costs, adoption, and outcomes are Lütke's account, not independently verified. Forecasts reflect the recording date.

Adoption metrics can become the wrong objective

  • The policy targeted net impact, not tool use alone. Lütke says Shopify made AI fluency an explicit performance expectation and supplied broad access, including an unlimited-token policy, because tool adoption had begun to separate employees who previously produced similar results. This is his rationale for the mandate, not evidence of a measured productivity multiplier. 18:39–19:42(opens in a new tab)
  • A token leaderboard distorted behavior. Shopify briefly ranked token consumption, then removed the leaderboard after undesirable effects. Usage remains visible on employee profiles by organizational percentile because the company tracks and budgets it, but Lütke does not equate that number with impact. This is a concrete example of a proxy becoming a target. 19:42–20:43(opens in a new tab)

Use agents to shrink coordination overhead

Lütke prefers teams of three to five people, but says specialist work often forced them to grow. For example, Shopify put support staff on teams and routed customer tickets to them. Agents now deliver feedback summaries to teams, which he says helps them stay small. The interview does not explain how summary quality is checked, which customer feedback is covered, or how teams can detect omissions. 23:37–24:39(opens in a new tab)

He also says Shopify's six-week project-review cycle had become too slow for AI-assisted work. The underlying point is to revisit cadence when implementation speed changes; it does not establish that every team should shorten its cycle or that review capacity increased at the same rate. 25:26–26:22(opens in a new tab)

Expertise still steers the work

  • Coding-friendly tools can help models with non-coding tasks. Lütke says agent systems can handle some non-coding work better when information, including memory, is stored in files that models can inspect with programming tools. The approach draws on their existing coding strengths. He expects it to become less necessary as models improve in other domains. 45:42–46:44(opens in a new tab)
  • Generated code does not remove the need to understand the problem. As coding harnesses improved, Lütke revised his initial view that AI-native newcomers might gain a lasting advantage. Experienced programmers recognize wrong directions because they understand the problem and have seen similar failure modes; small steering interventions can change the result. His conclusion is that adoption speed varies, but underlying capability remains important. 48:52–50:46(opens in a new tab)

What still holds in September 2026

The mandate itself is now less informative: by September 2026, the catalog contains many examples where agent use is routine. The durable lessons are the corrections around it. Consumption is not outcome, smaller teams still need inspectable customer evidence, and faster generation can merely transfer effort into review. That aligns with later evidence in Measuring the Impact of AI on Software Engineering and A Love Letter to Pi, which evaluate total workflow impact and make human review evidence explicit.

Editorial implication: track adoption only as an input. Pair it with useful, reviewable outcomes; inspect what summarization systems omit; and change delivery cadence only after checking whether product judgment, verification, and customer learning can keep up.

Search

Search field guides and resources across the site.

    Keyboard shortcuts

    Press ? to open this guide (usually Shift + /). Close it before using other site shortcuts.

    Show keyboard shortcuts
    ?
    Search site
    Ctrl + K
    Close the current dialog or menu
    Esc

    Site shortcuts are ignored while typing. Browser and extension shortcuts may take priority.