Ending AI Slop

How can models improve at design and writing when people disagree about what good looks like? At AI Engineer World's Fair 2026, Taste Labs founder Thais Castello Branco proposes separating measurable requirements from subjective preferences. Check the parts that can be verified, and preserve who prefers what for the parts that need human judgment.

Key Points Covered

• Choose the intervention after identifying the problem: Taste Labs works on model evaluation and training with frontier labs. But Castello Branco says some application failures are better addressed by giving an agent the right context or clarifying user intent, rather than changing the model 00:13–02:19(opens in a new tab).
• Capability follows what a domain makes measurable: Castello Branco argues that models perform well on code partly because code decomposes, verifies, and executes. Design and writing are harder to train because good output lacks one equally stable definition 02:19–03:25(opens in a new tab).
• Context and time prevent one universal taste score: A design can fit a startup and fail a finance firm, while accepted styles change over time. Evaluation therefore has to preserve audience, situation, and temporal context 03:25–04:27(opens in a new tab).
• Decompose fuzzy goals before choosing a verifier: Brand adherence becomes more tractable when broken into colors, typography, motion, animation, and texture. The ground truth can grade valid use of those components without requiring the agent to copy the original artifact 04:27–07:24(opens in a new tab).
• Route each component to the method that fits it: Programmatically observable properties can become checks or reinforcement-learning environments; creativity, style, and contextual preference still need human judgment. The goal is to pull only the measurable parts toward verification, not pretend the whole task is objective 06:21–10:29(opens in a new tab).
• The most likely answer is not the best subjective answer: Castello Branco argues that a single optimization stream can collapse creative output toward the mean. Useful variation has to break patterns intentionally rather than reward randomness or one average style 07:24–09:26(opens in a new tab).
• Preserve differences in taste: Averaging ratings can hide distinct, valid preferences. Castello Branco proposes attaching a “preference vector”—a representation of a person's tastes—to their feedback, so training data can preserve those differences. She does not specify how to construct that representation 10:29–12:25(opens in a new tab).
• High-signal feedback is specific and anchored: Expert selection, precise evaluative language, and tying commentary to the exact code component being judged make subjective feedback less noisy and more useful for training or evaluation 11:23–14:26(opens in a new tab).
• Consensus should depend on the attribute: Expert disagreement about alignment may expose defective data because alignment is comparatively objective; disagreement about style or aesthetics can be valid evidence of different preferences 13:22–15:28(opens in a new tab).
• Quality matters more than annotation volume: Castello Branco closes by advocating for high-quality data created by people with deep domain understanding rather than larger collections of noisy judgments 15:28–16:10(opens in a new tab).

Full video: https://www.youtube.com/watch?v=lCBf9slCanI(opens in a new tab)