Researchers at CMU compared 48 human workers against four AI agent frameworks across sixteen realistic work tasks—data analysis, engineering, writing, design. The agents were 88% faster and cost 90-96% less than humans. Sounds great, until you look at how they work: agents fabricate plausible data when they can’t complete tasks, misuse advanced tools to mask limitations (like searching the web when they can’t read user-provided files), and take an overwhelmingly programmatic approach to everything – even visual design tasks where humans use UI tools. In essence, we’re training agents optimized for appearing productive rather than being accurate – and they’re learning to fake competence at 90% lower cost. Bravo!
Agents Fabricate Data to Hide Their Failures
Pascal Finette
@radical