Arena Leaderboard Reaches $100M Annualized Run-Rate, Eight Months After Commercial Launch
UC Berkeley's crowdsourced AI evaluation platform hits $100M ARR on consumption-based pricing, up from $30M in January 2026.
#llm-evaluation
Sherlock Holmes Board Game Becomes LLM-Agent Evaluation Framework
Researchers use a classic mystery game to test how well AI agents reason through multi-step deduction and suspect elimination.
Noisy LLM Evaluators Prove Effective for Agent Training Despite Imperfection
Research shows that imperfect LLM-based evaluators can still meaningfully improve AI agent performance, challenging the assumption that evaluation noise is prohibitively harmful.