OpenAI's Astra for Law tops legal reasoning benchmark with efficiency gains
GPT-6.2 Astra configured for law achieves 20.8% full-credit pass rate on Harvey's LAB-AA, outpacing competitors while using less compute per task.
GPT-6.2 Astra configured for law achieves 20.8% full-credit pass rate on Harvey's LAB-AA, outpacing competitors while using less compute per task.
OpenAI releases GPT-6.2 Astra for complex work and GPT-6.2 Sol for everyday tasks, cutting input costs 20% while advancing on scientific and professional benchmarks.
Allen AI releases XYBench, an 8K-question benchmark measuring how well AI models recognize user misconceptions and redirect toward actual solutions.
Gemini 4 Carbon is now available to developers and enterprises, positioned as Google's most capable model for agentic workflows, scientific research, and cybersecurity defense.
OpenAI releases LifeVal, an evaluation framework measuring how well AI models handle real-world personal assistance tasks like childcare, finances, and government services.
Google's latest frontier model claims top spots on vulnerability-detection benchmarks and launches exclusively to defense contractors via its Fairwind Program.
Gemini 4 Carbon ranks #1 on Artificial Analysis Intelligence Index v5.0 and leads on vulnerability detection, exploitation, and patching tasks.
Microsoft reports its multi-agent AI diagnostic tool achieved 83% accuracy on medical case reports versus 36% for human doctors, but clinical validation remains pending.
A new benchmark evaluates 21 models against 7,980 authentic code reviews, revealing performance gaps between open and closed-weights systems.
GPT-6 Astra achieves 87.4% accuracy on multi-language database tasks, up from 26.8% for GPT-5.6 Sol, while COBOL migration accuracy reaches 97.9% at higher reasoning effort.
A developer's DIY evaluation tool reveals gaps in off-the-shelf benchmarks for specialized use cases.
IBM Research and Hugging Face introduce ScarfBench, a benchmark evaluating AI agents on framework migration tasks that require working, deployable code—not just syntactic translation.
OpenAI released GeneBench-Pro, a benchmark of 10 case studies designed to evaluate AI models on practical genomics reasoning tasks.
OpenAI introduces GeneBench-Pro, a 129-problem benchmark measuring whether AI models can make higher-order scientific judgments in genomics and translational medicine.
Researchers use a classic mystery game to test how well AI agents reason through multi-step deduction and suspect elimination.
OpenAI released LifeSciBench, a benchmark designed to measure whether AI systems can handle real-world life science research workflows, not just answer biology questions.
A new benchmark reveals that even the most capable AI systems struggle with diagnosing complex infrastructure failures, scoring below 50% on Site Reliability Engineering scenarios.
A Reuters review of federal AI procurement records found Grok appeared in only 3 of 400+ documented government uses, compared to 230+ for OpenAI and dozens for Anthropic and Google.
Google unveiled Gemini 3.5 Flash at I/O 2026 as an agent-first model claiming frontier intelligence at sub-flagship latency, with Gemini Omni adding physics-aware video generation.
A new benchmarking framework evaluates complete AI agent systems—not just models—across six diverse tasks, reporting both quality and cost metrics for practical deployment decisions.
A GitHub repository curates datasets for LLM fine-tuning, instruction tuning, and benchmarking across medical, NLP, multimodal, and code domains.