#evaluation
- EVA-Bench: End-to-End Framework for Evaluating Voice Agents ServiceNow AI research
- ARC Prize shows GPT-6 Astra's 99.9% ARC-AGI-3 score came from OpenAI's custom harness, not the model OpenAI research
- SOOHAK: Frontier LLMs Solve Hard Math But Fail to Recognize Unsolvable Problems research
- OpenAI Publishes Deployment Simulation: Predicting Model Behavior Before Release OpenAI research
- GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation GigaAI research
- Long-Horizon-Terminal-Bench: Testing Agent Limits on Long-Horizon Terminal Tasks Tencent Hunyuan research
- VGI-Bench: Probing Visual Intelligence in Video Generation Models research
- Specific releases Real-SWE benchmark built from private enterprise codebases Specific tools
- DataFlex-RL: evaluation platform finds RLVR data policies don't beat uniform sampling Peking University research
- LLM-as-a-Verifier: verification as an independent scaling axis for LLMs Stanford University / UC Berkeley / NVIDIA research
- ClawProBench: Trace-Aware Evaluation of AI Agents with Runtime Coverage research
- Repo-To-Skill: distilling GitHub repositories into reusable AI skills research
- Random Attention: rethinking KV cache eviction for efficient reasoning Salesforce AI Research research
- LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence Stable AI research
- Judge Circuits: Mechanistic Explanation of LLM-as-Judge Format Inconsistency research
- EvoArena: LLM Agents Score Only 40% on Dynamic Evolving Environments MIT / NUS / Salesforce research
- WeaveBench: Computer-Use Agents Fail at Hybrid GUI+CLI Tasks — 41% Pass Rate Microsoft Research research
- PerceptionRubrics: Atomic Rubric Evaluation Reveals 8% Perception Gap Between Open and Closed Models research
- AutomationBench-AA: 657-task independent benchmark for AI agent SaaS automation Artificial Analysis tools
- OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models NLP Group of The University of Hong Kong research
- Cline open-sources Terminal-Bench-based evals for open-weight coding agents Cline tools
- StudentSim: Training LLM-based Student Simulators Microsoft Research research
- Legibility is not interpretability: judged vs. actual step importance in chain-of-thought research
- AutoResearchBench — a benchmark for autonomous scientific literature search by AI agents BAAI research
- EvoArena: LLM Agents Score Only 39.6% on Dynamic Evolving Environments Benchmark MIT research
- FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines Cisco Foundation AI research
- AgenticSTS: Bounded-Memory Testbed for Long-Horizon LLM Agents Alaya Studio research
- EvoPolicyGym: Evaluating Iterative RL Policy Self-Improvement by Coding Agents University of Macau / CUHK research
- SWE-Together: Multi-Turn Benchmark for Coding Agent Evaluation research
- Ideas Have Genomes: Frontier LLMs Score Only 27% on Scientific Lineage Reasoning Shanghai Jiao Tong University research
- K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs research
- How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review research
- HarnessEval-W: Agentifying the Evaluation of Visual Worlds NTU / MirroS-Lab consortium research
- PAWBench: How Far Are We from Probabilistically Aligned World Modeling? research
- Benchmark Radar: a living database and search engine for AI benchmarks Carnegie Mellon University research
- Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agent Loops research
- Formalizing Latent Thoughts: Axiomatic Framework for Evaluating LLM Reasoning Representations University of British Columbia research
- Will Scaling Improve Social Simulation with LLMs? A Study of 85 Models Stanford / Columbia / Tsinghua research
- UniClawBench: Benchmark for Proactive AI Agents on Real-World Tasks University of Hong Kong research
- OSReward benchmarks cross-platform computer-use reward models University of Hong Kong research
- IBM Research benchmarks how much agentic memory each model tier actually needs IBM Research research
- FrontierChallenge: Evaluating Scientific Workflow Completion HKU + Apodex AI research
- xAI publishes LatchBio biosecurity evaluation of Grok 4.6 xAI research
- LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering research
- Last Translation Benchmark: human-authored tasks that still break frontier translation models research
- RealSWE: coding-agent benchmark rebuilt around real, casual user requests research
- Enoki: multi-level hallucination detection from one open IE pipeline Skoltech research
- An Empirical Study of Harness Design for Coding Agents Zoom research