GuardianAgentBench: Where Agents Fail and How to Guard Them
A new benchmark of 580 test scenarios across six domains evaluating safety and reliability of LLM agents built on LangChain, LlamaIndex, and Vectara with six frontier models. Even the strongest configuration only reaches 74.8% overall accuracy; stronger models tend to under-use available tools while weaker ones misuse them. Runtime guardrails recover 19.9% of failures at a 0.5% false-positive rate, outperforming system-prompt-based defenses.
Why it matters
Quantifies how far current agent frameworks are from reliable, safe operation and shows structural runtime guardrails beat prompting alone — directly relevant context alongside this week's OpenAI/Hugging Face sandbox-escape incident.
Importance: 2/5
Notable new agent-safety benchmark, single official source.