AI Agent and Tool-Use Benchmarks

ACE

watchpartial

2026-07-15

Tests

Agent capability across consumer task areas such as shopping, food, gaming, and DIY.

Best for

Tracking an emerging cross-domain agent benchmark; inspect task coverage before use.

Classification

benchmark · emerging · open

dataset →results →

AgentBench

referencerunnable

2026-07-15

Tests

LLM agent evaluation across multiple interactive environments.

Best for

General agent benchmarking

Classification

agent benchmark · legacy · open

repo →

APEX v1 Extended

specializedrunnable

2026-07-15

Tests

Agent performance on extended economically valuable tasks spanning multiple jobs.

Best for

Testing whether professional-work rankings survive broader task coverage.

Classification

benchmark · emerging · open

dataset →results →

APEX-Agents

specializedrunnable

2026-07-15

Tests

Cross-application completion of long-horizon professional-services tasks.

Best for

Shortlisting agent systems for realistic knowledge-work workflows.

Classification

benchmark · emerging · open

dataset →results →

AstaBench

recommendedrunnable

2026-07-15

Tests

Scientific research-agent suite covering literature understanding, code execution, data analysis, and end-to-end discovery workflows.

Best for

Scientific research agents

Classification

agent benchmark · current · open

source →repo →

Berkeley Function Calling Leaderboard / BFCL

recommendedrunnable

2026-07-15

Tests

Executable function-calling tests including multi-turn and tool-use scenarios.

Best for

Tool calling and API accuracy

Classification

benchmark · current · open

leaderboard →repo →

BrowseComp

recommendedrunnable

2026-07-15

Tests

Browsing-agent benchmark for hard-to-find web answers that require search and synthesis.

Best for

Web browsing agents

Classification

benchmark · current · open

source →repo →

CCTU

specializedrunnable

2026-07-15

Tests

Tool use across 200 cases containing resource, behavior, toolset, and response constraints.

Best for

Finding constraint violations that task-success-only agent evals miss.

Classification

benchmark · emerging · open

repo →paper →

Claw Bench

specializedrunnable

2026-07-15

Tests

Agent products on 314 reproducible tasks across 33 domains and four difficulty levels.

Best for

Regression testing general-purpose agents with inspectable end-state verifiers.

Classification

benchmark · emerging · open

repo →leaderboard →

Claw-Eval

specializedrunnable

2026-07-15

Tests

Practical task completion by tool-using agent systems.

Best for

A current, source-backed signal for open agent frameworks.

Classification

benchmark · emerging · open

dataset →results →

GAIA

recommendedpartial

2026-07-15

Tests

Real-world assistant tasks requiring reasoning, browsing, multimodality, and tool use.

Best for

General agent task solving

Classification

agent benchmark · current · mixed

dataset →leaderboard →

MCP-Atlas

recommendedrunnable

2026-07-15

Tests

Large-scale tool-use benchmark over real MCP servers and multi-call tasks.

Best for

Production-style MCP tool agents

Classification

agent benchmark · current · open

repo →leaderboard →paper →

Mind2Web

recommendedrunnable

2026-07-15

Tests

Web navigation tasks across many real websites and domains.

Best for

Web-agent planning and UI grounding

Classification

benchmark · current · open

repo →

OfficeBench

watchpartial

2026-07-15

Tests

Office-document, email, calendar, and productivity-task automation.

Best for

Knowledge-worker office workflows

Classification

agent benchmark · emerging · mixed

repo →

OSWorld

recommendedrunnable

2026-07-15

Tests

Desktop/web/app tasks with execution scripts in realistic computer environments.

Best for

Computer-use agents

Classification

agent benchmark · current · open

source →repo →

PaperBench

recommendedrunnable

2026-07-15

Tests

Research-replication benchmark for agents attempting to reproduce AI papers and artifacts.

Best for

Research replication agents

Classification

agent benchmark · current · open

source →repo →

PM-Bench

specializedrunnable

2026-07-15

Tests

Whether agents remember and execute delayed intentions during an ongoing simulated week.

Best for

Testing assistants that must reliably follow up when future cues occur.

Classification

benchmark · emerging · open

paper →

ResearchClawBench

specializedrunnable

2026-07-15

Tests

Automated research agents on rediscovery and new-discovery workflows.

Best for

Comparing research-agent scaffolds rather than chat-model knowledge alone.

Classification

benchmark · emerging · open

dataset →results →

runescape-bench / runebench

specializedpartial

2026-07-15

Tests

Game-world agent benchmark using RuneScape-like task environments.

Best for

Long-horizon game agents

Classification

agent benchmark · current · mixed

repo →results →

SkillsBench

specializedrunnable

2026-07-15

Tests

Whether packaged skills improve agent performance across diverse task environments.

Best for

Evaluating skill libraries, harness design, and portable agent procedures.

Classification

benchmark · emerging · open

dataset →results →

SOP-Bench

specializedrunnable

2026-07-15

Tests

Agents on thousands of multi-step procedures across industrial domains.

Best for

Evaluating process adherence, completion, and tool accuracy in operations workflows.

Classification

benchmark · emerging · mixed

repo →paper →

TheAgentCompany

watchpartial

2026-07-15

Tests

Digital-worker benchmark involving browsing, coding, programs, and simulated coworkers.

Best for

Professional task automation

Classification

agent benchmark · emerging · mixed

source →

ToolBench / ToolLLM

referencerunnable

2026-07-15

Tests

API-use tasks for tool-augmented LLMs across large tool collections.

Best for

Tool-use research baselines

Classification

benchmark · legacy · open

repo →

ToolSandbox

recommendedrunnable

2026-07-15

Tests

Stateful conversational tool-use tasks with tool execution and environment state.

Best for

Robust multi-turn tool agents

Classification

agent benchmark · current · open

repo →

TRAJECT-Bench

specializedpartial

2026-07-15

Tests

Agent tool use with evaluation of the intermediate trajectory as well as the final answer.

Best for

Diagnosing how an agent arrived at an outcome, not only whether it succeeded.

Classification

benchmark · emerging · open

source →

VisualWebArena

recommendedrunnable

2026-07-15

Tests

Browser tasks requiring visual understanding of websites and UI state.

Best for

Multimodal web agents

Classification

agent benchmark · current · open

repo →

WebArena

recommendedrunnable

2026-07-15

Tests

Self-hosted realistic websites and natural-language browser tasks.

Best for

Browser automation agents

Classification

agent benchmark · current · open

source →repo →

WildClawBench

specializedrunnable

2026-07-15

Tests

Agent performance on diverse real-world tasks and environments.

Best for

Comparing general-purpose agent systems on operational work.

Classification

benchmark · emerging · open

dataset →results →

WorkArena / WorkArena++

recommendedrunnable

2026-07-15

Tests

Enterprise workflow tasks modeled in ServiceNow-style environments.

Best for

Business process agents

Classification

agent benchmark · current · open

repo →

YC-Bench

specializedrunnable

2026-07-15

Tests

Agent decision-making across a simulated year of startup operations.

Best for

Exploring long-horizon business-agent behavior and trade-offs.

Classification

benchmark · emerging · open

dataset →results →