Artificial Analysis
- Tests
- Model intelligence, coding, math, science, speed, latency, and pricing across commercial and open models.
- Best for
- Model shortlist and cost-performance tradeoffs
- Classification
- scoreboard · current · mixed
AI evaluation domain · 13 source-backed records
General AI Model Benchmarks
Use these benchmarks to build an initial model shortlist, inspect broad reasoning and factuality, and understand how aggregate leaderboards are assembled. They are scouting tools—not substitutes for task-specific evaluation.
Decision guidance
Which benchmarks are useful for comparing general-purpose AI models?
Start with one transparent aggregate or harness, then inspect the component benchmarks behind any composite score. Prefer fresh, contamination-aware sets and confirm that the tested model snapshot and inference settings match what you can actually deploy.
13
Benchmarks in this domain
Sources first · alphabetical collection order
Runnability describes access to a usable repository, dataset, or harness—not whether setup is easy.
2026-07-15
Benchmark Health Index
- Tests
- Benchmarks themselves across discrimination, anti-saturation, and ecosystem impact.
- Best for
- Choosing which public benchmarks still carry useful signal.
- Classification
- toolkit · emerging · open
Benchmark²
- Tests
- Benchmark quality through cross-ranking consistency, discriminability, and capability alignment.
- Best for
- Auditing whether an evaluation measures the capability its label implies.
- Classification
- toolkit · emerging · open
EleutherAI LM Evaluation Harness
- Tests
- Standardized runner for many classic language-model tasks and benchmark suites.
- Best for
- CI-style baseline model evaluation
- Classification
- harness · current · open
HELM
- Tests
- Holistic evaluation framework with transparent scenarios, metrics, prompts, and model predictions.
- Best for
- Academic reproducibility and broad model audits
- Classification
- harness · current · open
Hugging Face Open LLM Leaderboard
- Tests
- Open-weight model leaderboard and result archive; useful historically, but check freshness before treating it as current.
- Best for
- Open model comparison and historical baselines
- Classification
- scoreboard · legacy · mixed
IFEval
- Tests
- Instruction-following benchmark with verifiable formatting and constraint adherence.
- Best for
- Production assistants with strict output rules
- Classification
- benchmark · current · open
IFStruct v1.0
- Tests
- Whether models satisfy compositional structural constraints in JSON and YAML outputs.
- Best for
- Comparing schema compliance before testing private structured-output contracts.
- Classification
- benchmark · emerging · open
LMArena / Chatbot Arena
- Tests
- Blind pairwise human preference for chat, coding, and style-controlled model comparisons.
- Best for
- Human preference and conversational quality
- Classification
- scoreboard · current · hosted
OpenCompass / CompassRank
- Tests
- Open evaluation platform across many datasets with public and private benchmark dimensions.
- Best for
- Broad open-source evaluation runs
- Classification
- harness · current · open
SimpleQA
- Tests
- Short factual questions with clear answers for hallucination and factual recall checks.
- Best for
- Factuality and hallucination screening
- Classification
- benchmark · current · open
Toloka Arena
- Tests
- Hosted agentic-intelligence leaderboard with composite pass rates and enterprise-domain datasets.
- Best for
- Commercial agent benchmark comparison
- Classification
- scoreboard · current · commercial
TruthfulQA
- Tests
- Questions designed to test whether models repeat common falsehoods.
- Best for
- Truthfulness baseline and regression testing
- Classification
- benchmark · legacy · open