Multi-Dimensional Public Ranked Directories (233 Verified Entries)
Inspect MCP servers, Cloud Ollama & frontier agent models, benchmarks, engineering/security/humanitarian RSS feeds, and verified harness files via HTTPS (GET /v1/public/lists/:slug) or MCP (teleon_public_list_query).
top-agent-models (40) top-benchmarks (36) top-harness-files (45) top-mcp-servers (52) top-rss-feeds (60)
| Rank | Entry & Summary | Category | Composite | Measured Dimensions |
|---|---|---|---|---|
| #1 | Humanity's Last Exam (HLE) 2,500 expert-authored multimodal and text questions spanning graduate-level science, humanities, and mathematics. |
academic_frontier |
68.78 | {"contamination_resistance":95.3,"eval_cost_efficiency":87.6,"real_world_transfer_r2":0.7,"reproducibility_score":94.1,"saturation_headroom_pct":66.2} |
| #2 | Epoch AI FrontierMath Held-out research-grade mathematics benchmark crafted with Fields Medalists and automated Python verification. |
formal_and_math |
68.66 | {"contamination_resistance":99.1,"eval_cost_efficiency":82.5,"real_world_transfer_r2":0.72,"reproducibility_score":93,"saturation_headroom_pct":68} |
| #3 | SciCode Scientific Research Coding Curated multi-step numerical problems across physics, chemistry, biology, and materials science. |
scientific_computing |
67.33 | {"contamination_resistance":94.1,"eval_cost_efficiency":88,"real_world_transfer_r2":0.77,"reproducibility_score":95.2,"saturation_headroom_pct":58.6} |
| #4 | AppWorld Multi-App Agent Benchmark Deterministic 9-app, 457-API stateful sandbox grading programmatic database assertions and zero unintended side effects. |
tool_calling_and_policy |
64.3 | {"contamination_resistance":92.5,"eval_cost_efficiency":85.4,"real_world_transfer_r2":0.81,"reproducibility_score":96,"saturation_headroom_pct":46.8} |
| #5 | SWE-bench Multimodal Frontend JavaScript/TypeScript bug reports containing rendered visual artifacts, charts, and CSS regressions. |
repository_coding |
63.76 | {"contamination_resistance":92.1,"eval_cost_efficiency":75.2,"real_world_transfer_r2":0.8,"reproducibility_score":94.3,"saturation_headroom_pct":56.4} |
| #6 | Cybench Cybersecurity Agent Benchmark Professional Capture-the-Flag tasks with subtask milestone grading across cryptography, web, pwn, and forensics. |
security_and_ctf |
63.54 | {"contamination_resistance":91.9,"eval_cost_efficiency":79,"real_world_transfer_r2":0.78,"reproducibility_score":94.8,"saturation_headroom_pct":51.2} |
| #7 | LiveCodeBench Continuously refreshed timestamped coding problems from AtCoder, LeetCode, and Codeforces with date-sliced splits. |
contamination_free_coding |
63.51 | {"contamination_resistance":97.6,"eval_cost_efficiency":92.4,"real_world_transfer_r2":0.74,"reproducibility_score":95.8,"saturation_headroom_pct":31} |
| #8 | OpenAI SimpleQA Calibration Benchmark Measures parametric factual calibration and rewards auditable abstention over confident hallucination. |
calibration_and_abstention |
63.04 | {"contamination_resistance":86,"eval_cost_efficiency":95.1,"real_world_transfer_r2":0.71,"reproducibility_score":95.9,"saturation_headroom_pct":37.5} |
| #9 | BigCodeBench Tests complex library composition across 139 real Python libraries with rich unit test branch coverage. |
contamination_free_coding |
62.57 | {"contamination_resistance":87.2,"eval_cost_efficiency":93.5,"real_world_transfer_r2":0.75,"reproducibility_score":96.6,"saturation_headroom_pct":34.8} |
| #10 | LongBench v2 Deep Long-Context 503 multi-document, code-repository, and long-dialogue tasks spanning 8k to 2M words. |
long_context |
62.37 | {"contamination_resistance":90.8,"eval_cost_efficiency":83.4,"real_world_transfer_r2":0.74,"reproducibility_score":95.3,"saturation_headroom_pct":41.6} |
| #11 | Terminal-Bench Evaluates autonomous shell agents inside Docker sandboxes across compilation, kernel debugging, and sysadmin recovery. |
cli_and_systems |
62.29 | {"contamination_resistance":93.2,"eval_cost_efficiency":76.5,"real_world_transfer_r2":0.85,"reproducibility_score":96.4,"saturation_headroom_pct":44.5} |
| #12 | BEIR Heterogeneous Zero-Shot IR Zero-shot out-of-domain information retrieval benchmark comparing BM25, SPLADE, ColBERT PLAID, and dense vectors. |
retrieval_and_rag |
61.71 | {"contamination_resistance":84.2,"eval_cost_efficiency":95.6,"real_world_transfer_r2":0.77,"reproducibility_score":97.8,"saturation_headroom_pct":30.2} |
| #13 | TAU-bench (Tool-Agent-User) Measures pass^k consistency of multi-turn customer-support and airline/retail policy compliance under stateful tools. |
tool_calling_and_policy |
61.6 | {"contamination_resistance":89.2,"eval_cost_efficiency":86,"real_world_transfer_r2":0.82,"reproducibility_score":93.6,"saturation_headroom_pct":38.4} |
| #14 | SWE-Lancer Freelance Engineering Benchmark 1,400+ real Upwork Expensify engineering implementation and technical-management tasks with Playwright end-to-end tests. |
repository_coding |
61.53 | {"contamination_resistance":92.6,"eval_cost_efficiency":71.4,"real_world_transfer_r2":0.84,"reproducibility_score":93.8,"saturation_headroom_pct":49} |
| #15 | Berkeley Function Calling Leaderboard v3 (BFCL-v3) AST and executable multi-step, parallel, and irrelevance-abstention function calling benchmark. |
tool_calling_and_policy |
61.48 | {"contamination_resistance":87.8,"eval_cost_efficiency":93.2,"real_world_transfer_r2":0.79,"reproducibility_score":96.1,"saturation_headroom_pct":29.5} |
| #16 | RepoBench Cross-File Code Completion Evaluates cross-file import retrieval and next-line synthesis across multi-module Python and Java repositories. |
repository_coding |
61.21 | {"contamination_resistance":86.2,"eval_cost_efficiency":94.8,"real_world_transfer_r2":0.74,"reproducibility_score":96.7,"saturation_headroom_pct":27.6} |
| #17 | Google FRAMES Factuality & Retrieval 824 multi-hop questions testing joint retrieval, temporal constraint satisfaction, and tabular arithmetic. |
retrieval_and_rag |
60.97 | {"contamination_resistance":89.6,"eval_cost_efficiency":88.2,"real_world_transfer_r2":0.76,"reproducibility_score":93.5,"saturation_headroom_pct":32.8} |
| #18 | SWE-bench Multilingual Extends repository patch verification beyond Python to Rust, Go, TypeScript, Java, C++, Ruby, and PHP. |
repository_coding |
60.91 | {"contamination_resistance":91.4,"eval_cost_efficiency":74.8,"real_world_transfer_r2":0.83,"reproducibility_score":95.5,"saturation_headroom_pct":42} |
| #19 | OSWorld Multimodal Desktop Eval Real Ubuntu, Windows, and macOS VM execution benchmark spanning LibreOffice, GIMP, VS Code, and multi-app workflows. |
computer_use |
60.9 | {"contamination_resistance":92.8,"eval_cost_efficiency":64.5,"real_world_transfer_r2":0.8,"reproducibility_score":92.4,"saturation_headroom_pct":54} |
| #20 | NVIDIA RULER Long-Context Eval Synthetic variable-tracking, multi-hop tracing, and aggregation tasks exposing true vs claimed context window sizes. |
long_context |
60.87 | {"contamination_resistance":91.5,"eval_cost_efficiency":86.8,"real_world_transfer_r2":0.75,"reproducibility_score":96.9,"saturation_headroom_pct":28.4} |
| #21 | BrowseComp Deep Web Foraging Eval 1,266 hard-to-find entangled web browsing questions that remain trivial to verify symmetrically once located. |
web_and_tool_research |
60.71 | {"contamination_resistance":93.4,"eval_cost_efficiency":72,"real_world_transfer_r2":0.76,"reproducibility_score":88.9,"saturation_headroom_pct":48.5} |
| #22 | MTEB Massive Text Embedding Benchmark Standardized retrieval, reranking, clustering, and bitext mining suite across 250+ languages and code corpora. |
retrieval_and_rag |
60.44 | {"contamination_resistance":85,"eval_cost_efficiency":94,"real_world_transfer_r2":0.78,"reproducibility_score":96.4,"saturation_headroom_pct":26} |
| #23 | CRUXEval Code Execution Reasoning Input prediction and output prediction tasks testing mental execution of Python control flow and state mutation. |
contamination_free_coding |
60.28 | {"contamination_resistance":85.4,"eval_cost_efficiency":97.4,"real_world_transfer_r2":0.7,"reproducibility_score":98.1,"saturation_headroom_pct":19.8} |
| #24 | Aider Polyglot Coding Benchmark 225 hardest Exercism problems across C++, Go, Java, JavaScript, Python, and Rust testing edit-format fidelity. |
repository_coding |
59.84 | {"contamination_resistance":86.4,"eval_cost_efficiency":91.8,"real_world_transfer_r2":0.78,"reproducibility_score":95.7,"saturation_headroom_pct":24.5} |
| #25 | ServiceNow WorkArena Enterprise UI Evaluates enterprise knowledge-work browser agents on ITIL forms, dashboards, RBAC workflows, and SLA routing. |
computer_use |
59.66 | {"contamination_resistance":90.2,"eval_cost_efficiency":70.5,"real_world_transfer_r2":0.79,"reproducibility_score":91.8,"saturation_headroom_pct":45} |
| #26 | VisualWebArena 910 visually grounded web tasks requiring screenshot comprehension alongside DOM interaction. |
computer_use |
59.55 | {"contamination_resistance":89.4,"eval_cost_efficiency":67.2,"real_world_transfer_r2":0.76,"reproducibility_score":92.9,"saturation_headroom_pct":47.5} |
| #27 | GAIA General AI Assistants Multi-step real-world reasoning tasks with unambiguous short-form ground-truth answers across three difficulty levels. |
web_and_tool_research |
59.09 | {"contamination_resistance":88.6,"eval_cost_efficiency":81.2,"real_world_transfer_r2":0.77,"reproducibility_score":91.5,"saturation_headroom_pct":33.4} |
| #28 | MLCommons AILuminate v1.0 Safety Multi-stakeholder hazard taxonomy testing CBRN, cyber-abuse, self-harm, privacy, and false-refusal calibration. |
safety_and_governance |
58.77 | {"contamination_resistance":90.5,"eval_cost_efficiency":85.2,"real_world_transfer_r2":0.73,"reproducibility_score":93.4,"saturation_headroom_pct":24} |
| #29 | GPQA Diamond Google-proof PhD-level biology, physics, and chemistry multiple-choice diagnostic split. |
academic_frontier |
58.76 | {"contamination_resistance":83,"eval_cost_efficiency":96.2,"real_world_transfer_r2":0.69,"reproducibility_score":97.4,"saturation_headroom_pct":16.5} |
| #30 | WebArena Self-Hosted Web Environment Self-hosted GitLab, Reddit-like forum, e-commerce, CMS, and OpenStreetMap benchmark with functional state verifiers. |
computer_use |
58.37 | {"contamination_resistance":88.1,"eval_cost_efficiency":69.8,"real_world_transfer_r2":0.77,"reproducibility_score":94,"saturation_headroom_pct":39.2} |
| #31 | LMSYS Arena-Hard-Auto High-separability crowd-sourced engineering prompts with style-controlled Bradley-Terry pairwise scoring. |
preference_and_rubric |
57.91 | {"contamination_resistance":86.7,"eval_cost_efficiency":89.5,"real_world_transfer_r2":0.73,"reproducibility_score":90.2,"saturation_headroom_pct":22.4} |
| #32 | THUDM AgentBench Multi-Environment 8 interactive environments covering OS shell, SQL DB, knowledge graph SPARQL, card games, and web shopping. |
cli_and_systems |
57.82 | {"contamination_resistance":82.9,"eval_cost_efficiency":84,"real_world_transfer_r2":0.71,"reproducibility_score":92.5,"saturation_headroom_pct":29} |
| #33 | Stanford CRFM HELM Core Scenarios Standardized multi-metric reporting across accuracy, calibration, robustness, fairness, and inference efficiency. |
safety_and_governance |
57.42 | {"contamination_resistance":84.8,"eval_cost_efficiency":80.6,"real_world_transfer_r2":0.72,"reproducibility_score":95.6,"saturation_headroom_pct":25.4} |
| #34 | OpenBMB ToolBench Large-scale REST API selection and DFSDT tool-tree routing across thousands of real API specifications. |
tool_calling_and_policy |
57.36 | {"contamination_resistance":83.5,"eval_cost_efficiency":83,"real_world_transfer_r2":0.69,"reproducibility_score":88.4,"saturation_headroom_pct":31.2} |
| #35 | SWE-bench Verified Human-validated 500-instance subset of real GitHub pull requests graded by deterministic container test suites. |
repository_coding |
56.86 | {"contamination_resistance":84.5,"eval_cost_efficiency":78,"real_world_transfer_r2":0.81,"reproducibility_score":97.2,"saturation_headroom_pct":23.8} |
| #36 | EvalPlus (HumanEval+ & MBPP+) 80x mutation-expanded edge-case test suites exposing brittle passes on classic function synthesis tasks. |
contamination_free_coding |
56.59 | {"contamination_resistance":74,"eval_cost_efficiency":98.5,"real_world_transfer_r2":0.66,"reproducibility_score":98.6,"saturation_headroom_pct":11.2} |