Open-Source LLM Benchmarks vs Closed Frontier Models

SWE-bench Verified over time

Agentic issue resolution (%). Thin lines connect each class's releases (extreme outliers omitted); dashed trendlines: one per closed lab and one per open size class, fit to each series' running-best frontier (outliers excluded) and extended past today. Hollow points: approximate release date. Hover the list to locate a model.

SWE-rebench: decontaminated resolved rate over time

Continuously refreshed with post-release GitHub tasks (harder than SWE-bench Verified; honest for open models). Fully rebuilt from the live leaderboard on every refresh.

BFCL v4: tool calling over time

Berkeley Function Calling overall (%), live leaderboard, best variant per model.

LiveCodeBench over time

Contamination-free code generation (%); live leaderboard plus wiki-curated edge checkpoints.