Updated 2026-09-25

Terminal-Bench-Fn and 2.1 Leaderboard

Every model is run on two benchmarks with the same 89 tasks: Terminal-Bench 2.1 (TB-2.1) and TB-fn, Fidian's hardened variant of those tasks, on which most models score lower (how it's built). We run everything ourselves with the Terminus-2 agent in Daytona sandboxes, so scores are comparable across models but won't match other leaderboards exactly.

Ranked by pass@1 — the mean of per-task pass rates. The 95% score band and the 2.5–97.5 percentile rank interval both come from a within-task bootstrap (resampling attempts within each fixed task): where rank intervals overlap, the order isn't meaningful — treat nearby models as a rough group, not a strict order. Click any column to re-sort. How intervals work ↓

What's new · past week

Scores, ranks and costs in each entry are the current board's, recomputed on every update.

  1. #

    GPT-6 Sol (max) leads the hardened TB-fn set outright: 90.5% pass@1 (#1, clear of the field), the best hardened-set score we've measured. On the original TB-2.1 set it scores 87.7% (#1), statistically tied with Claude Opus 5.5 at high and medium effort. It was a clean run with no refusals and no throttling, and it is one of seven models that score slightly higher on TB-fn than on TB-2.1 (+2.8, within the noise); most models drop. pass@3 clears 95% on both. It's also cheap for a leader: about $0.52 per solved task on TB-fn (billed, at OpenAI's $2.00/$10.00 per-MTok rate with prompt caching).

  2. #

    GPT-6 Luna (max) is OpenAI's small, low-cost model and a strong value pick: 76.9% pass@1 on the hardened TB-fn set (#11) and 79.7% on the original TB-2.1 set (#14), with 87.5% / 89.7% pass@3. It sits on the cost-efficiency frontier — solid accuracy (#11 on TB-fn) at a small fraction of the leaders' price — about $0.07 per solved task on TB-fn (billed, at OpenAI's $0.10/$0.50 per-MTok rate with prompt caching).

  3. #

    Claude Opus 5.5 — read the effort sweep, not the max run. At high effort it is statistically tied for first on the original TB-2.1 set (87.4% pass@1, #2, just behind GPT-6 Sol's 87.7%) and top-5 on the hardened TB-fn set (84.5%, #4), with pass@3 near 90% on both and almost no timeouts (~1–3%). Medium effort is right behind (83.3% TB-fn / 87.0% TB-2.1). On TB-2.1 both are a clear step up over Opus 5 (~82%); on TB-fn the gain is clear at high effort but slight at medium (83.3% vs Opus 5's best, 82.4%). Cost is roughly $0.35–0.75 per solved task (Anthropic's $4/$20 per-MTok rate, cheaper than Opus 5's $5/$25). The effort=max run is a separate story: it collapses to 59.1% TB-fn / 70.1% TB-2.1 — not a capability ceiling but an over-generation failure mode, where the model spends whole turns reasoning and exhausts the agent's time budget (timeouts jump to ~21–29% of trials and cost to ~$5–8 per solved task). Like Opus 5 and Fable, it declines a cluster of security/bio tasks. Bottom line: run Opus 5.5 at high (or medium); max is counterproductive here.

  4. #

    Grok 4.7 (high) is a lateral release rather than a step up from Grok 4.6: 73.1% pass@1 on the hardened TB-fn set (#19) and 82.4% on the original TB-2.1 set (#9). The TB-fn figure is a touch below Grok 4.6 (76.8%), but the gap sits just inside the noise on the board's 95% intervals (z ≈ 1.9, short of significance). pass@3 stays strong (84.1% on TB-fn, 93.1% on TB-2.1).

Show earlier updates (12)
  1. #

    DeepSeek V4.1 Flash (max, via Fireworks) — final numbers on both sets: 74.1% pass@1 on the hardened TB-fn (#18) and 72.4% on the original TB-2.1 (#26). It holds up as the tasks get harder: its TB-fn score is level with TB-2.1 (+1.7, within the noise), while most models drop. pass@3 is strong on both (87.5% on TB-fn, 82.0% on TB-2.1), so the capability is there; the main limiter is discipline on easy tasks, where it can over-deliberate — spending its full budget theorizing on simple problems rather than just solving them. Cost-per-solved works out to ~$0.14 on TB-fn, estimated from token counts at the rate Fireworks charged for these runs ($0.22 input / $0.007 cached / $0.66 output per MTok; most trials came back unbilled, and the ones that billed match that rate). Very cheap for the accuracy, though GPT-6 Luna has since taken the low end of the cost-efficiency frontier.

  2. #

    GPT-6 Astra (max) — final numbers are in. On the hardened TB-fn set it reaches 84.6% pass@1 (#3 overall); on the original TB-2.1 set it scores 81.7% (#12). A strong showing on both, and the only model whose TB-fn gain over TB-2.1 (+3.0) is statistically significant. As flagged during the run, Astra declines some offensive-security tasks via a content policy — we score those refusals as failures rather than dropping them, so the numbers reflect real task coverage.

  3. #

    Muse Spark 1.3 is an incremental release, landing mid-board. On the hardened TB-fn set it reaches 73.0% pass@1 (#20), a clear gain over Muse Spark 1.2 (65.9%); on the original TB-2.1 set it scores 76.8% (#19), up from 1.2's 71.7%. Competitive with the strong open-weight models, but well short of the leaders.

  4. #

    Gemini 3.8 Flash shows one of the board's most striking splits. On the original TB-2.1 set it scores 83.9% pass@1 — #5 overall and a big jump over 3.7 Flash (74.2%) and 3.6 Flash (72.4%) — but on the hardened TB-fn set it falls to 75.3% (#15), one of the larger drops as the tasks get harder. Top-5 accuracy on the original set, at an estimated ~$1.40–2 per attempt (Google returned no billing, so this uses the effective rate from Gemini 3.6 Flash's bills; the model leans on heavy, mostly-cached input), with an essentially clean run (almost no timeouts).

  5. #

    Claude Fable 5.1 posts 70.1% on the hardened TB-fn set and 68.0% on the original TB-2.1 set. Its clean runs are excellent (~94% pass), so the gap to the headline is reliability, not raw capability, and it comes from two places. Most of it is genuine: it exhausts the agent's turn budget on long-horizon engineering tasks (make-doom, the MIPS interpreter, corewars). The rest is safety: on a cluster of bio and cybersecurity tasks it returns an empty, blocked response, and we score those blocked responses as failures (no fallback model is used).

  6. #

    Gemini 3.7 Flash: 71.2% on the hardened TB-fn set and 74.2% on the original TB-2.1 set — a modest step up from 3.6 Flash (65.2% on TB-fn, 72.4% on TB-2.1), with most of the gain on the harder tasks.

  7. #

    Two additions. DeepSeek V4 Pro 0813 (DeepSeek's dated August 13 checkpoint) scores 69.3% on the hardened TB-fn set and 74.2% on the original TB-2.1 set, in the lower half of the board. GLM-5.3 Flash — a fast, low-cost model — comes in at 67.8% on TB-fn and 70.8% on TB-2.1: near the bottom of the current board, but strong for its price point.

  8. #

    Nemotron 3 Ultra (NVIDIA, 550B-parameter mixture-of-experts) was measured but lands well below the pack — roughly 29% on the hardened TB-fn set and 50% on the original TB-2.1 set, with a high rate of genuine timeouts. We're keeping it off the public board until it's competitive.

  9. #

    GLM-5.3 is one of the strongest open-weight models we've tested: 77.5% on the hardened TB-fn set (#10) and 83.4% on the original TB-2.1 set (#7) — top 10 on both and the best open-weight model on the board, a clear jump over GLM-5.2 (69.1% on TB-fn, 75.1% on TB-2.1).

  10. #

    DeepSeek V4 Pro (max) was measured (52.1% TB-fn / 67.6% TB-2.1) but has since been dropped from the board: the generic Fireworks `deepseek-v4-pro` is deprecated, superseded by V4.1. The pinned 'DeepSeek V4 Pro 0813' checkpoint (OpenRouter, ~69% TB-fn) remains.

  11. #

    Grok 4.6 (high) is a strong entry: 76.8% on the hardened TB-fn set (#13) and 82.9% on the original TB-2.1 set (#8) — a big jump over Grok 4.5, especially on the hardened tasks (69.7% → 76.8%).

  12. #

    Kimi K3 (max), run via Fireworks: 77.3% on the original TB-2.1 set and 74.7% on the hardened TB-fn set. Route caveat: the serving route looks like it matters — our OpenRouter/Moonshot run of the same model scored 82.6% on TB-2.1 (~+5 points), at a similar cost ($0.57/attempt billed, vs ~$0.46 estimated for the Fireworks run). That comparison is confounded by later methodology and task-version changes, so we keep Fireworks as the board entry pending a controlled OpenRouter re-run — treat Kimi's route as a live caveat.

Scores and rank intervals

Both benchmarks are shown per model. Circle = pass@1, square = pass@3; filled = TB-fn, hollow outline = TB-2.1. The bold marker is the benchmark you order by (the TB-2.1 / TB-fn toggle), with its 95% CI as a soft band; the pale marker is the other benchmark, and the hairline connects them — black where the TB-fn drop is significant (95% CI excludes 0), red where it exceeds four standard errors. The bootstrapped rank interval is at left. Axis truncated to the visible scores. Hover a row for exact numbers.

50556065707580859095RANKMODELSCORE (%) · AXIS STARTS AT 50[1–1]GPT-6 Sol (max) NEW[2–5]GPT-5.6 Sol (max)[2–6]GPT-6 Astra (max)[2–6]Claude Opus 5.5 (high) NEW[2–8]Claude Opus 5.5 (medium) NEW[3–9]Claude Opus 5 (xhigh)[4–10]Claude Opus 5 (max)[5–10]GPT-5.6 Sol (xhigh)[6–10]Claude Opus 5 (medium)[9–16]GLM-5.3[10–17]GPT-6 Luna (max) NEW[10–16]Claude Opus 4.8 (max)[10–17]Grok 4.6 (high)[10–20]GPT-5.6 Terra (max)[11–20]Gemini 3.8 Flash[11–22]Kimi K3 (max)[13–20]Qwen 3.8 Max[12–23]DeepSeek V4.1 Flash (max) NEW[14–25]Grok 4.7 (high) NEW[15–24]Muse Spark 1.3[15–25]Claude Fable 5[18–28]Claude Sonnet 5 (max)[18–28]Gemini 3.7 Flash[21–29]Claude Fable 5.1[19–29]GPT-5.6 Luna (max)[21–29]Grok 4.5 (high)[22–29]DeepSeek V4 Pro 0813[21–30]GLM-5.2 (max)[24–31]GLM-5.3 Flash[28–31]Muse Spark 1.2 (xhigh)[29–31]Gemini 3.6 Flash[32–32]Claude Opus 5.5 (max) NEW

Accuracy vs cost — the efficiency frontier

pass@1 against cost per attempt (log scale, cheaper to the right). The efficiency frontier — models for which no cheaper alternative scores higher — is joined by a dashed line, and models off it are dimmed. Some costs are estimates; see Cost below.

556065707580859095$0.02$0.05$0.1$0.2$0.5$1$2$5pass@1 (%)cost (USD per task attempt) — right is cheaperGPT-6 Sol (max) ·NEWGPT-5.6 Sol (max)GPT-6 Astra (max)Claude Opus 5.5 (high) ·NEWClaude Opus 5.5 (medium) ·NEWClaude Opus 5 (xhigh)Claude Opus 5 (max)GPT-5.6 Sol (xhigh)Claude Opus 5 (medium)GLM-5.3GPT-6 Luna (max) ·NEWClaude Opus 4.8 (max)Grok 4.6 (high)GPT-5.6 Terra (max)Gemini 3.8 FlashKimi K3 (max)Qwen 3.8 MaxDeepSeek V4.1 Flash (max) ·NEWGrok 4.7 (high) ·NEWMuse Spark 1.3Claude Fable 5Claude Sonnet 5 (max)Gemini 3.7 FlashClaude Fable 5.1GPT-5.6 Luna (max)Grok 4.5 (high)DeepSeek V4 Pro 0813GLM-5.2 (max)GLM-5.3 FlashMuse Spark 1.2 (xhigh)Gemini 3.6 FlashClaude Opus 5.5 (max) ·NEW

The table below lists the top 20 models (by TB-fn) plus any added in the last 21 days. 11 older models hidden by default.

32 models · 89 tasks · each cell shows TB-fn over TB-2.1 · ordered by TB-fn pass@1

Models ranked by TB-fn pass@1; each cell shows the TB-fn value above the TB-2.1 value.
#1 #1 GPT-6 Sol (max)NEW OpenAI API · 98% coverage +2.8 90.5 87.7 20.6 19.5 0.83M 0.65M 20.7k 17.3k 1.01k 0.89k $0.47 ~$0.37 $0.52 ~$0.42
#2 #4 GPT-5.6 Sol (max) OpenAI API +0.0 85.2 85.2 14.9 16.2 0.40M ·91% 0.40M ·92% 30.4k 29.5k 2.04k 1.83k $1.32 $1.27 $1.53 $1.49
#3 #12 GPT-6 Astra (max) OpenAI API +2.9 84.6 81.7 12.5 11.7 0.29M 0.24M 17.0k 13.7k 1.37k 1.17k $2.11 $1.69 $2.50 $2.08
#4 #2 Claude Opus 5.5 (high)NEW Anthropic API · 98% coverage −2.9 84.5 87.4 15.1 13.5 0.43M ·95% 0.34M ·94% 22.1k 15.0k 1.46k 1.11k $0.63 $0.46 $0.75 $0.53
#5 #3 Claude Opus 5.5 (medium)NEW Anthropic API · 98% coverage −3.7 83.3 87.0 14.8 12.1 0.37M ·95% 0.19M ·92% 13.9k 10.3k 0.94k 0.85k $0.44 $0.32 $0.53 $0.36
#6 #13 Claude Opus 5 (xhigh) Anthropic API +1.1 82.4 81.3 21.9 21.7 0.78M ·94% 0.68M ·94% 57.2k 50.6k 2.62k 2.33k $2.08 $1.83 $2.49 $2.25
#7 #10 Claude Opus 5 (max) Anthropic API −1.0 81.3 82.3 24.0 24.2 0.93M ·94% 0.90M ·95% 84.3k 74.3k 3.51k 3.07k $2.90 $2.58 $3.53 $3.14
#8 #6 GPT-5.6 Sol (xhigh) OpenAI API −2.7 81.1 83.8 15.2 12.9 0.43M ·94% 0.25M ·90% 21.6k 15.3k 1.42k 1.19k $1.02 $0.73 $1.25 $0.87
#9 #11 Claude Opus 5 (medium) Anthropic API −1.1 80.7 81.8 17.5 16.1 0.55M ·96% 0.36M ·95% 18.8k 14.1k 1.07k 0.88k $0.88 $0.65 $1.09 $0.79
#10 #7 GLM-5.3 OpenRouter −5.9 77.5 83.4 27.1 24.4 1.26M ·97% 1.00M ·96% 68.4k 44.6k 2.52k 1.83k $0.68 $0.50 $0.87 $0.60
#11 #14 GPT-6 Luna (max)NEW OpenAI API · 98% coverage −2.8 76.9 79.7 22.8 22.3 1.69M 1.36M 42.3k 35.5k 1.86k 1.59k $0.05 ~$0.04 $0.07 ~$0.05
#12 #20 Claude Opus 4.8 (max) Anthropic API +1.1 76.8 75.7 25.4 24.9 0.85M ·92% 0.77M ·93% 105.9k 86.3k 4.17k 3.47k $3.46 $2.84 $4.43 $3.75
#13 #8 Grok 4.6 (high) OpenRouter −6.1 76.8 82.9 26.8 22.5 1.12M ·95% 0.70M ·92% 43.7k 27.8k 1.64k 1.24k $0.94 $0.60 $1.22 $0.73
#14 #15 GPT-5.6 Terra (max) OpenAI API −2.0 75.5 77.5 21.8 21.3 0.89M ·96% 0.81M ·95% 75.2k 71.9k 3.45k 3.37k $1.17 $1.12 $1.53 $1.45
#15 #5 Gemini 3.8 Flash Google Vertex (Express) −8.6 75.3 83.9 75.4 67.2 4.93M ·86% 3.47M ·84% 41.9k 32.5k 0.56k 0.48k ~$1.99 ~$1.43 ~$2.64 ~$1.70
#16 #16 Kimi K3 (max) Fireworks AI −2.6 74.7 77.3 22.0 19.6 0.92M ·96% 0.62M ·96% 19.5k 14.1k 0.89k 0.72k ~$0.66 ~$0.46 ~$0.88 ~$0.60
#17 #18 Qwen 3.8 Max OpenRouter −2.1 74.7 76.8 23.3 21.8 0.70M ·89% 0.58M ·86% 75.5k 60.9k 3.24k 2.80k $0.77 $0.65 $1.02 $0.85
#18 #26 DeepSeek V4.1 Flash (max)NEW Fireworks AI · 99% coverage +1.7 74.1 72.4 31.8 32.2 1.54M ·97% 1.48M ·68% 119.3k 114.9k 3.75k 3.57k ~$0.10 ~$0.19 ~$0.14 ~$0.26
#19 #9 Grok 4.7 (high)NEW OpenRouter · 98% coverage −9.3 73.1 82.4 36.3 31.1 2.68M ·90% 1.63M ·88% 31.3k 21.9k 0.86k 0.70k $1.92 $1.08 $2.63 $1.31
#20 #19 Muse Spark 1.3 OpenRouter −3.8 73.0 76.8 45.6 46.7 2.28M ·77% 2.57M ·78% 169.4k 134.5k 3.71k 2.88k $1.63 $1.58 $2.23 $2.05
#21 #21 Claude Fable 5 Anthropic API −2.4 72.7 75.1 13.6 12.7 0.34M ·92% 0.22M ·89% 23.8k 18.3k 1.75k 1.44k $1.84 $1.41 $2.52 $1.87
#22 #28 Claude Sonnet 5 (max) Anthropic API −0.9 71.2 72.1 38.5 31.0 2.34M ·96% 1.51M ·96% 158.0k 117.4k 4.10k 3.79k $2.24 $1.62 $3.06 $2.25
#23 #24 Gemini 3.7 Flash Google Vertex (Express) −3.0 71.2 74.2 52.6 41.9 3.48M ·90% 2.02M ·88% 30.9k 22.9k 0.59k 0.55k $0.61 $0.40 $0.86 $0.54
#24 #32 Claude Fable 5.1 Anthropic API · 99% coverage +2.1 70.1 68.0 16.5 15.2 0.43M ·84% 0.35M ·87% 101.5k 78.2k 6.16k 5.15k $6.05 $4.57 $8.63 $6.49
#25 #23 GPT-5.6 Luna (max) OpenAI API −4.8 70.0 74.8 27.6 26.3 1.59M ·97% 1.27M ·96% 68.7k 56.7k 2.49k 2.16k $0.13 $0.11 $0.18 $0.14
#26 #17 Grok 4.5 (high) OpenRouter −7.6 69.7 77.3 30.2 22.9 1.40M ·95% 0.79M ·94% 24.9k 16.1k 0.82k 0.70k $0.74 $0.43 $1.06 $0.55
#27 #25 DeepSeek V4 Pro 0813 OpenRouter −4.9 69.3 74.2 32.0 27.2 1.61M ·93% 1.18M ·75% 85.9k 71.6k 2.69k 2.63k $0.66 $0.85 $0.95 $1.14
#28 #22 GLM-5.2 (max) Fireworks AI · 99% coverage −6.0 69.1 75.1 25.7 23.1 1.33M ·96% 1.13M ·96% 75.8k 63.4k 2.95k 2.75k $0.58 $0.50 $0.83 $0.67
#29 #30 GLM-5.3 Flash OpenRouter · 99% coverage −3.0 67.8 70.8 25.9 23.6 0.99M ·74% 0.74M ·44% 58.0k 48.5k 2.24k 2.06k ~$0.09 ~$0.10 ~$0.13 ~$0.14
#30 #29 Muse Spark 1.2 (xhigh) OpenRouter −5.8 65.9 71.7 39.2 43.2 2.38M ·50% 2.85M ·70% 107.6k 116.3k 2.74k 2.69k $2.12 $1.87 $3.16 $2.61
#31 #27 Gemini 3.6 Flash Google Vertex (Express) −7.2 65.2 72.4 72.4 60.9 5.50M ·86% 4.09M ·86% 46.4k 36.7k 0.64k 0.60k $2.24 $1.68 $3.45 $2.32
#32 #31 Claude Opus 5.5 (max)NEW Anthropic API · 98% coverage −11.0 59.1 70.1 13.7 12.9 0.31M ·71% 0.27M ·77% 209.5k 150.0k 15.26k 11.59k $4.68 ~$3.42 $7.92 ~$4.87

Scoring. Each model makes several independent attempts per task: 3 on TB-fn; on TB-2.1, 5 for models measured before September and 3 since (a few tasks carry an extra attempt from reruns). pass@1 is the mean of per-task pass rates; pass@3 is the estimated share of tasks solved in at least one of three attempts. Effort levels (medium, high, xhigh, max) are each provider's reasoning-effort setting. Following Artificial Analysis's convention, attempts that fail for infrastructure reasons are excluded; agent timeouts, verifier timeouts and policy refusals (including provider-side blocks) count as failures.

Intervals. Both the 95% score band and the rank interval are a within-task bootstrap: each of 5,000 resamples draws attempts (with replacement) within every fixed task and recomputes the score, holding the task set constant; we then take the 2.5–97.5 percentiles for both the score and each model's rank. This captures run-to-run / attempt sampling on a fixed suite — the convention Artificial Analysis uses for its repeat-based intervals — so it answers "how precisely is this number pinned down," not "would a different task set reorder the board." (This is the same interval method as our benchmark post; the figures differ because the board has more trials and models since.)

Reading the board. Each cell shows TB-fn over TB-2.1, including the rank on each benchmark; the table is ordered by (and sortable on) TB-fn pass@1, single-shot reliability. Δ is TB-fn minus TB-2.1 pass@1 in points. The resource columns — turns, input and output tokens, and output tokens per turn — are per-attempt averages, with the input-token cache-hit rate shown alongside (blank for GPT-6 models: our harness didn't record their cached tokens on these runs).

Cost. Plain $ figures are what the provider billed. A ~$ figure is an estimate. Where a model was billed on another run, its unbilled trials are priced at list rate scaled by that run's observed billed-to-list ratio, which accounts for prompt caching. Gemini 3.8 Flash returned no billing, so it is priced at a cache-adjusted rate taken from Gemini 3.6 Flash's real bills. Kimi K3, DeepSeek V4.1 Flash and GLM-5.3 Flash were mostly unbilled; they are token counts × each route's rates (as published, or as charged where the two differ), with cached input at the cached-input rate. Where a few of their trials did bill, the estimate matches those bills.

Coverage. Some models were scored on fewer than all 89 tasks because a task environment had stopped building by the time they were run or re-run. Models added on or after September 21 are missing qemu-alpine-ssh on both benchmarks and qemu-startup on TB-2.1. Claude Fable 5.1, DeepSeek V4.1 Flash and GLM-5.2 are missing qemu-alpine-ssh on TB-fn, and GLM-5.3 Flash is missing schemelike-metacircular-eval on TB-fn. The QEMU tasks are ones most models solve (about 97% of attempts on TB-fn and 90% on TB-2.1), so leaving them out changes most models' scores by less than a point. A "· NN% coverage" note in the table marks every affected model, showing the lower of its two benchmarks. The board is fully recomputed from all recorded attempts on every update.