Who is at the frontier of terminal tasks?
We ran the top 20 models from Terminal-Bench 2.1 on TB-fn, our harder variant of the same 89 tasks. Only OpenAI and Anthropic held the top tier, and every model cost more per task.
Research
A living leaderboard we update regularly, and the occasional deep dive into how we got the numbers.
Leaderboard ยท updated regularly
Terminal-Bench-Fn & 2.1
Our runs of Terminal-Bench 2.1 and its hardened TB-fn variant, ranked by pass@1 with 95% confidence and rank intervals, pass@3, and cost per solved task.
