LiveUpdated 2026-11-07 16:42 UTC

I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard.

Name: I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard.
Uploaded: 2026-11-05T16:04:08.000Z
Channel: Joël Niklaus
Description: I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard. Which harness should you run your open model in? I ran 28,849 trials to find out. The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model,…

This is a dev post classified by AGK Intelligence as AI dev tools (a launch), kept by the Dev Radar because it carries real work, not commentary.

I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard. Which harness should you run your open model in? I ran 28,849 trials to find out. The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model, provider and reasoning effort stay fixed, and every task gets three attempts. The trials ran GLM-5.3-Flash, DeepSeek-V4.1-Flash and Flint on Terminal-Bench 2.1 and DeepSWE, for $4,746 of inference. Averaged over the three models, mini-swe-agent (@KLieret), a bash-only agent, solves 51.4% of tasks and

Posted by Joël Niklaus (3.4k followers) 2 days ago · 2.3k likes · 252.1k views · view the original post on X. Kept by the Dev Radar as AI dev tools. Tools mentioned: harness-leaderboard, harness-lens.

More dev work like this

💥细节感拉满！Qwen 4.0 蜜蜂向日葵测试炸了！ — @NFT_Chen
Introducing Keras Core! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT… — @gp_pulipaka
ReRun Viewer! Visualize, Query, And Stream To Train On Multimodal Robotics Data.… — @gp_pulipaka
#Keras with #PyTorch! @Yampeleg #BigData #Analytics #DataScience #AI #MachineLearning… — @gp_pulipaka
I Know but Falls Short of Its Promise! Insight Into LLM Context Failures! #BigData… — @gp_pulipaka
Weight Initialization in Neural Networks. #BigData #Analytics #DataScience #AI… — @gp_pulipaka
不想反复截图发 K 线？试试 tradingview-mcp。 — @bkdgiffug
🫢绝了！让你的AI Agent 3步变身逆向专家！Awesome-REA正式上线！AI代理有了一个专用逆向工程的武器库！ — @NFT_Chen

Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.8k posts from 5k X accounts over the last 21 days, 9.6k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-07 16:42 UTC. Full method.