llm-bench-rig
llm-bench-rig is Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards. - notwitcheer/llm-bench-rig. It is ranked #1203 on the Dev Radar, in AI dev tools, first seen 4 days ago and shared in 3 posts (17.6k views).
llm-bench-rig/reports/toolcall-v1.md at main · notwitcheer/llm-bench-rig · GitHub Skip to content Navigation Menu Sign in Appearance settings Search / Sign in Sign up Appearance settings You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Repository navigation Files Expand file tree main / toolcall-v1.md Copy path Blame More file actions Blame More file actions Latest commit History History History 95 lines (73 loc) · 14.8 KB main /…
What people said about llm-bench-rig on X
which file of Qwen3.8-27B should you download if you want it to use tools, the way an agent like Hermes Agent does? I tested six files of the same model, from 22.9 GB down to 5.9 GB, on 100 tasks where the model has to keep calling tools over several turns. TL;DR: the 3-bit file (11.9 GB) is as good as the 6-bit one…
— @witcheer, 4 days ago · 201 likes · see the post
Ornith 1.5 122B runs on one 32 GB card, but the 35B is within noise of it on every test except MMLU, and 6x faster. TL;DR: - 122B, 3-bit file (46.1 GiB): 47.6 tok/s on one RTX 5090, with part of its experts in system RAM - 35B, Q4_K_M (20.2 GiB): fits fully on the card, 303 tok/s - first token on a 16k prompt: ~28 s…
— @witcheer, 3 days ago · 50 likes · see the post
which settings are worth changing when a local model runs Hermes Agent, and which change nothing you can measure? Qwen3.8-27B on one 32 GB RTX 5090, one setting per run, the same 100 multi-turn tool-call tasks every time. the clip runs one Hermes Agent task twice on the same model, thinking on and then thinking off,…
— @witcheer, 2 days ago · 32 likes · see the post
Alternatives to llm-bench-rig
- goo.gle — Officially in beta, the AppFunctions API is opening up powerful new ways to integrate your functionality.
- Kimi AI with K3.1 Pro — Try Kimi K3.1 Pro to tackle complex coding projects, build 3D models, conduct financial research, and take on more…
- Agents in flight — Give it a try
- Baseten — This session is our most in-demand yet. See you there
- project-maya — Checkout Maya
- deepinfer — You gotta love Deepseek. This is awesome stuff!
llm-bench-rig in numbers
- Rank on the Dev Radar: #1203 of 8801
- Shared in 3 posts by 1 account: @witcheer
- 17.6k views on those posts
- First seen 4 days ago, last shared 2 days ago
- Pricing seen by AGK Intelligence: open source
- Market: AI dev tools
FAQ
What is llm-bench-rig?
Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards. - notwitcheer/llm-bench-rig It was first shared on X 4 days ago and is ranked #1203 on the Dev Radar.
Is llm-bench-rig free?
It is open source.
Who shared llm-bench-rig?
1 account on X, including @witcheer, in 3 posts totalling 17.6k views.
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.2k posts from 5k X accounts over the last 21 days, 8.8k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-10-30 21:12 UTC. Full method.