vLLM gave Local AI AMD users something more interesting than another decode speedup.
This is a dev post classified by AGK Intelligence as AI dev tools (a tool drop), kept by the Dev Radar because it carries real work, not commentary.
vLLM gave Local AI AMD users something more interesting than another decode speedup. ... like 4.6× the KV-cache capacity on the same GPUs? so... vLLM 0.33.0 is out and buried in the release is ROCM_OCTAVE, a new 3–4-bit KV-cache backend for AMD GPUs. In the developers' tests 🎮 4× Radeon RX 7900 XTX 🧠 Qwen3.8-27B 💾 FP16 KV cache → baseline 🔥 ROCM_OCTAVE → 4.6× as many KV-cache tokens A lot more room for context and concurrent requests. And there's an actual dual-GPU community test too. 2× RX 7900 XTX running Qwen3.8-27B at ~180K context ⚡ 3-bit Octave → 44.8 tps ⚡ Separate INT8 backend → 3
Posted by David Hendrickson (12.7k followers) 1 h ago · 1 likes · 252 views · view the original post on X. Kept by the Dev Radar as AI dev tools.
More dev work like this
- Another stealth model 👀 — @ItsmeAjayKV
- Anthropic 把自家做 Claude 应用的示范代码整套放了出来，GitHub 上已经 1.8 万 star。 — @bkdgiffug
- 28 paid model calls a week just to say "no change." 𝗛𝗲𝗿𝗺𝗲𝘀 Agent skips them. 💸 — @BkashJosi
- Finding Claude Code skills shouldn’t take longer than choosing what to build. — @DanKornas
- Meet ManniX-ITA/opencoti-llamafile: a self-contained AI model that runs anywhere. No… — @HuggingModels
- HERMES CAN SPLIT ONE BIG JOB ACROSS 10 AGENTS WORKING AT THE SAME TIME. — @HermesWatcher
- Sushi 🍣 1.5.0 is here! with all new WebUI and Sushibar. DeepSeek also writes 10x less… — @Beamsters1
- 根本没玩过blender.也不会3D建模，看了几个教程一头雾水。按键都记不住。没办法，把blender接好MCP后让Gemini给我搞的，图二是修复后的，我已经很满意了。一开始用… — @eastwoodnet
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.4k posts from 5k X accounts over the last 21 days, 9.2k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-04 03:52 UTC. Full method.