💥卧槽！本地LLM推理迎来真正突破！uzu引擎比llama.cpp快4倍，M5 Mac上9B模型直接飙到92 tok/s！
This is a dev post classified by AGK Intelligence as AI dev tools (a tool drop), kept by the Dev Radar because it carries real work, not commentary.
💥卧槽！本地LLM推理迎来真正突破！uzu引擎比llama.cpp快4倍，M5 Mac上9B模型直接飙到92 tok/s！ 一次解码平均产出7.5个token，彻底撕开本地推理的内存带宽天花板！ 核心亮点： 🔹实测碾压：在基础版M5 MacBook Pro上跑Qwen3.5 9B（4-bit），llama.cpp仅22.0 tok/s、MLX 25.1 tok/s，uzu直接92.1 tok/s，约4倍速度。 🔹技术黑科技：采用激进投机解码（Speculative Decoding）。小模型DFlash一次性提案未来16个位置，56.7M参数的Weaver适配器构建连贯树，主模型单次pass验证整棵树并保留最长正确路径。普通解码每pass仅1 token，uzu平均7.5 token。 🔹硬件协同设计：Rust + Metal内核，专为Apple Silicon优化，充分利用M5的int8矩阵路径和统一内存。支持Python / Swift / TypeScript / Rust绑定，可直接嵌入Mac/iOS应用。 🔹完全开源 + 本地优先：Mirai Labs出品，零延迟、全隐私、无推理成本。自带量化检查点（Mirai-M等），开箱即用。 关键使用步骤： 1️⃣最简安装：brew install mirai（或从http://trymirai.com下载App），直接
Posted by SuSu_酥酥👅 (24.5k followers) 2 days ago · 93 likes · 6k views · view the original post on X. Kept by the Dev Radar as AI dev tools. Tools mentioned: Mirai Labs, uzu.
More dev work like this
- 长PDF要是拆成单页再往一块拼，跨页表格和阅读顺序十有八九会乱套，这事你八成也烦过。 — @bkdgiffug
- Local models on Apple Silicon finally feel like a product, not a weekend project. — @PixelRainbowNFT
- GPT 6.1 Terra max vs GPT 6.1 Sol max — @HarshithLucky3
- Asked at the Meetup: "Does sbx run mistral work?" — @ajeetsraina
- Most agents freeze once deployed. — @GoogleCloudTech
- Your $250 GPU can now run a 35B coding model at 50 tokens/sec. — @0x0SojalSec
- Next week we're showing how to build self improving products. — @spenserskates
- Seedance 2.5 US Draft mode is now available on fal — @fal
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.2k posts from 5k X accounts over the last 21 days, 8.8k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-10-30 22:17 UTC. Full method.