18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
This is a dev post classified by AGK Intelligence as AI dev tools (a free resource), kept by the Dev Radar because it carries real work, not commentary.
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over. And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams. So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit: - how many experts, how big, shared or not? - why is half my step spent in all-to-all, and how do I hide it? - why is my run 25x slower with no error in the logs? - softmax, sigmoid or sqrt(soft
Posted by Thomas Wolf (136.7k followers) 2 h ago · 505 likes · 16.2k views · view the original post on X. Kept by the Dev Radar as AI dev tools. Tools mentioned: moe-blogpost.
More dev work like this
- You can now build live apps in Cursor and share them with a link. — @cursor_ai
- 🚨 STOP INSTALLING MOVING TARGETS INTO HERMES AGENT. 🪽 — @tonysimons
- TensorFold 1.2.3 is out. 🚀 — @ashxhart
- Hello, XLA Serving Engine! 👋 — @Arm
- 这款翻译工具被不少人直接封神，如今总算开源了！ — @bkdgiffug
- NEAR co-founder Illia Polosukhin opens a public contest to improve AI models — @BSCNews
- Agentic AI For Platform Engineering: Rethinking Developer Platforms With Agentic AI,… — @gp_pulipaka
- Agentic Architectural Patterns for Building Multi-Agent Systems: Proven Design Patterns… — @gp_pulipaka
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.7k posts from 5k X accounts over the last 21 days, 9.4k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-05 22:02 UTC. Full method.