New from @Meta Engineering: FlashAttention-4 extended with MXFP8 support for @nvidia…
This is a dev post classified by Jev as AI dev tools (a launch), kept by the Dev Radar because it carries real work, not commentary.
New from @Meta Engineering: FlashAttention-4 extended with MXFP8 support for @nvidia Blackwell—from forward and backward kernels to fused quantization and jagged cross-attention. The team developed an end-to-end jagged module with fused quantization, FP8 activation and compute which is being used internally at Meta for GEM training. On the latest gen hardware, LP FA4 kernel reaches 2.85 PFLOP/s forward and 2 PFLOP/s backward, with up to a 1.30× end-to-end module speedup. ✍️ Devashish Shankar, Santosh Mohan, Jiaqi Xu, Darren Liu, Han Xu Explore the design and open source implementation in
Posted by PyTorch (512.7k followers) 2 days ago · 81 likes · 16.2k views · view the original post on X. Kept by the Dev Radar as AI dev tools.
More dev work like this
- Nimble is able to process images! — @madiator
- Your agent already solved this. Finding the session is the problem. — @DanKornas
- Trains your own large language model from scratch using plain PyTorch. — @tom_doerr
- gpt 5.6 terra = gpt 6 astra — @notjazii
- your rtx 3060 was running bonsai 2 at 26 tok/s this morning and now does 40 tok/s, i… — @sudoingX
- Turning a novel into a game is more than turning chapters into dialogue. — @DanKornas
- Meet DeepSeek-V4.1-Flash: a blazing fast image-text-to-text model. It takes both images… — @HuggingModels
- How to use Jev, and where it gives you the most advantage: — @0x_rody
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 12.2k posts from 4.7k X accounts over the last 21 days, 1.4k tools, 12 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-19 18:06 UTC. Full method.