A 7B model stored at 2 bytes per weight is about 14 GB.
This is a dev post classified by Jev as Hosting & infra (a free resource), kept by the Dev Radar because it carries real work, not commentary.
A 7B model stored at 2 bytes per weight is about 14 GB. Put it on an H100 with 80 GB of HBM and it fits easily. That tells you almost nothing about how fast inference will be. If one decode step had to stream all 14 GB of weights, and the GPU somehow sustained the H100’s advertised 3.35 TB/s entirely on those reads, the weight sweep alone would take about 4.18 ms. Roughly 239 tokens/s, before KV-cache traffic, activations, synchronization, or imperfect bandwidth use. It is only a sanity-check bound, but it changed how I think about "GPU memory." Capacity is only one number. The other quest
Posted by Tech with Mak (44.7k followers) 5 days ago · 100 likes · 4.4k views · view the original post on X. Kept by the Dev Radar as Hosting & infra.
More dev work like this
- Last chance: 4 days left to get your ticket to @WeAreDevs! — @Docker
- #MachineLearning with #AmazonSageMaker Cookbook! #BigData #Analytics #DataScience #AI… — @gp_pulipaka
- GSP644: Build a Serverless App with Cloud Run that Creats PDF Files 📄☁️ — @orbitofops
- One cluster. Multiple workloads. 🔥 — @k8sAMD
- i need a usa vpn like 3 times per year so paying for any vpn monthly/annually doesnt… — @thekitze
- The OpenInfra community in East Africa is expanding with the launch of the OpenInfra… — @openinfradev
- Kubernetes Pod Anti-Affinity for better replica distribution — @twtayaan
- Just 3 days to go! The stage is set for #ApsaraConference2026. — @alibaba_cloud
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 12.2k posts from 4.7k X accounts over the last 21 days, 1.4k tools, 12 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-19 18:06 UTC. Full method.